It's a bit more involved than that.

The part where BIOT is located is geographically close to Mauritius, which used to be part of the Empire. When decolonization happened, part of the principle was that the entire constituent part was to be decolonized. But at the time, the Vietnam war was happening, the US wanted the UK to contribute, the UK did not want to send armed forces, so the compromised by offering Diego Garcia as an airbase. But to do that, they needed to keep control over it, so they simply ignored the principle of keeping the decolonized territory intact, kept the Chagos, kicked out everyone who lived there, and there we are.