I think a house isn't the best comparison here, as a house isn't a public space, whereas the Fediverse is. A better comparison might be a town square or a park. Anyone is welcome to be there, but if they do something bad, or it becomes obvious that they are going to do something bad, then they can be removed from that space. Otherwise they should be allowed to exist in that space.