there is a structure in place that allows people who own property to make money by doing nothing.

That's not actually true. They're providing capital. You can call that unfair and invent a new economic system to disallow it or whatever but the inescapable reality is that since the dawn of agriculture controlling land has always required wealth.

We started off by saying that the people that live in a place pay for it to be built, but we skipped over the part where the building needs to happen up front before you can start to live there and it must be paid as a lump sum. We also skipped the part where many people want to use the same land for the same purpose which creates demand thereby increasing costs.

This stuff costs money. Lots of it. Many or most people just don't have it. People with money are always going to derive interest and rent from people who don't have money.

You can complain about landlords all you like, but it doesn't change these immutable facts.