You realise that the government can (and does) just make new money? There is no need for the government to borrow any money (but it does it anyways for other more complex reasons). And if done in moderation the resulting inflation is in effect identical to taxes, except that it automatically "taxes" the rich more and incurages further investments.

And with sufficient inflation, businesses will have no problem to attract investments as people will want to have investments that retain their value, no interest needed.

Similarly, investing money in houses is sound business when inflation is relatively high, both individually and for housing companies.

But the real problem is anyways not basic interest, but compound interest that forces the borrower to repay the lend amount many times over and still be in debt afterwards.