So buying a handful of massive companies is just an inefficient way to buy the S&P, which is just an inefficient way of buying the whole market.

Mostly, sure. But, the difference is that the index contains some of the smaller companies too. If there's something that hits all the big companies at the same time (like robust anti-trust enforcement actions), the mid-sized companies are the ones who are most likely to benefit because they can take advantage of the pressure on the big companies.

If you just invested in the big companies you could be hurt, but theoretically if you invest in the whole market (or an index representing it) you could have the losses in the big tech stocks offset by growth in the mid-sized ones.

Anyhow, I think we mostly agree. The market is manipulated, many traders, even pros, are irrational. So, even if you know what you're doing, it's smart to hedge and spread your risk around.