The problem I have with starting with lower scores is that it's almost always better to take a feat than a stat score increase in 5e, and even if you did take the stat increases, you wouldn't be able to hit 18 in your primary stat til level 8. Pair that with the fact that those increases come every four class levels, not player level, so multi-classing options are pretty limited if you don't want to delay or lose a feat/stat increase.

I also believe we play DND and other fantasy RPGs to feel cooler, smarter, stronger, or more charming than we are in real life. Most of us are average or slightly above average in real life, I don't want my players feeling average when they're role playing their character. Making things challenging by limiting the players abilities isn't fun. I like to let them be badasses, but challenge them to strategize an encounter, think creatively, or just do something really fucking cool to win a fight. Rule of Cool is always active in my games!