Sorry to repost, but coincidentally I made a comment a few days ago answering this:

First past the post elections is the answer. If we had ranked choice or runoff elections, more parties would appear.

Instead, in FPTP, every vote that is not for one of the two highest-polling candidates is objectively a wasted vote. Game theory dictates that the only rational choice is a vote for one of those two candidates, since the possibility of a third party gaining enough votes to win in any single election is nearly infinitesimal. So instead of many parties, all candidates self-sort into one of the two viable parties. Any candidate that does not is a protest candidate or deluded, but in either case, there is no hope of actually winning.

So what about primaries? The primary system decides the candidates, but even that is tainted by FPTP, because primary voters have to guess which will perform better in a FPTP general election and often vote against their ideal candidate in the hopes of winning (or, not losing) the general.

In short, until we structurally reform elections to be ranked/STAR/runoff/etc to remove the punitive effect of voting for your actual ideal candidate, we're stuck with a prisoner's dilemma election every time.