They are being written to be more efficient in inference, but the gains are being offset by trying to wring more capabilities out of the models by ballooning token use.

Which is indeed a form of Jevon's paradox

[–] errer@lemmy.world 32 points 2 days ago (1 children)

Costs have been dropping by a factor of 3 per year, but token use increased 40x over the same period. So while the efficiency is contributing a bit to the use, the use is exploding even faster.

[–] VibeSurgeon@piefed.social 19 points 2 days ago (1 children)

I think we're meaning the same thing.

[–] Bronzebeard@lemmy.zip 4 points 2 days ago (1 children)

Yes, but have you considered if I just rephrase what you just said but from a slightly different perspective?

[–] felbane@lemmy.world 1 points 1 day ago

"Haeeves?? Ohh, you were saying what I was saying in a different way!"