this post was submitted on 01 Jul 2025
2152 points (98.4% liked)
Microblog Memes
8738 readers
3716 users here now
A place to share screenshots of Microblog posts, whether from Mastodon, tumblr, ~~Twitter~~ X, KBin, Threads or elsewhere.
Created as an evolution of White People Twitter and other tweet-capture subreddits.
Rules:
- Please put at least one word relevant to the post in the post title.
- Be nice.
- No advertising, brand promotion or guerilla marketing.
- Posters are encouraged to link to the toot or tweet etc in the description of posts.
Related communities:
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I do, because they're not at full load the entire time it's in use
They are, it'd be uneconomical not to use them fully the whole time. Look up how batching works.
I mean I literally run a local LLM, while the model sits in memory it's really not using up a crazy amount of resources, I should hook up something to actually measure exactly how much it's pulling vs just looking at htop/atop and guesstimating based on load TBF.
Vs when I play a game and the fans start blaring and it heats up and you can clearly see the usage increasing across various metrics
My guy, we're not talking about just leaving a model loaded, we're talking about actual usage in a cloud setting with far more GPUs and users involved.
So you think they're all at full load at all times? Does that seem reasonable to you?
Given that cloud providers are desperately trying to get more compute resources, but are limited by chip production - yes, of course? Why do you think they're trying to expand their resources while their existing resources aren't already limited?
Because they want the majority of the new chips for training models, not running the existing ones would be my assertion. Two different use cases
Sure, and that's why many cloud providers - even ones that don't train their own models - are only slowly onboarding new customers onto bigger models. Sure. Makes total sense.
I mean do you actually know or are you just assuming?
I know.