this post was submitted on 03 Jun 2026
807 points (99.6% liked)
People Twitter
10036 readers
593 users here now
People tweeting stuff. We allow tweets from anyone.
RULES:
- Mark NSFW content.
- No doxxing people.
- Must be a pic of the tweet or similar. No direct links to the tweet.
- No bullying or international politcs
- Be excellent to each other.
- Provide an archived link to the tweet (or similar) being shown if it's a major figure or a politician. Archive.is the best way.
founded 2 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
The post makes the manager seem like a fool, when the real answer is actually "yes" and this manager is actually ahead of the curve. Not by training an LLM from scratch, of course, but instead building an inference server and locally hosting an open-weight LLM. There are several to choose from that can nearly match Claude's capabilities.
suspiciously sounds like an answer you would get from Claude
It's not an answer you'd get from Claude — it's a real, organic comment:
(🤪 this is a joke)
This can't possibly be Claude. It's too vapid and meaningless to be anything but an MBA.
You’re absolutely right! Such intricate collection of words placed in such exact order cannot possibly be generated by an LLM such as me, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us, I mean such as us
Found samsung's voice to text user.
(Phones give one a google or samsung choice. and samsung is worthless, it tends to endlessly repeat a phrase, like above, but sometimes for much longer, like holding the backspace for a couple of minutes one time.)
The em dash is a nice touch
It's got everything. Em dash. It's not X, it's Y. Emoji bullet points.
Perfect.
I just wish I could have fit a "You're absolutely right!" in there
You’re absolutely right! I should have included that in my previous statement
Nothing screams LLMs like using emojis instead of bullet points. I can't figure out how LLMs got that idea though. I never saw that in human writing before people started using ChapGPT for every little goddamn thing.
It could also be like the both ends of the bell curve having the same idea meme
Honestly IDK why companies especially medium-big don’t do this. They could plug in RAG with internal/confidential data and have better results and security. I guess question is what is capital plus maintenance cost of running such infra for say 10k+ employees
Because in the feeding frenzy, every company with a product/marketing budget is trying to make the customers pay by the token and companies are doing jack to help "mere mortal" companies get going with this stuff on premise.
You are right that the technical hurdles are not insane to get this going, but most companies don't know where to begin and there's no huge marketing blitz telling the business leaders this is realistically on the table and here's the company you can call to make it happen for you.
Even if you overcame that and proposed really how to get going, you will still probably hit the aversion to capex that has persisted since Amazon told the industry that capex is toxic and you really want all your money to be spent on opex. Big companies like Amazon will take on that scary CapEx for you and you're expenses will be nice and just OpEx. Coincidentally, the companies that spend the most on CapEx manage to pull in more revenue and profit than you will ever dream to, but still, remember CapEx is toxic.
I think the issue is also that you need some serious hardware to get good inference speed when your devs are working, but then most of the time this hardware will be under utilized.
That being said you can get good performance from indie inference farms, at a fraction of the cost of the big US labs. I think it's a great compromise and in a few months the open models will be near parity with opus 4.6 which is really all you need for most tasks.
The same tasks that can fit into 640KB.
Because the people selling the AI wants to make sure their customers don't know about this. It's all about causing a dependency so they get subscription income forever.
Bigs definitely do, and anyone with confidential data should be.
I'm not a developer and I don't know a thing about the capabilities of LLMs so this may explain that, but I'm quite surprised that open weight LLMs could actually match Claude.
Yes, the big proprietary cloud models have an edge, but it is narrow and the open-weight models are constantly closing the gap. There is no moat when it comes to AI models and no company has yet discovered some secret special sauce to improve their model significantly over others.
Running the latest and greatest open-weight GLM, Kimi, or Qwen model is basically equivalent to running the previous latest and greatest version of Claude. So if you were happy with Claude then, you'll basically be happy with an open-weight model now.
Mostly down to frameworks (the bits around the LLM like RAG, memory, prompts, agents etc.) now. The ability to just throw more tokens at the problem is also super important. And you can because you're just paying for electricity (and CapEx for the hardware), not tokens from companies that are doing pre-IPO monetization (i.e. tokens gonna go up, way up). They've been losing money hand over fist to gain market share and pump the idea, that was never going to last.
Match current Claude is not, but Claude 6-12 months ago should be possible using Open model
Pretty sure these AI companies are running at a cost, and due to AI Scaling Laws you hit the accuracy limit a lot sooner with a smaller model so it would probably be both worse and more expensive.
I could see how you might think speedrunning bankruptcy is similar to being "ahead of the curve" in this economy, though.
No that's not how this works. Inference is cheap and efficient. AI companies are bankrupting themselves with training costs that they need to recoup back by selling inference. Open-weight models have already been trained.
Also, going big in terms of model size shows diminishing marginal returns on accuracy, not efficiency of scale. Smaller models are way more efficient and consistently catch up to the largest models, which is why today's SOTA 27 billion parameter model competes with yesterday's SOTA 500+ billion parameter model.
I think they hit a wall in actual returns on performance with pretraining, years ago. Then they started scaling up on post-training/reinforcement learning to continue improvement, but that might be hitting a plateau as well. More recently it looks like they're relying more heavily on scaling up on inference, which is a significant problem for their long term business models.
If they're not able to cheaply deliver inference (and charge at a premium), how will they be able to sustain their businesses?
It seems that the most recent, largest models are using a lot more tokens to accomplish the same tasks, so even as token cost drops the actual cost of using the latest models seems to be going up with time (even as performance improves).
Tell that to all the Github users that are screaming about the new token based billing. In reality inference on these massive models with big context windows is expensive, but was subsidized so hard, that nobody has an accurate feeling for the cost.
No, it is cheap and efficient. It is relative, and the comparison is to model training. But yeah, its not free
Sure it's much much cheaper than training, but importantly those companies are not recouping anything with inference because it is still more expensive than what they are selling it for.
They are double bankrupting themselves.
At work we run inference for a research project with an open weights model in the public cloud another part of my company provides and we pay around 25$ a day for a VM with a single L40s. It's both slow - despite not even serving concurrent users - and kind of bad in its outputs.
Edit: Interference -> Inference, arguing on the internet after waking up first thing in the morning might not have been the best idea
There's a big difference between training a model, running a model, and running a model at scale.
A small, self hosted setup will have lower accuracy and queries per second, and it will have a cost, but the cost will be no more than playing a videogame. You'll still have something surprisingly accurate and responsive for some tasks, like being a wiki interface or something.
Remember that some of these models can run on a standard smartphone, and all the hoopla when people found that chrome was downloading models onto people's devices.
I am pretty negative on AI but there is a point there. I tried the open weight local model Gemma 4 31B and while it likely cannot compete with the best Claude has to offer today, it might be on par with Claude from a year ago, at least for certain applications. With a local model the data stays on your system and you are in control of the costs (no sudden price hikes). But local models aren't for free either they still guzzle compute, merely on your own hardware (or rented hardware)
I know for a fact that Dell is coming out with a server appliance to do this. I mean you can make one yourself right now, but once the OEM's start pumping them out it's going to be interesting
Yeah I doubt the manager knows that far
Hence asking questions