LocalLLaMA

3096 readers

47 users here now

Welcome to LocalLLaMA! Here we discuss running and developing machine learning models at home. Lets explore cutting edge open source neural network technology together.

Get support from the community! Ask questions, share prompts, discuss benchmarks, get hyped at the latest and greatest model releases! Enjoy talking about our awesome hobby.

As ambassadors of the self-hosting machine learning community, we strive to support each other and share our enthusiasm in a positive constructive way.

founded 2 years ago

MODERATORS

SkySyrup@sh.itjust.works

pax@sh.itjust.works

noneabove1182@sh.itjust.works

Smokeydope@lemmy.world

MonsterBug@sh.itjust.works

AMD’s Untether AI Deal - Bad Signs for GPU-Driven AI training (semiconductorsinsight.com)

submitted 23 hours ago by Eyekaytee@aussie.zone to c/localllama@sh.itjust.works

13 comments fedilink hide all child comments

It looks like AI has followed Crypto chip wise in going CPU > GPU > ASIC

GPUs, while dominant in training large models, are often too power-hungry and costly for efficient inference at scale. This is opening new opportunities for specialized inference hardware, a market where startups like Untether AI were early pioneers.

In April, then-CEO Chris Walker had highlighted rising demand for Untether’s chips as enterprises sought alternatives to high-power GPUs. “There’s a strong appetite for processors that don’t consume as much energy as Nvidia’s energy-hungry GPUs that are pushing racks to 120 kilowatts,” Walker told CRN. Walker left Untether AI in May.

Hopefully the training part of AI goes to ASIC's to reduce costs and energy use but GPU's continue to improve inference and increase VRAM sizes to the point that AI requires nothing special to run it locally

you are viewing a single comment's thread
view the rest of the comments

[–] artifex@lemmy.zip -1 points 16 hours ago (4 children)

It’s like bitcoin mining all over again. And providing roughly an equal amount of value.

[–] Eyekaytee@aussie.zone 6 points 16 hours ago* (last edited 16 hours ago) (3 children)

If you think AI provides the same value as bitcoin you are brain dead

[–] CancerMancer@sh.itjust.works 0 points 6 hours ago (2 children)

Consider the perspective of the average person: who is gaining value from AI in its current form? It sure as hell isn't us.

[–] brucethemoose@lemmy.world 2 points 5 hours ago* (last edited 5 hours ago)

It could be if it’s run locally.

If you agents run on your hardware, navigate crappy apps and websites and such for you, what do you need the corporate cloud for? How can they show ads or monetize you through that?

That’s the war raging right now, open-weights vs closed weights.

[–] Eyekaytee@aussie.zone 1 points 5 hours ago* (last edited 5 hours ago)

Consider the perspective of the average person: who is gaining value from AI in its current form? It sure as hell isn’t us.

Ah yeah, the millions of people using it to generate images, generate summaries, translate documents, clean up photos etcetcetc are getting no value from it at all 🙄

Perplexity received 780 million queries in May, CEO Aravind Srinivas shared onstage at Bloomberg’s Tech Summit on Thursday. Srinivas said that the AI search engine is seeing more than 20% growth month-over-month.

edit: why am I having this discussion, if you don't like AI feel free to go literally anywhere else on Lemmy