564
School shooting survivor sues AI gun detection firm after system failed to spot weapon
(arstechnica.com)
this post was submitted on 07 Jun 2026
564 points (99.3% liked)
Technology
85208 readers
5112 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
I am so tired of AI being shoved into everything and then people surprised when it doesn’t work. There’s no AI I think that could have detected a small firearm easily concealed. Hell as it is with legal concealed carry you can’t tell who is legally carrying as it is even with some of the most observant eyes watching.
People (and by this I mean the company) keep think that AI can give actual answers. It can't. It's a non-detrrminustic system, but they want it to behave deterministically. I'm sure the engineers gave the probability stats up to the business and marketing, who then immediately lowered their pants and shit on them, and then rolled it out as the perfect amazing product
The people who profit from this company don’t think that. They think that dumb school administrators think that, and will spend money on it.
The idea with these kinds of systems are meant to allow early warning when possible.
No system is going to be 100%.
The cheap system I have with a Google Coral and FOSS software is pretty good about detecting people and dogs from my camera streams. Sometimes it detects my one dog as a small bear if I haven't cut his hair recently.
Having such systems as a later if defense is good. As the only defense, not so much.
AI in this context is useless though, you could paint marker "not a gun" on the side of a gun and guess what would happen.
It has some uses, but 95% of what is being used for and 100% of the data centers aren't it.
It would flag it as a gun. How do I know? I worked on and developed a similar system at one point. It worked extremely well. We weren't an American company and ultimately covid killed us (it was US American orgs that were the most interested in our stuff).
Do you think LLMs are being used for this sort of thing? Putting aside the sheer technical mountain of a hurdle that slapping an LLM vision model on top of dozens and dozens of real-time camera streams, the hardware requirements would put the company out of business before they made their first sale.
Computer vision models, which are NOT LLMs, have been around for quite a while now and are very good at doing one thing and one thing only. And they'll do it well for a miniscule fraction of what it takes to run an LLM.
No, datacentres are not being used for real-time gun detection. The company might have other kinds of infrastructure located in a DC, but not the main video processing hardware.
Yes. It took all of five seconds to find out too.
You've already been wrong once, care to try for two?
Didn't I just say that slapping an LLM vision model on to dozens of camera streams would be a near impossible technical hurdle?
I never said vLLM models don't exist. I said they're impractical for this use case.
Haven't been wrong yet. You on the other hand...
There are several examples of exactly what I said, contradicting your repeated claim. Since I don't want to talk to someone with the conversational ability of Donald Trump demanding things be true in spite of evidence they're not im going to be blocking you now. Have a nice day.
Using a LLM for detecting a specific object on an image is possible but stupid: if your object is always the same (like in this case) it's several orders of magnitude cheaper to train once on that specific object then use the computer vision model running directly on the local server that's recording the video.
Otherwise:
Use cases for LLM-based image recognition is if the object changes at every request or it's ultra specific with brands and colors
It isn't the same though. A large gauge shotgun and a small gauge pistol are pretty different looking. Compare those to a .22 rifle with a scope, and those to a decked out ar15. That's a lot of different always the sames. What if it's a revolver? Or has a folded stock? Or a sawed off stock? Will it recognize a derringer or a mac10 with a large capacity mag as guns?
We can because they make us dead. We have valid reason to fear them which is a great motivator for most species to learn to recognize the danger. You'd still recognize a ring gun as a gun, without getting specifically trained to do so a machine will identify it as jewelry.
You seem to think that computer vision models can only be trained on a single thing. You simply train your modem on as many object types as you want it to be aware of. That's it.
so, train the computer vision model for a gun and train again for a shotgun. Run the two detection models at the same time.
Your approach is the typical "but if you really want you can use an atomic bomb to kill mosquitoes" - yes, you could do that, but nobody is paying $1 mil/year in inference costs (+some expensively licensed software to wrap around that) when it can be done locally with a $300 GPU (+ some expensively licensed software to wrap around that)
I gave a lot more than two examples and it was hardly exhaustive.