rook

joined 2 years ago
[–] rook@awful.systems 1 points 2 days ago

Here’s one from the archives: https://spectrum.ieee.org/children-beating-up-robot

Children Beating Up Robot Inspires New Escape Maneuver System

tl;dr: small children have no empathy for your robot and will torture it and hound it to death if they can. The safest thing to do is to head for the nearest adult-sized people, who will hopefully be less inclined to kill it simply because it is a robot.

[–] rook@awful.systems 5 points 3 days ago (2 children)

I wouldn’t want to do that myself… personally too much tracking gear in there, and it’s easy to make a mistake and not disable it all. Also, you just know that if you get caught, they’ll try and prosecute it like you kidnapped and dismembered a regular officer.

Now, I’m more surprised that they don’t get black bagged and tipped over. Maybe they only ever use them in super thoroughly surveilled areas with nearby human backup, but you’d expect at least one successful tipping to make the news somewhere.

[–] rook@awful.systems 12 points 4 days ago (5 children)

How it started: in 2025, the city of dublin, ohio (the latter detail missed by quite a lot of reporting,because there are no other dublins it might get confused with, I guess) gets an autonomous? ai powered police surveillance robot.

City officials are encouraging residents to interact with Dubbot—ask questions, take selfies, and experience firsthand how AI is shaping public safety. The goal is to foster transparency and gather feedback to refine the robot’s role in the community.

How it’s going

The person-sized, camera-covered robot that looked like it rolled right out of a sci-fi movie did not identify any criminal incidents, issue any tickets or help with any arrests in its nearly 10 months on the job.

On the other hand, I bet it didn’t shoot anyone’s dog, so who’s to say that the $64k was wasted.

[–] rook@awful.systems 6 points 4 days ago (1 children)

Anyone who names a security company “sauron” has critically failed their reading comprehension. Mercifully, there’s no company naming itself saruman, because that character was breathtakingly dumb in so many ways that perhaps even the y-combinator set are dimly aware of them.

[–] rook@awful.systems 10 points 4 days ago (5 children)

Who even has time for that? Do you think that the people behind palantir, icarus and sauron have time to read google summaries? They’re too busy remaking the world!

Anyway, if you’re successful enough you’ll eclipse the original source in terms of importance and all the search engine summaries will be about you anyway, so any time spent learning anything before that will have been completely wasted.

 

In an idle moment, I thought I’d explore the space of ridiculously bad ai company names. Literally the very first dystopia I thought of already has three ai companies named after it, and it hardly seemed worth exploring any further.

Because no one has got around to repealing poe’s law, I cannot tell if these are a bunch of idiot techbros, or people taking the piss out of idiot techbros, so I leave you to judge for yourself. Behold, people who think that “we tortured a child to bring you glossy web UIs” is a great corporate image:

  1. Omelas AI

AI-driven software development. Enterprise platforms delivered at startup speed.

I think they’re a consultancy? “One developer with AI produces what a 30-person agency does. 10+ production platforms in under two years.

  1. Omelas IO

Omelas is the maker of Atreus, the leading AI research companion for foreign policy, national security, and geopolitical risk. Atreus has access to the Omelas database, multidomain intelligence, and unique research methods, yielding unparalleled insights in minutes.

Atreus is the AI workbench purpose-built for intelligence work, fusing unique feeds, open-source intelligence, commercial data, satellite imagery and telemetry data into reports your analysts can act on immediately” which I guess means that they’re palantir wannabes, with the USP that they’ve grossly misunderstood le guin instead of Tolkien.

  1. omelas.tech

Omelas builds software across privacy, social connection, developer tools, and AI — designed and engineered in the Netherlands.

Another consultancy. They claim they make “thoughtful products”, hopefully with more thought than they put into their branding. Proof that inability to understand fantasy and sci-fi isn’t limited to silicon valley, or native english speakers.

[–] rook@awful.systems 9 points 6 days ago (1 children)

Some folks, who may be familiar to some or more of you, accidentally discovered that if your git repo symlinks CLAUDE.MD to, say, /dev/urandom, it breaks Claude code.

the reason why this works is exactly the reason why claude code sucks so bad. there are protections against this in the file reading tool. however because everything in claude code is implemented in 5 million different ways, those protections are a completely orthogonal set of codepaths from how CLAUDE.md files are read. conversely, the file read tool seems to be completely naive to symlinks while the CLAUDE.md reader is not. this is the fucking swiss cheese security model of the fucking gold standard of what AI programming can do.

https://neuromatch.social/@jonny/116779793188712173

The thread is actually about trying to attract and manipulate autonomous coding agents, but they’ve only had limited success so far, which may have been slowed down by the above symlink trick.

[–] rook@awful.systems 6 points 1 week ago

I think part of the issue is that historical software quality was an artefact of its time… if you can’t easily patch your released products, you need to work harder to ensure they’re functional. If the only way for people to learn about how your product works in the documentation you ship with it, the docs need to be useful and comprehensive.

The combination of software needing no guarantee of merchantability or fitness for any particular purpose and the internet rendered those pressures obsolete. Ship shit, fix later. Mass-scale a/b testing over past decade or two shows that most people seemingly don’t care if their software runs like absolute garbage, and is covered in adverts, and harvests all their personal data and the leaks all of it that wasn’t sold.

An incident-to-pr ratio that’s up by 250% is unfortunate, but it is not yet so bad that the end-users actually care enough to do anything about it, even assuming they can do anything.

[–] rook@awful.systems 9 points 1 week ago (2 children)

This is by an llm-boosting firm, so be aware that it’ll have a lot of marketing in it. It doesn’t say nice things about vibe code (presumably because the authors want to sell you a solution) but the numbers are interesting even so.

https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways

A few choice snippets, none of which will surprise anyone here:

  1. For every code change merged, the probability of a production incident has more than tripled.

The incidents-to-PR ratio is up 242.7% as teams move from low to high AI adoption.

  1. Bugs are accelerating, not stabilizing.

In our 2025 AI engineering report on the AI Productivity Paradox, bugs per developer were up 9% as AI adoption grew. In this dataset, that figure has risen to 54%

  1. The most experienced people in your organization are being buried.

Median time to first PR review is up 156.6%. Average time spent in code review is up 199.6%. Median time in review is up 441.5%. The engineers with the deepest knowledge of the system are spending their most valuable hours unraveling plausible-looking code that should never have reached them in the state it did.

[–] rook@awful.systems 5 points 3 weeks ago (4 children)

This seems like it is probably a good thing.

https://leidendeclaration.ai/

It does feel a bit “art of war” though… someone patiently explaining to a bunch of people who really should know better that they shouldn’t do obviously bad and wrong things.

[–] rook@awful.systems 11 points 4 weeks ago (7 children)

It’s probably a coincidence, but there have been a whole bunch of minor regression bugs in recent point releases of rsync, and also there are a whole bunch of commits from “tridge and claude”.

[–] rook@awful.systems 11 points 4 weeks ago (6 children)

because there’s no economic incentive to hire them to do that kind of work.

isn’t that the old “basic science is boring and unsexy” issue though? There are economic incentives, but not in a short term-big-bux sort of way, so capitalism can’t be trusted with it.

To conjure up a recent example, something like “The number of curves of genus two with elliptic differentials”, published back in 1997, probably had limited commercial value at the time, but 20 years later completely sunk a promising post-quantum cryptography algorithm (“An efficient key recovery attack on SIDH”) which might have had some non-trivial commercial implications if SIKE had got through the key exchange algorithm competition.

Anyway, the Erdős problems are good candidates for llm work because they have been specified in a careful and formal way, which requires a reasonably competent mathematician to do. That then opens up mathematics to the same deskilling problem that other sectors afflicted with llms have, and because capitalism is shortsighted and stupid we don’t know what the future economic impact of that will be, right?

[–] rook@awful.systems 12 points 4 weeks ago (5 children)

In the same way that lazy studios need to produce a film for each element of the powerset of character IPs they own, I guess we were overdue a Rationalist x Pickup Artist episode. I’m slightly surprised the whole “model women as quasi-sentient deterministic sex machinery” idea wasn’t already very popular there, but maybe I’ve just missed that part of their culture.

view more: next ›