AI

6523 readers

10 users here now

Artificial intelligence (AI) is intelligence demonstrated by machines, unlike the natural intelligence displayed by humans and animals, which involves consciousness and emotionality. The distinction between the former and the latter categories is often revealed by the acronym chosen.

founded 5 years ago

155

Nobody puts Claude in a corner (lemmy.ml)

submitted 2 days ago by davel@lemmy.ml to c/artificial_intel@lemmy.ml

20 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] pixxelkick@lemmy.world 13 points 2 days ago (2 children)

This is pretty much entirely why I built https://github.com/SteffenBlake/cli-mcp-mapper

I map bespoke CLI commands, with parameters, to MCP tools, and then 100% just turn off the ability to invoke the CLI itself entirely.

That way it can only explicitly invoke the whitelisted commands I have granted it access to run, with only the parameters I have allowed it to.

So for example I can explicitly expose git commit, status, stage, unstage, but everything else git related is not available, not even git push or pull.

This substantially seems to help prevent the LLM from going off the rails on stupid CLI nonsense, like when it starts cating files instead of just fucking reading them.

[–] bloubz@lemmygrad.ml 4 points 2 days ago (2 children)

Did you write it or did Claude write it?

[–] mlatu@moist.catsweat.com 2 points 2 days ago

farted out in 19 commits, over 3 days using copilot... scraped thoughtlessly with love <3

[–] azimir@lemmy.ml 1 points 2 days ago (3 children)

Did Claude write the compiler or interpreter? Those can choose what instructions get run, regardless of your source code.

There's an old computer science discussion about trust and compilers. If I give you a compiler you can't trust that it won't insert a backdoor when you compile. So... You compile your own compiler from source.... But that compiler puts a backdooring code into your compiler.... What can you trust?

[–] bloubz@lemmygrad.ml 5 points 2 days ago

Well, we use CPUs that are not open source (I don't yet, at least)

[–] eleitl@lemmy.zip 1 points 2 days ago* (last edited 2 days ago)

You can bootstrap your compiler toolchain from known good sources, or even from scratch (bootstrap C via Forth via assembly or machine code).

[–] eldavi@lemmy.ml 0 points 2 days ago

There’s an old computer science discussion about trust and compilers. If I give you a compiler you can’t trust that it won’t insert a backdoor when you compile. So… You compile your own compiler from source… But that compiler puts a backdooring code into your compiler… What can you trust?

burn it all down and go back to using abacuses and fingers! lol

[–] TechLich@lemmy.world 1 points 2 days ago (1 children)

That's a good idea. Be careful of the more advanced models finding ways to abuse/circumvent it though, even with only a limited set of parameters. Eg. A lot of git commands can be tricked into dropping into less which can drop into a full shell.

They know all the gtfobins.org tricks.

[–] pixxelkick@lemmy.world 1 points 1 day ago

It should theoretically be impossible with the way you set it up.

You explicitly, opt in, expose which parameters are available to the LLM, it cant invoke any params beyond that.

And you can set params to hard values that arent exposed.

Finally, you can use the enum config to further specify a whitelist of valid values for a param, anything else gets rejected.

This pretty much locks most tools down tight.

Though, wouldn't hurt to add integration tests covering scenarios like this explicitly.