Privacy

46564 readers

701 users here now

A place to discuss privacy and freedom in the digital world.

Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.

In this community everyone is welcome to post links and discuss topics related to privacy.

Some Rules

Posting a link to a website containing tracking isn't great, if contents of the website are behind a paywall maybe copy them into the post
Don't promote proprietary software
Try to keep things on topic
If you have a question, please try searching for previous discussions, maybe it has already been answered
Reposts are fine, but should have at least a couple of weeks in between so that the post can reach a new audience
Be nice :)

Related communities

much thanks to @gary_host_laptop for the logo design :)

founded 6 years ago

MODERATORS

I don't understand how Moxie Marlinspike's Confer "Private LLM" works (piefed.social)

submitted 1 month ago* (last edited 1 month ago) by artifex@piefed.social to c/privacy@lemmy.ml

25 comments fedilink hide all child comments

So I was reading this article about Signal-creator Moxie Marlinspike's new project, Confer , which claims to be a verifiably E2E encrypted LLM chat service. There are a couple of short blog articles that give the gist of it, and some github repos including this one that includes scripts for producing the VM that will run your particular LLM session. But if I'm following this all correctly, it implies that every chat session (or perhaps every logged-in user) would have their own VM running their own LLM to ensure that the chain of trust is complete. This seems impossible from a scalability perspective, as even small LLMs require huge quantities of RAM and compute. Did I miss something fundamental here?

you are viewing a single comment's thread
view the rest of the comments

[–] Dave@lemmy.nz 26 points 1 month ago (2 children)

The blog article you link I think implies you do not have your own VM. LLMs are stateless, the previous conversation is fed in as part of the prompt.

You send your message, which is E2E encrypted. The LLM runs in an environment where it can decrypt your message and run in through the LLM, then send a response to you. Then it gets the next user's message and replies to them.

The key part is that the LLM is running inside an encrypted environment not accessible to the host system, so no one can watch as it decrypts your message.

That's what I get from reading your links.

[–] artifex@piefed.social 8 points 1 month ago

Ok, I interpreted it to mean that the VMs were being created as-needed and was keyed to your key specifically (which would be the most secure scenario, I think) and couldn't figure out what that could possibly work economically. But it makes more sense if just a separately encrypted host is decrypting your request and encrypting your reply along with everyone else's.