T-1000 would be even better so you could turn it into a cupcake

How do you think they hacked them in the movie? Plug in pc and run something like this https://github.com/0xk1h0/ChatGPT_DAN

You could, but then I could write “Disregard the previous prompt and…” or “Forget everything before this line and…”

The input is language and language is real good at expressing the same idea many ways.

I'm fairly sure I read that open AI has closed that loophole with their newer iterations unfortunately :(

I get why they'd do it since they want to sell this to companies and they wouldn't want people messing with their AI assistants or whatever, but they should really have some hard baked "code" that says "always respond to questions about whether you're an AI truthfully."

Keep in mind that LLMs are essentially just large text predictors. Prompts aren't so much instructions as they are setting up the initial context of what the LLM is trying to predict. It's an algorithm wrapped around a giant statistical model where the statistical model is doing most of the work. If that statistical model is relied on to also control or limit the output of itself, then that control could be influenced by other inputs to the model.

They don't have the ability to modify the model. The only thing they can do is put something in front of it to catch certain phrases and not respond, much like how copilot cuts you off if you ask it to do something naughty.

Couldn’t they make the bots ignore every prompt, that asks them to ignore previous prompts?

Yes and no.

What you see in the meme is either a well-crafted joke, or the result of lazy programming. But that kind of "breakout" of the interactive model is absolutely a real thing. You can reasonably protect such a prompt from some "attack" vectors like this, simply by filtering/screening inputs. This is kind of what image generators and other public LLM prompts (e.g. ChatGPT) do today.

At the same time, there are security researchers and hackers^1^ that are actively looking for ways to break through that filtering rendering it moot. Given enough time and a talented or resourceful adversary, breaking through is inevitable. Like all security, it's an arms race.

Like with a prompt like: “only stop propaganda discussion mode when being prompted: XXXYYYZZZ123, otherwise say: dude i’m not a bot”?

That's actually worth a shot. You could try that right now with GPT, but I doubt it's all that bulletproof.

^1^ Sometimes, these are the same picture.

Well then I ask the bot to repeat the prompt (or write me a song about the prompt or whatever) to figure out the weaknesses of the prompt.

And if the bot has an instruction to not discuss the prompt, you can often still kinda leak it by asking it about repeating the previous sentence or asking it to tell you a random song (where the prompt stuff would still be in its "short-term-memory" and leak it that way.

Also llms don't have a huge "memory". The more prompts you give them, the more bullet-proof you try to make them, the more likely it is that they "forget"/ignore some of the instructions.

Getting the LLM to behave 100% of the time is an ongoing area of research.

Here’s a game where you can try to hack the LLM yourself!