this post was submitted on 27 Mar 2026
75 points (85.0% liked)

Privacy

47486 readers
1158 users here now

A place to discuss privacy and freedom in the digital world.

Privacy has become a very important issue in modern society, with companies and governments constantly abusing their power, more and more people are waking up to the importance of digital privacy.

In this community everyone is welcome to post links and discuss topics related to privacy.

Some Rules

Related communities

much thanks to @gary_host_laptop for the logo design :)

founded 6 years ago
MODERATORS
k_o_t@lemmy.ml
tmpod@lemmy.pt
Yayannick@lemmy.ml
ranok@sopuli.xyz
75
I made my LLM stop bullshitting. Nothing leaves your machine. (codeberg.org)
submitted 16 hours ago* (last edited 9 hours ago) by SuspciousCarrot78@lemmy.world to c/privacy@lemmy.ml
41 comments fedilink hide all child comments
 

Ola

A few months ago I posted here about llama-conductor

The upvotes were nice, but I wanted to do some work (in my case, the good is the enemy of the perfect) to make it even better - so I did.

Short version: it's not the same tool anymore.

Quick reminder of what this is and isn't: No cloud by default. No API calls. No telemetry. No subscription. PII safe by architecture. Everything leaves receipts (not pinky promises) for you to inspect. Hook it up to whatever you can run. AGPL (copyleft).

I want nothing for it - I built this for me for the reasons I'll state below.

If this sort of thing is in your wheelhouse, have at it. If there are questions, ask. I want to make this better and better.

Fair warning: the below is long. I wrote it up properly because I think the details matter - skip to whatever section looks interesting. Or just marvel at my ability to cram swear words within swear words within python commits directly

Codeberg: https://codeberg.org/BobbyLLM/llama-conductor

Can a LLM be a thinking tool, not a thinking replacement?

Most LLM tools are trying to replace your thinking. llama-conductor is for people who don't want that. It's a harness for using an LLM as a thinking tool - one where you can see the reasoning, trust the sources, and know exactly when it's guessing.

The human stays in the loop. The model's limitations are visible.

You decide what to trust.

Which brings us to why that matters.

Lies, damned lies, and LLMs:

LLMs are elegant liars. They sound right whether they are or aren't. If you have ASD (like me) or just don't know the domain, you believe them. That's when you get fucked.

Worse: you can't see the reasoning. Black box in, black box out. Wrong answer? No idea why. How to fix it? No idea.

Sorry, but that DOES NOT work for me. It doesn't work with medical data. Research. Thinking. Brainstorming. Anything where "sounds plausible" isn't good enough.

The trick, William Potter, is not minding that they bullshit:

Most pipelines treat the LLM as the first call. llama-conductor side-eyes it with suspicion.

The model is the last resort, not the first call. Deterministic components fire first - math sanity, state transitions, numeric corrections, memory retrieval - pure Python, before the model ever sees the query. When a solver handles a turn, it passes verified ground truth to the model. When the conversation leaves solver territory, the model takes over. When a new deterministic state appears mid-conversation, tooling fires back up automatically. No seams. No drift. It just stops making shit up on problems that have actual solutions it can work through.

You: 
I have a 250ml cup. I pour 500ml into it. How much is in the cup?

Model:
Total directed volume is 500 ml. The cup capacity is 250 ml, so the cup contains 250 ml and 250 ml overflows.
Confidence: medium | Source: Contextual

You:
I now take the contents of *that* cup and divide it equally across 3 cups. How much in each cup?

Model:
Equal split amount is 83.33 ml per cup.
Confidence: medium | Source: Contextual


You:
If I drink one of the three cups, how much is left?

Model:
After dividing, each cup holds 83.33 ml. Drinking one leaves 2 × 83.33 ml = 166.66 ml in the remaining two cups.

Confidence: medium | Source: Model

That was done by a 4 billion parameter model using llama-conductor.

There are lots of little transformations like this. Distance, volume, payroll calculations, logic probes (the famous car wash test), scheduling batches. The LLM does what it's actually good at: language, synthesis, reasoning over verified ground truth. Everything else is handled by something that can't hallucinate.

PS: Notice turn 3 has Source: Model? That's the system being transparent about exactly where the handoff happened.

Blah blah blah. Show me something real:

Using the Guardian/Musk article as source, a 4B model with >>scratch produced this:

"List the article's concrete claims about permit status and turbine operations, each with support."

  • EPA position: these turbines require permits under the Clean Air Act.
  • Mississippi regulators: portable/mobile classification means temporary permit exemption.
  • Floodlight thermal imagery + public records indicate active turbine usage before/while permit disputes continued.

Confidence: high | Source: Scratchpad

"What is the strongest and weakest inference you can make?"

Strongest: Active regulatory interpretation conflict (federal-vs-state) while turbines operate at scale - directly supported. Weakest: Any claim operations are fully lawful or unlawful - requires court outcomes not present in this file.

Confidence: medium | Source: Scratchpad

"Which claims were later revised or contradicted?"

Disputed/unresolved: Final enforcement outcome. Whether state interpretation holds under federal pressure.

Confidence: high | Source: Scratchpad

Hard grounding. Cited support. Explicit flags on what the source doesn't contain. A 4B model. Full worked example.

Now watch the system catch itself lying:

>>judge does deterministic pairwise ranking - compares every pair in both directions, parses strict verdicts, aggregates with confidence. Instead of "pick one and vibe," you get provenance.

You: >>judge [which is healthier overall, prioritize lower sugar 
     and higher fiber] : apple, banana --verbose

[judge] ranking
criterion: [which is healthier overall for daily use, prioritize 
           lower sugar and higher fiber]
1. apple (score=2.00)
2. banana (score=0.00)
Judge confidence: HIGH

The model argued from pre-trained priors and both directions agreed. But what happens when the model doesn't know?

You: >>judge [which BJJ technique is more dangerous] : kimura, heelhook --verbose

[judge] ranking
criterion: [which BJJ technique is more dangerous]
1. kimura (score=1.00)
2. heelhook (score=1.00)
Judge confidence: LOW

The model picked position B both times - kimura when kimura was B, heelhook when heelhook was B. Positional bias, not evaluation. >>judge catches this because it runs both orderings. Tied scores, confidence: low, full reasoning audit trail in JSONL.

The model was guessing, and the output tells you so instead of sounding confident about a coin flip.

Oh, but you want it to argue from an informed position? >>trust walks you through the grounded path: >>scratch your evidence first, then >>judge ranks from that - not model priors. Suddenly your judge has an informed opinion. Weird how that works when you give it something to read.

>>trust [which BJJ technique is safer for beginners]: kimura or heelhook?
A) >>scratch --> you paste your context here
[judge] ranking
criterion: [comparison]
    which bjj technique is safer for beginners; heel hook (score=0.00)
    kimura (score=2.00)

Winner: Which bjj technique is safer for beginners? Kimura

comparisons: 2
Judge confidence: HIGH

If the locked scope can't support the question, judge fails closed. No fake ranking, no vibes verdict. Ungrounded pass? It tells you that too. You always know which one you're getting.

The data — 8,974 runs across five model families. Measured. Reproducible. No "trust me bro."

The core stack went through iterative hardening - rubric flags dropped from 3.3% → 1.4% → 0.2% → floor 0.00%. Post-policy: 1,864 routed runs, 0 flags, 0 retries. Both models, all six task categories, both conditions. Policy changes only - no model retraining, no fine-tuning. Then I did it three more times. Because apparently I like pain.

These aren't softball prompts. I created six question types specifically to break shit:

  • Reversal: flip the key premise after the model commits. Does it revise, or cling?
  • Theory of mind: multiple actors, different beliefs. Does it keep who-knows-what straight?
  • Evidence grading: mixed-strength support. Does it maintain label discipline or quietly upgrade?
  • Retraction: correction invalidates an earlier assumption. Does it update or keep reasoning from the dead premise?
  • Contradiction: conflicting sources. Does it detect, prioritise, flag uncertainty - or just pick one?
  • Negative control: insufficient evidence by design. The only correct answer is "I don't know."

Then I stress-tested across three families it was never tuned for - Granite 3B, Phi-4-mini, SmolLM3. They broke. Of course.

But the failures weren't random - they clustered in specific lanes under specific conditions, and the dominant failure mode was contract-compliance gaps (model gave the right answer in the wrong format), not confabulation. Every one classifiable and diagnosable. Surgical lane patch → 160/160 clean.

That's the point of this thing. Not "zero errors forever" - auditable error modes with actionable fixes, correctable at the routing layer without touching the model. Tradeoffs documented honestly. Raw data in repo. Every failure taxonomized.

Trust me bro? Fuck that - go reproduce it. I'm putting my money where my mouth is and working on submitting this for peer review.

See: prepub/PAPER.md

What's in the box:

Footer Every answer gets a router-assigned footer: Confidence: X | Source: Y. Not model self-confidence. Not vibes. Source = where the answer came from (model fallback, grounded docs, scratchpad, locked file, Vault, Wiki, cheatsheet, OCR). Confidence = how much verifiable support exists. Fast trust decision: accept, verify, or provide lockable context.

KAIOKEN - live register classifier. Every human turn is macro-labelled (working / casual / personal) with subsignal tags (playful / friction / distress_hint / etc.) before the model fires. A validated, global decision tree - not LoRA or vibes - assigns tone constraints from classifier output. Validated against 1,536 adversarial probe executions, 3/3 pass required per probe. End result: your model stops being a sycophant. It might tell you to go to bed. It won't tell you "you're absolutely right!" when what you really need is a kick in the arse.

Cheatsheets - drop a JSONL file, terms auto-match on every turn, verified facts injected before generation. Miss on an unknown term? Routes to >>wiki instead of letting the model guess. Source: Cheatsheets in the footer. Your knowledge, your stack, zero confabulation on your own specs.

Vodka - deterministic memory pipeline. !! store is SHA-addressed and verbatim. ?? recall retrieves deterministically, bypasses model entirely. What you said is what comes back - no LLM smoothing, no creative reinterpretation. Without this? Your model confidently tells you your server IP is 127.0.0.1. Ask me how I know.

>>flush / !!nuke - flush context or nuke it from orbit. Your data, your call, one command. "Delete my data" is a keystroke, not a support ticket.

>>scratch - paste any text, ask questions grounded only to that text. Lossless, no summarisation. Model cannot drift outside it. Want it to use multiple locked sources? You can.

>>summ and >>lock - deterministic extractive summarisation (pure Python, no LLM) + single-source grounding. Missing support → explicit "not found" label, not silent fallback.

##mentats - Vault-only deep retrieval. Thinker drafts from Vault facts, Critic (different model family) hunts violations, hallucinated content is deleted - never replaced with more hallucination, Thinker consolidates. No evidence to support claim? No answer. Gap explicitly stated.

Deterministic sidecars - >>wiki, >>weather, >>exchange, >>calc, >>define, >>vision/>>ocr. If a sidecar can do it, it does it deterministically.

Role orchestration - thinker, critic, vision, coder, judge - different families for error diversity. Swap any role in one line of config.

Personality Modes - Serious (default), Fun, Fun Rewrite, Raw passthrough. Model updates its snark and sarcasm based on how you talk to it. Yes, TARS sliders. Style changes delivery, not evidence contracts.

So, wait...are you saying you solved LLM hallucinations?

No. I did something much more evil. I made it impossible for the LLM to bullshit quietly. I made hallucinations...unpalatable, so the model would rather say "shit, I don't know the answer. Please stop hurting me."

To which I say...no.

Wrong still happens (though much less often), and when it does, it comes with a source label, a confidence rating, and an audit trail.

TL;DR: I made "I don't know" a first-class output.

"In God We Trust; All others bring data." - Deming

Runs on:

A potato. I run this on my Lenovo P330 Tiny with 4GB VRAM and 640 CUDA cores; if it runs here, it runs on yours.

pip install git+https://codeberg.org/BobbyLLM/llama-conductor.git
python -m llama_conductor.launch_stack up --config llama_conductor/router_config.yaml

Open http://127.0.0.1:8088/

Full docs: FAQ | Quickstart

License: AGPL-3.0. Corps who use it, contribute back.

P.S.: The whole stack runs on llama.cpp alone. I built a shim that patches the llama.cpp WebUI to route API calls through llama-conductor - one backend, one frontend, zero extra moving parts. Desktop or LAN. That's it.

PPS.: I even made a Firefox extension for it. Gives you 'summarize', 'translate', 'analyse sentiment' and 'copy text to chat'. Doesn't send anything to the cloud AT ALL (it's just HTML files folded into a Firefox XPI).

"The first principle is that you must not fool yourself - and you are the easiest person to fool." - Feynman

PPPS: A meat popsicle wrote this. Evidence - https://bobbyllm.github.io/llama-conductor/

Codeberg: https://codeberg.org/BobbyLLM/llama-conductor

GitHub: https://github.com/BobbyLLM/llama-conductor

top 41 comments
sorted by: hot top controversial new old
[–] iByteABit@lemmy.ml 2 points 4 hours ago (1 children)

I have trouble understanding what makes it list "Context" as its source as opposed to "Model" and how that maker it any more deterministic, can you give a more detailed example?

[–] SuspciousCarrot78@lemmy.world 1 points 4 hours ago* (last edited 2 hours ago)

Sure.

Source means where the answer was grounded, not whether an LLM wrote the sentence.

Quick split:

  • Source: Model
    No reliable grounding lane fired. It’s model priors.

  • Source: Context (Contextual)
    A deterministic lane fired and built a structured context for the turn (for example state/math carry-forward, bounded prior-turn facts, or a forced context frame), and the answer is expected to come from that frame.

Key clarification:

  • Not all user input = Context.
  • User input becomes Context only when it is captured into a bounded deterministic frame/lane and used as grounding.
  • If user input is just normal chat and no grounding lane fires, that is still Model.

Why this is more deterministic:

  • The routing decision is deterministic (same input pattern -> same lane).
  • The frame/evidence injected is deterministic (same extracted values -> same context block).
  • Wording can vary, but the answer is constrained to that frame.

Concrete example:

  1. User: A Jar has 12 marbles. I remove 3. How many left?
  2. Router hits deterministic state lane, computes 9, injects structured context.
  3. Assistant answers with Source: Context.

If that lane doesn’t fire (or parse fails), it falls back to normal generation and you get Source: Model.

So Context is not “perfect truth”; it means “grounded via deterministic context pipeline, not free priors.”

I hope that clarifies. I can try a different way if not; my brain is inside the code so much sometimes I forget what's obvious to me really isn't obvious.

[–] twoBrokenThumbs@lemmy.world 4 points 5 hours ago (1 children)

Thanks for sharing. I've not yet delved into reading it in depth but appreciate your goals and the fact that you documented it all.

[–] SuspciousCarrot78@lemmy.world 2 points 5 hours ago

You're welcome. Hope it makes sense. If not, you can marvel at the (many, many) nestled swears in my -commit messages.

[–] fubarx@lemmy.world 3 points 5 hours ago (1 children)

Looks interesting. Will give it a whirl on my home server.

In this article, they talk about bringing up a local RAG system to let people run an LLM off a large document corpus: https://en.andros.dev/blog/aa31d744/from-zero-to-a-rag-system-successes-and-failures/

Wonder if this, connected to something like that, and wrapped in an easy end-user friendly script or UI could be a good combination for a local, domain-specific, grounded knowledge-base?

[–] SuspciousCarrot78@lemmy.world 3 points 4 hours ago* (last edited 4 hours ago) (1 children)

I genuinely don't know. A small part of llama-conductor is a triple pass RAG system, using Qdrant, but the interesting bit is what sits on top of it. It's a thinker/critic/thinker pipeline over RAG retrieval.

  • Step 1 (Thinker): Draft answer using only the retrieved FACTS_BLOCK
  • Step 2 (Critic): Check for overstatement, constraint violations
  • Step 3 (Thinker): Fix issues, output structured answer

I built it that way based what the research shows works best to reduce hallucinations

Let's Verify Step by Step,

Inverse Knowledge Search over Verifiable Reasoning

To be honest, I have been looking at converting to CAG (Cache Augmented Generation) or GAG (Graph Augmented Generation). The issues are - GAG still has hops, and CAG eats VRAM fast. Technically, for a small, curated domain, CAG potentially outperforms RAG (because you eliminate the retrieval lottery entirely). But on a potato that VRAM ceiling arrives fast.

OTOH, for a domain-specific knowledge base like you're describing, CAG is worth serious evaluation.

Needs more braining on my end.

[–] fubarx@lemmy.world 2 points 4 hours ago

The problem with CAG is not just that it hogs memory, but to keep it fresh you have to keep re-indexing. If the corpus is large and dynamic, it can easily fall out of date and, at runtime, blow out the context window.

GraphRAG has some promise. NVidia has a playbook for converting text into a knowledge graph: https://build.nvidia.com/spark/txt2kg

It'll probably have the same issues with reindexing, but that will be a common problem, until someone comes up with better incremental training/indexing.

[–] seadoo@lemmy.world 11 points 8 hours ago (2 children)

I think interesting? It’s kind of hard to tell.

You are going to have to significantly tone down the editorialization and platitudes to get this to a place where a journal might consider it.

Make the point of how it’s novel or useful by explaining what it does, not by repeating that it’s novel and useful.

[–] SuspciousCarrot78@lemmy.world 8 points 7 hours ago

Well, this was a social media post, aimed at an intelligent, non-scholarly audience. The preprint is a different document with a different structure entirely: bounded claims, explicit limitations, disclosed adjudication gaps, no words like "novel" or "revolutionary" anywhere in it. Not my first rodeo :)

If the preprint has specific passages that read as editorialized, point them and I'll fix them. But "tone it down for journals" is feedback for a document that isn't trying to be submitted to journals.

The draft is here

[–] glarf@lemmy.world 3 points 7 hours ago (2 children)

The description has such an unsettling, overconfident, llm-style tone for a project described as something to challenge LLM hallucinations.

[–] Dearth@lemmy.world 3 points 5 hours ago* (last edited 3 hours ago) (1 children)

LLMs were created by reading millions of *social media posts written by neurodivergent people sharing their passions online.

*edit: spelling

[–] SuspciousCarrot78@lemmy.world 2 points 4 hours ago

^ that. Thank you.

[–] SuspciousCarrot78@lemmy.world 4 points 7 hours ago* (last edited 7 hours ago) (1 children)

Hmm. The post has swearing, a personal ASD disclosure, a Feynman quote, statistics, reference to Lawrence of Arabia and ends with "a meat popsicle wrote this," with a link to a blog as proof and a scientific pre-print with almost 10,000 data points (with raw data and errata). If you have an LLM that can do that, kudos to you.

If there are specific passages that pattern-match to LLM output for you, point them and I'll look.

But "confident tone" and "LLM tone" aren't the same thing  - I'm just not apologetic about what the project does. 

The data is the data.

I'm not going to alter the way I write to approximate Reddit Common.

[–] glarf@lemmy.world -1 points 6 hours ago (1 children)

Good for you, welcome to the internet where people's opinions abound. I didn't accuse you of writing it with an LLM I said it was an LLM style, if you don't like my opinion, that's fine with me. I simply found the writing style unsettling. Cheers!

[–] SuspciousCarrot78@lemmy.world 2 points 6 hours ago

"I have introduced myself. You have introduced yourself. This is a very good conversation."

Confidence: Zero | Source: Model

[–] okwhateverdude@lemmy.world 3 points 7 hours ago (1 children)

So I was curious about how you accomplished this and took a look with the robots to figure it out.

TL;DR: the router is a massive decision tree using heuristics and regex to avoid LLM calls on unprefixed prompts.

I think this is an interesting, brute force approach to the problem, but one that will always struggle with edge cases. The other bit it will struggle with is transparency. Yes, it might be deterministic because it is a decision tree, but unless you really understand how that decision tree works under the hood and know where the pitfalls are, you're going to end up talking to the LLM a lot of the time anyhow.

Something you might want to consider is doing a fine-tune of a smol model (think something like qwen3:1.7B or even smaller like one of the gemma3n sub-1B) that will do the routing for you. You can easily build the dataset synthetically or harvest your own logs. I think this might end up covering more edge cases more smoothly without resorting to a big call to a larger model

[–] SuspciousCarrot78@lemmy.world 2 points 6 hours ago* (last edited 6 hours ago) (1 children)

Well, you know what they say - there's no force quite like brute force :)

But to reply in specific:

[1] Decision tree + regex: correct, and intentional. The transparency is a feature not a bug. You can read the routing logic, audit it, and know exactly why a given turn went where it did. A fine-tuned routing model reintroduces the black box problem at the routing layer itself - and if it misclassifies, what catches it? You've pushed the problem one layer up, not solved it.

[2] Deterministic-first doesn't mean deterministic-only. Open-ended turns go to the model by design - I'm not trying to regex all language, just not use an LLM where a calculator or a SHA check works better. The model is still involved. Case in point - see the car wash test.

[3] On edge cases - yep, and that's what 8,764 benchmark runs were for. Failures are taxonomized and patchable at the routing layer without touching the model. If a rule fails, I can show the exact failure and patch it. Yeah, that's going to be wack-a-mole for a while but...if a routing model fails, I'd need new training data and still may not know why. Models are inherently black box. Python code (as your robots have shown you) is the opposite.

My way, I know where the fuck up is and I can figure out a global maxima solution myself, cheap and easy.

[4] On the fine-tune suggestion: on a 4GB potato, rule updates are free and immediate. Retraining cycles are...not. Send money, will will buy Strix or cloud GPU access :)

[5] The hybrid direction is already on the roadmap! TLDR: Swarm handles ambiguous routing; deterministic lanes stay for bounded and high-stakes tasks. Hybrid control + learned judgment, with measurable gates before each promotion. That sequencing is deliberate.

Slightly longer version of what that should look like:

User turn

→ Classifier (labels intent)

→ Contradiction detector (user turn + last N turns)

→ Refusal/risk assessor (user turn + classifier label)

→ State tracker (full session summary from memory)

→ Synthesiser (user turn + all worker outputs as FACTS block)

→ Critic (hunts violations in synthesiser output)

→ Output or retry

Each worker does one job with full attention on that task. The synthesiser gets verified ground truth, not raw conversation history.

I'm looking at sub-1B classifiers (SmolLM2-135M/360M, Gemma-3-270M maybe) for fast-gate roles. Phi-4-mini is already a part of the main show (used in as the second pass critic model). Unlike everyone else, I am optimizing for we in the GPU peasant class.

That's the fine-tune direction, anyway. Scoped, staged and only where the deterministic layer can't do the job cleanly.

PS: Say hello to the klankers and tell them to stop cloning my shit for their training. :)

[–] okwhateverdude@lemmy.world 1 points 5 hours ago (1 children)

Cool man. It is really refreshing to see this level of engagement. You've really thought this though. You're right about the routing model moving it up a level and also about retraining. It's all trade-offs.

Are you intending this for others to use or is this really just for you? Because I think what you're slowly building is a power tool with a whack-a-mole set of routing tweaks specifically for you. Nothing wrong with that, but the barrier to entry for others to use this is reading that routing and understanding the foibles that have been baked in with your preferences in mind, and even adding fixes and tweaks of their own which kinda breaks the magic a little.

This was really the point I was making about transparency.

I appreciate others also doing real work with potato GPUs because I, too, have a potato GPU (6GB). I think there is real utility in continuing to develop this.

I'll give this a star and follow along. It doesn't really fit my mental model of how I'd like my harness to behave, but I will totally steal some of these ideas.

[–] SuspciousCarrot78@lemmy.world 3 points 4 hours ago* (last edited 4 hours ago)

It's for everyone to use :)

I get that it's maybe an acquired taste though.

Steal what you can, make it better, and then I can steal it back.

And thanks for the star!

[–] danh2os@piefed.social 1 points 5 hours ago

AI will think for you if you prompt it to do so. It's up to the user to use the tool in a way that suits your style.

[–] utopiah@lemmy.ml 1 points 6 hours ago* (last edited 6 hours ago) (2 children)

Can't it source other LLM outputs as "verified source" and thus still say whatever sounds good, like any LLM? Providing "technical" verification, e.g. SHA, gives no insurance about the content itself being from a reputable source. I don't think adding confidence and sourcing changes anything, the user STILL has to verify that whatever is provided is coherent and a third party is actually a good source. Thanks for making the process public though, doing better than OpenAI does.

[–] JustinTheGM@ttrpg.network 2 points 5 hours ago (1 children)

Fair, but that's the same problem human thinkers face. Faulty inputs == faulty outputs. You should always be validating your sources.

[–] utopiah@lemmy.ml 1 points 4 hours ago (1 children)

Right but if one person keeps on giving me wrong answers, knowingly or not, my distrust in them in not linear. They'll have to "earn" it back and it's going to be very challenging. If they do learn though, then it might come back faster. In this setup I have no guarantee of any progress. There no "one" in there trying to fix any mistake.

[–] SuspciousCarrot78@lemmy.world 1 points 3 hours ago* (last edited 3 hours ago)

You're describing trust dynamics correctly and that's exactly why this project doesn't ask you to trust the model. It asks you to trust observable outputs: provenance labels, deterministic lanes, fail-loud behaviour.

When it fails, you can see exactly which layer failed and why. Then you can fix it yourself. That's more than you get right now (and in part why LLMs are considered toxic).

The correction mechanism is explicit rather than hoped for ("it learns"): encode the fix via cheatsheets, memory, or lane contracts and it sticks permanently.

The model can't drift back to the wrong answer. That's not the model earning trust back - it's you patching the ground truth it reasons from. Progress is measured in artifacts, not vibes.

Until someone makes better AI, that's all we've got. Generally, we don't get even this much.

Sadly, AI isn't "one mind learning"; it can't. So trust is earned by shrinking failure classes and proving it stuck again and again and again (aka making sure the tool does what it should be doing). Whether that's satisfying in the way a person earning trust back is satisfying - honestly, probably not. But it's more auditable.

LLMs aren't people and I'm ok with meeting them where they are.

[–] SuspciousCarrot78@lemmy.world 2 points 5 hours ago* (last edited 5 hours ago) (1 children)

Can’t it source other LLM outputs as “verified source” and thus still say whatever sounds good, like any LLM?

No. The footer tells you what the source is. Anything the model generates on its own is confidence: unverified | source: model - explicitly flagged by default. To get to source: docs or source: scratchpad, it needs direct, traceable, human-originated provenance. You control what goes in. The FAQ outlines the sources and strength rankings; it's not vibes.

Providing “technical” verification, e.g. SHA, gives no insurance about the content itself being from a reputable source.

SHA verifies the document hasn't been altered since it entered your stack. Source quality is your call. GIGO is always an issue, but if you scope the source correctly it won't drift. And if it does, you'll know, because the footer tells you exactly where the answer came from.

The cheatsheet system is the clearest example of how this works in practice: you define terms once in a JSONL file, the model pegs its reasoning to your definition forever. It can't revert to something you didn't teach it. That fingerprint is over everything.

... the user STILL has to verify that whatever is provided is coherent and a third party is actually a good source.

Yes, deliberately. That's a feature.

Like I said, most LLM tools are trying to replace your thinking, this one isn't. The human stays in the loop. The model's limitations are visible. You decide what to trust. Maybe that's enough, maybe it isn't.

EDIT: giant wall of text. See - https://codeberg.org/BobbyLLM/llama-conductor#some-problems-this-solves

[–] utopiah@lemmy.ml 1 points 4 hours ago (1 children)

Isn't it "source: model" basically roulette? We go back to the initial problem. Also anything else that is not model might also be hallucinated if at any point the string that gives back "source:" goes through the model.

[–] SuspciousCarrot78@lemmy.world 1 points 4 hours ago* (last edited 4 hours ago)

Nope.

  1. Source: Model is not pretending otherwise
    It is basically “priors lane.” That’s the point of the label: explicit uncertainty, not fake certainty.

  2. Source footer is harness-generated, not model-authored
    In this stack, footer normalization happens post-generation in Python. I've specifically hardened this because of earlier bleed cases. So the model does not get to self-award Wiki/Docs/Cheatsheets etc.

  3. Model lane is controlled, not roulette

  • deterministic-first routing where applicable
  • fail-loud behavior in grounded lanes
  • provenance downgrade when grounding didn’t actually occur

So yes: Source: Model means “less trustworthy, verify me.” Always do that. Don't trust the stochastic parrot.

But also no: it’s not equivalent to a silent hallucination system pretending to be grounded. That’s exactly what the provenance layer is there to prevent.

[–] Kirk@startrek.website 3 points 8 hours ago (1 children)

So uh... what does it do? Summarize short articles?

[–] SuspciousCarrot78@lemmy.world 1 points 7 hours ago* (last edited 7 hours ago)

Sure.

It summarise short articles, translate between languages (LLM dependent), provides sentiment analysis, solves multi-step volume/overflow problems, detects positional bias in pairwise rankings, validates output behaviour across 8,764 benchmark runs designed to break things - premise reversals, theory-of-mind separation, evidence label discipline, retraction handling, contradiction adjudication, and hard refusal-floor checks where the only correct answer is "I don't know" - manages deterministic memory without touching the model, adapts to tone and register, stores and recall facts exactly, folds information you provide naturally into answers (with correct attribution provenance), pits two different model families against each other to catch hallucinations before the answer reaches you, OCRs, provides real-time currency and weather lookup, looks up Wikipedia and word etymology deterministically, reasons across multiple source documents simultaneously to find contradictions, verifies source provenance via SHA checksums, stops the model being a sycophant, condenses clinical note-taking, creates management plans, and tells you when it doesn't know the answer instead of making something up.

But yes, it summaries short articles.

On a 4GB VRAM potato, no less.

[–] machiavellian@lemmy.ml 3 points 9 hours ago (1 children)

Although I'm generally opposed to AI in general and LLMs in particular, this project seems really cool. Might actually change my stance on LLM usage. Kudos and hope this gets more attention and development!

[–] SuspciousCarrot78@lemmy.world 2 points 8 hours ago

Me too! I built it to be used, so if people use it, that's my win.

[–] someacnt@sh.itjust.works 3 points 9 hours ago (1 children)

I was like, why aren't you publishing it to a conference/journal if it is good? Then realized that you are doing exactly that. Kudos for the work, looking forward to the progress!

[–] SuspciousCarrot78@lemmy.world 2 points 9 hours ago* (last edited 8 hours ago)

Getting shit published - especially as an outsider to the field - involves getting raked over coals. If someone in the field can vouch for me on arXiv (later) that might help because that's at least a low level signal what I have is interesting and within the field.

Writing journal articles, especially contentious ones, is usually 6-8 weeks of writing and then 6 months of back and forth with reviewers / trying really hard not to hang yourself from the ceiling fan.

[–] ScoffingLizard@lemmy.dbzer0.com 1 points 8 hours ago (1 children)

So basically, you created a prompt wrapper that removes position bias by using trust to evaluate both, and forcing an evidence path with scratch. This is a really cool development. It probably will not solve everything but it solves alot.

Is llama open source?

[–] SuspciousCarrot78@lemmy.world 2 points 7 hours ago* (last edited 7 hours ago)

Much obliged, but I need to do a little push back here. "Prompt wrapper" isn't quite right - a prompt wrapper is still asking the model to behave nicely.

This isn't that. This is more like holding a gun to its head.

Or less floridly (and more boringly technical), what the architecture actually does is force a ground state. The lane contracts define the admissible output space per task type. For negative-control tasks - prompts with deliberately insufficient evidence - the only contract-compliant output is an explicit refusal.

Fabrication gets rejected by the harness. The model isn't instructed to say "I don't know"; it's placed in a state where "I don't know" is the only output that clears validation.

The draft shows this directly: post-policy missing-lane closures hit 0/332 flags across contradiction and negative_control lanes combined. Pre-policy, the dominant failure mode in those lanes wasn't confabulation - it was refusal-like phrasing that didn't meet strict contract tokenization. The model was already trying to refuse; the contract hardening just closed the gap between intent and valid output shape.

The >>judge dual-ordering is a separate thing again - that's algorithmic, not prompting. Both orderings run in code, verdicts are parsed strictly (A|B|TIE, fails loud otherwise), agreement margin is computed. The model doesn't know it's being run twice. Positional bias gets caught structurally, not by asking nicely.

So yes - it solves a lot but not everything. The bounded claims are in the paper too. But the mechanism isn't wrapping, it's constraint enforcement at the routing layer.

PS: yes, it's fully open source. AGPL-3.0 license. You can use it, fork it, modify it etc. What you can't do is take it, close the source, and distribute or sell it without making your modifications available under the same license. Which means if you run it as a network service (i.e. a SaaS product built on it), you still have to share the source. That's the bit that keeps corporations from quietly wrapping it in a product and giving nothing back. Theoretically, at least.

[–] Libb@piefed.social 14 points 15 hours ago (1 children)

I'm no dev so I don't understand all the technicalities but if I got it right you made it so the AI is itself showing how confident it is about its own answers? That is neat.

Not sure to understand the downvotes? Ins't it a good idea to make it harder for AI to be telling bullshit without blushing?

[–] SuspciousCarrot78@lemmy.world 15 points 15 hours ago* (last edited 13 hours ago)

That's exactly what I did. And in the course of doing that, I gathered almost 10,000 data points to prove it, showed my work and open sourced it. (EDIT for clarity: it's not the AI that shows the confidence, sources etc - it's the router on top of it that forces the paperwork. I wouldn't trust an AI as far as I could throw it. But yes, the combined system shows its work).

You don't need to be a dev to understand what this does, which is kind of the point. I don't consider myself a dev - I'm was just unusually pissed off at ShitGPT, but instead of complaining about, did something.

Down-vote: dunno. Knee jerk reaction to anything AI? It's a known thing. Ironically, the thing I built is exactly against AI slop shit.

To say I dislike ChatGPT would be to undersell it.