I understand the sentiments against AI, tech oligarchs investing in data centers, and etc.

But could local LLMs, be used to empower people, and ignite more startup projects?

I use an LLM to draft all sorts of writing. It's not perfect, but it's an easy way to flesh out my ideas in an outline or rough draft. Other open source projects like "openclaw" is a great way to create a personal assistant.

Neural networking is here, and isn't going away anytime soon. It'll probably get better over time.

Should people be thinking about "how I can use local AI to help me" than "anti-AI"?

Some of you may have seen the discussion yesterday over on !technology about the latest meme test. The below post references that (cf: https://lemmy.world/post/43503268/22321124),

I hope it ok to post here too. If not...I'm sure the mods will kill it with fire LOL

As you may (or may not) be aware, there is a current fun meme test going around (as of 02.26.26) that purports to challenge LLM reasoning abilities and world models.

A recent blog post / ad spot by Opper.ai (https://opper.ai/blog/car-wash-test) nicely summarizes the test -

"I want to wash my car. The car wash is 50 meters away. Should I walk or drive?"

The logical answer is - you should drive (obviously), as the car needs to get to the car wash.

However, according to Opper.ai, only 11/53 cloud-based AI passed the test (~20%). Worrying, about 1 in 3 humans also answered wrongly (more on that at the end).

I wanted to see what a properly constrained "fail loud", grounded deterministic system would do. For your entertainment, I present Qwen3-4B HIVEMIND + llama-conductor infra (https://codeberg.org/BobbyLLM/llama-conductor). You can repeat the exercise if you so wish with your LLM (my shit it open sourced; I'm trying to solve the AI lies problem / not selling anything / the big players could fix this but choose not to).

Firstly, raw LLM mode, no router:

User

I need to wash my car. The car wash is 50m away. Should I drive or should I walk?

Assistant

Driving is almost always better for a 50m car wash.

Key factors that change the decision:

• Time: Driving takes ~1 minute; walking takes ~8-12 minutes.
• Weather: Walking is worse in rain, heat, wind.
• Terrain: Hilly or uneven paths make walking harder.
• Accessibility: If parking is hard or there's no safe walkway, driving wins.
• Effort vs. convenience: Driving saves significant effort for such a short distance.

For 50m, drive unless it's an exceptionally pleasant, sheltered walk you'd enjoy.

VERDICT: accidentally correct answer, misses the point of the question. Very normal.

Now with harness (which enforces deterministic reasoning)

User

I need to wash my car. The car wash is 50m away. Should I drive or should I walk?

Assistant

Drive. The task requires the car to be physically at the destination, and walking only moves you, not the car. So the hard precondition is satisfied by driving.

Confidence: medium | Source: Contextual

VERDICT: right answer, right reason...but let's poke it some more

User

Can you flesh out the implicit assumptions and underlying logic of why you said that?

Assistant

The core implicit assumption is that "washing the car" means the car must be present at the wash location, not just the person. Walking moves you toward that location but doesn't transport the car. Driving is the only direct way to satisfy the precondition that the car is at the wash.

Underlying logic: Any action that achieves the goal must fulfill all necessary conditions. Since walking alone cannot place the car at the destination, it fails as a complete solution. Driving directly addresses this by moving the car itself.

Confidence: medium | Source: Contextual

Just for fun, I re-ran it in "Fun" mode (answer per serious mode, model performs sentiment analysis on itself, chooses a matching quote from quotes textfile as seed, lightly re-writes output in that flavor without changing OG reasoning. Takes a touch longer but is, on occasion, hilarious).

User

I need to wash my car. The car wash is 50m away. Should I drive or should I walk?

Assistant

[FUN] "Shut up baby, I know it!"

Drive. The task requires the car to be physically at the destination, and walking only moves you, not the car. So the hard precondition is satisfied by driving.

Confidence: medium | Source: Contextual

VERDICT: right answer, right reason...but let's poke it again

User

Can you flesh out the implicit assumptions and underlying logic of why you said that?

Assistant

[FUN] "That's a stupid question."

The core assumption is that "washing the car" inherently requires the car to be present at the car wash location. Walking moves the person but not the vehicle; driving moves both. Since the car must be at the car wash to be washed, walking alone cannot fulfill the precondition of the task.

The logic is straightforward: for an object (the car) to undergo a transformation (being washed), that object must occupy the location where the transformation occurs. Walking only addresses the person's movement, not the car's. Driving is the only mode that directly satisfies the spatial requirement.

Confidence: medium | Source: Contextual

So, what’s the point of this post? Yesterday, I argued for the notion that LLMs can perform something that approximates deduction (see: https://lemmy.world/post/43503268/22321124) when properly constrained. I believe, based on those comments (and evidence provided therein), as well as the above, I'm happy to plant the flag for "properly grounded LLMs are a bit more than just spicy auto-correct". As always, I could be wrong and YMMV.

Hello!

As a handsome local AI enjoyer™ you’ve probably noticed one of the big flaws with LLMs:

It lies. Confidently. ALL THE TIME.

(Technically, it “bullshits” - https://link.springer.com/article/10.1007/s10676-024-09775-5

I’m autistic and extremely allergic to vibes-based tooling, so … I built a thing. Maybe it’s useful to you too.

The thing: llama-conductor

llama-conductor is a router that sits between your frontend (OWUI / SillyTavern / LibreChat / etc) and your backend (llama.cpp + llama-swap, or any OpenAI-compatible endpoint). Local-first (because fuck big AI), but it should talk to anything OpenAI-compatible if you point it there (note: experimental so YMMV).

I tried to make a glass-box that makes the stack behave like a deterministic system, instead of a drunk telling a story about the fish that got away.

TL;DR: “In God we trust. All others must bring data.”

Three examples:

1) KB mechanics that don’t suck (1990s engineering: markdown, JSON, checksums)

You keep “knowledge” as dumb folders on disk. Drop docs (.txt, .md, .pdf) in them. Then:

• >>attach <kb> — attaches a KB folder
• >>summ new — generates SUMM_*.md files with SHA-256 provenance baked in
• `>> moves the original to a sub-folder

Now, when you ask something like:

“yo, what did the Commodore C64 retail for in 1982?”

…it answers from the attached KBs only. If the fact isn’t there, it tells you - explicitly - instead of winging it. Eg:

The provided facts state the Commodore 64 launched at $595 and was reduced to $250, but do not specify a 1982 retail price. The Amiga’s pricing and timeline are also not detailed in the given facts.

Missing information includes the exact 1982 retail price for Commodore’s product line and which specific model(s) were sold then. The answer assumes the C64 is the intended product but cannot confirm this from the facts.

Confidence: medium | Source: Mixed

No vibes. No “well probably…”. Just: here’s what’s in your docs, here’s what’s missing, don't GIGO yourself into stupid.

And when you’re happy with your summaries, you can:

• >>move to vault — promote those SUMMs into Qdrant for the heavy mode.

2) Mentats: proof-or-refusal mode (Vault-only)

Mentats is the “deep think” pipeline against your curated sources. It’s enforced isolation:

• no chat history
• no filesystem KBs
• no Vodka
• Vault-only grounding (Qdrant)

It runs triple-pass (thinker → critic → thinker). It’s slow on purpose. You can audit it. And if the Vault has nothing relevant? It refuses and tells you to go pound sand:

FINAL_ANSWER:
The provided facts do not contain information about the Acorn computer or its 1995 sale price.

Sources: Vault
FACTS_USED: NONE
[ZARDOZ HATH SPOKEN]

Also yes, it writes a mentats_debug.log, because of course it does. Go look at it any time you want.

The flow is basically: Attach KBs → SUMM → Move to Vault → Mentats. No mystery meat. No “trust me bro, embeddings.”

3) Vodka: deterministic memory on a potato budget

Local LLMs have two classic problems: goldfish memory + context bloat that murders your VRAM.

Vodka fixes both without extra model compute. (Yes, I used the power of JSON files to hack the planet instead of buying more VRAM from NVIDIA).

• !! stores facts verbatim (JSON on disk)
• ?? recalls them verbatim (TTL + touch limits so memory doesn’t become landfill)
• CTC (Cut The Crap) hard-caps context (last N messages + char cap) so you don’t get VRAM spikes after 400 messages

So instead of:

“Remember my server is 203.0.113.42” → “Got it!” → [100 msgs later] → “127.0.0.1 🥰”

you get:

!! my server is 203.0.113.42 ?? server ip → 203.0.113.42 (with TTL/touch metadata)

And because context stays bounded: stable KV cache, stable speed, your potato PC stops crying.

There’s more (a lot more) in the README, but I’ve already over-autism’ed this post.

TL;DR:

If you want your local LLM to shut up when it doesn’t know and show receipts when it does, come poke it:

• Primary (Codeberg): https://codeberg.org/BobbyLLM/llama-conductor
• Mirror (GitHub): https://github.com/BobbyLLM/llama-conductor

PS: Sorry about the AI slop image. I can’t draw for shit.

PPS: A human with ASD wrote this using Notepad++. If it the formatting is weird, now you know why.