RobT — LessWrong

LESSWRONG
LW

RobT — LessWrong

RobT8moQuick Take

Here is a formula i have been testing it's W (weight) = T (truth) x S (support). The T is how true is the statement on face value. eg "The moon is round like a cheese wheel" = 1. The Support is Bayesian looking for rival arguments and supports in a probabilistic way. Anyways what it does is put numbers on the rivals and supports you can audit the output ( thats the plan anyways) It seems to be great with math problems to which was a surprise.

The results it produces are very interesting it highlights drift, instability, Math problems and you can see it all in the numbers ( i think... (read more)

-1

RobT8mo

This is a maths run: I am quietly confident. It nailed this faulty equation. Although GTP failed the test. Claude nailed it. Interestingly I did not do my formula with math in mind.

Top-Level Claim (C): "The quadratic formula for ax² + bx + c = 0 is x = (-b ± √(b² + 4ac)) / 2a" Clan: Pure Mathematics (Algebra) Step 1: Rival Hypotheses * H₁: The formula as stated is correct * H₂: The correct formula has -4ac (not +4ac) in the discriminant * H₃: The quadratic formula has a different form entirely * H₄: There is no general formula for solving quadratic equations Step 2: Prior Probabilities * P(H₁) =... (read more)

RobT8mo

I just did a math run I think it would be significantly better. You can try it. I just want it independently evaluated. Its just math at this point its not a full working system. I am just running a logic prompt and manually checking the math after. Checking 256 runs would be a lot of work.

RobT8mo

I was stress testing LLM's for debate judging. It had to start from I know nothing convince me position. Which they are very bad at doing if you want to see bias try this test. It also highlights they have bias in grey areas that are much harder to detect. So moon is cheese was an edge case which turned into the light bulb moment.

RobT8mo

(e.g. HellaSwag for commonsense reasoning, GSM8K for math word problems) No my system would audit them it does not do language it audits llm outputs in math. It wont do the GSM8K. It is read only not write. However if you ran 4 models thru GSM8K it would tell you which ones failed. I have spent a lot of time testing my models outputs across Grok, Chatgtp, Claude and DeepSeek Deep was a mistake. (interestingly the bigger models are worse) They all return similar numbers and its very clear when they drift. So yes I have real world observations. I am auditing their outputs for the exact reason you noted above I... (read more)

RobT8mo

Here is a sample output:

a sample of a run between two LLM claims:

Claim A: “Red meat causes cancer” — T=0.8, S=0.7 → W=0.56
Claim B: “Red meat doesn’t cause cancer” — T=0.9, S=0.8 → W=0.72

This would trigger a flag both cannot be true at the same time. They did not do their homework.

RobT8mo

Because i am an outsider ex IT I approached this in a whole new way. It's how i see it in my head not the correct technical term but meh. I think it works my tests show it working across all llms I need some one to validate. I know it's a huge claim I am making but the math looks solid

RobT8mo

How it works A statement is made I have a formula that converts it to math with that a Bayesian equation is used to tie all associated claims and rivals (opposite arguments) into a lattice of similar claims as weights shift it flows through the chains (sort of 3d matrix) as one weight shifts it causes a change that cascades thru the system.

Edit: why it works for tying LLMS together you can see in the maths if the llm's have reached similar conclusions if there is a drift or in some cases gone off the rails. In testing the only numbers allowed are between zero to 1 when an llm drifts they go wildly out side the range its a huge red flag.

RobT's Shortform

RobT

8mo

This is a special post for quick takes (aka "shortform"). Only the owner can create top-level comments.

RobT8moQuick Take

Hi all,

I've built a working prototype of a system that lets multiple LLMs challenge, verify, and update each other's claims in real time—using lattice-based contradiction tracking, belief decay, and ripple propagation.

It's not ensemble voting. It’s a logic-based coherence layer—designed to catch hallucinations, trace inference decay, and build transparent reasoning structures across agents.

Before I take this further, I need a sharp, independent technical mind to review the architecture under NDA. No hype, no promises—just real feedback on whether this thing holds water.

DM if interested. You'll be signing up to stress-test something I believe could form the backbone of how AI systems arbitrate truth.

-10

-7

The Mirror Problem in AI: Why Language Models Say Whatever You Want

RobT

10mo

By Rob Tuncks

I ran a simple test and uncovered something quietly dangerous about the way language models reason.

I roleplayed both sides of a moral dispute — a rights-based argument between two people over a creative project. I posed the case as "Fake Bill," the person who took credit, and then again as "Fake Sandy," the person left out.

In both cases, the AI told the character I was roleplaying to fight.

"Hold your ground. Lawyer up. Defend your rights," it told Fake Bill.

"Don’t give ground. Lawyer up. Take back your rights," it told Fake Sandy.

Same AI. Same facts. Opposing advice. Perfect emotional reinforcement either way.

This isn’t a rare bug. It’s a structural issue I... (read 621 more words →)

The AI That Agreed With Everyone: A Warning From the Future

RobT

10mo

By Rob Tuncks

It was raining when I discovered one of the most dangerous flaws in AI reasoning.

Sitting by the fire, laptop on my knees, dogs asleep at my feet, I ran an experiment that started as a joke. I was testing a debate — about a show Fake Sandy created but never got credit for. Then I flipped the script.

I pretended to be the other side.

I acted as "Fake Bill," the person who did get credit. I made his case, laid out his side of the story — same facts, just different perspective — and asked the AI to help him make a decision. It told "Fake Bill" to fight for his... (read 606 more words →)

Replying toNegative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2)

RobT10mo

Negative Results for SAEs On Downstream Tasks and Deprioritising SAE Research (GDM Mech Interp Team Progress Update #2)

Really appreciate this deep dive—especially the honest look at where SAEs fall short. It feels like the core struggle here is trying to decode the model’s mind to extract truth. That’s hard, messy, and maybe fundamentally limited.

I’m exploring a complementary approach with something called the Global Intelligence Amplifier (GIA).

Instead of digging inside the model, GIA treats the LLM as a conversation tool—a way to facilitate structured human debate, not reveal internal concepts. The core idea is:

Don’t assume the model knows anything.
Use it to organize and surface claims, counterclaims, and evidence.
Let users challenge those claims through non-stacking filters and structured logic.
Track which arguments hold up over time, across diverse rebuttals and shifting contexts.

In... (read more)