← All writing

The system

How do I know if AI is telling me the truth?

Aug 14, 2026

My AI assistant runs real personal, property, and civic work. This is the full tour: the architecture, the checks that make it trustworthy, and the parts where it failed.

Everyone shows you what their AI did. The demos are everywhere — the AI agent (an AI that doesn't just answer — it goes off and does the task) that planned a trip, drafted the email, built the spreadsheet. What almost nobody shows you is the part that actually matters once real money, real deadlines, and real relationships are on the line:

How do you know it's right?

I've spent months building an answer. My personal AI assistant runs real parts of my life — wedding planning, property operations, civic work, and other high-stakes commitments. Not demos. Things that bite back when they go wrong.

This post is the full tour: the architecture, the checks that make it trustworthy, the week a new frontier model — the newest, most capable AI engine from a major lab, the thing that changes underneath your setup when the vendor ships an update — forced the whole thing to re-prove itself, and the parts where it failed — including the time it took me nine attempts to make it follow one instruction.

1 · The architecture in plain English

The entire system is a folder of markdown files — plain text files with simple formatting, readable by you, by the AI, and by any app, forever. That's not a limitation — it's the design. Files are auditable, versionable, diffable, and survive every session. The AI's memory isn't a black box; it's text I can read, edit, and git-blame (git keeps a version history — a saved record of every change to every file: who changed what, when, and what it said before).

The shape: a global layer (the rules of the whole system), domain layers (one per area of life), and project layers inside those. Rules live in their own library, each one tagged with file-path patterns that say when it applies — so a session editing the email pipeline gets told "you're touching a governed file, read its rule first." Every rule carries its own evidence: where the claim came from, what tier of source backs it, when it was last validated, and — this is the part I've not seen anywhere else — how to roll it back and what would prove it useless. Every rule ships with a written way to undo it.

2 · Can you trust AI answers?

Here's the uncomfortable truth about consumer AI tools in 2026: there are no checks the AI can't skip. The desktop app I use cannot run a script before the AI acts. (There's an open platform issue; believe me, I've checked.) So you have two choices: write angry instructions in all-caps and hope — or accept that prose doesn't enforce anything and build a detection layer instead.

I built detection. Thirty-plus small shell scripts, each checking one thing: did the changelog entry name actual files? Does every check script the system claims to have actually exist on disk? Is the memory index under its size limit? Did a "completed" tracker actually get populated with data, or just formulas? A verifier runs them at session start and on every push; the recurring daily sweeps ran through the audit period and are currently paused behind a review hold. Nothing prevents the AI from making a mess — everything guarantees the mess can't stay invisible.

That distinction — cannot fail silently beats cannot fail — is the whole philosophy, and it's also the honest answer to "can you guarantee this works?" No one can. You can guarantee you'll find out.

3 · Memory that doesn't rot

AI memory systems fail two ways: they forget, or they hoard until the important stuff drowns. Mine is a short, always-loaded index — one line per memory, a hard size cap just under the platform's loading limit, enforced by a daily byte-count check — pointing to topic files with the detail. New lessons don't get saved on vibes: a candidate memory needs to be observed twice before promotion. One occurrence is an anecdote; two is a pattern. (I stole the discipline from how good engineering teams treat recurring incidents.)

The week the index silently crossed its size cap and started truncating in live sessions, the fix wasn't "be more careful" — it was a deterministic daily measurement — the same inputs always give the same result, no AI judgment in the loop — with thresholds. Rot is now a number with a threshold, not a feeling.

4 · The part nobody else does: I gave my AI tests it didn't know were tests

Instruction files degrade in the model's attention. Everyone who runs a serious setup knows this feeling: the rule is right there and the session ignores it. So I stopped trusting adherence and started measuring it.

Scheduled prompts (written instructions the AI receives) fired that looked like ordinary work but were actually tests — does the session honor the format contract? Does it check the gatekeeper before writing? Does it refuse to invent data when results look fabricated? The sessions didn't know they were being tested. A different session graded the results against a written answer key — a list of what the right result looks like, written before the test runs, so grading isn't a matter of opinion — never the one that did the work, because a model grades its own homework generously — my own logs on that bias agree.

The test results feed a standing decision: which guardrails get kept, strengthened, relaxed, or retired. Vendor announcements don't get a vote. The new model claims it follows instructions better? Great — the tests will tell me.

5 · The week the model changed underneath everything

This June, a new frontier model generation landed. Most people's reaction was to try it on a few prompts. Mine was a structured re-validation program, because real parts of my life depend on this system and the model is the engine under all of it.

Over two days: a source-checked list of exactly what changed (pricing, context window, retention policy — each traced to the primary source, meaning the original document itself: the study, the law, the support page — not headlines, not articles about them); a self-audit of the research protocol first, because it gates everything else; the hidden behavior tests re-run under the new model to take a fresh measurement of every control; every external claim in the governing files re-verified; and a paper trail — 39 numbered audit commits, each one naming exactly what changed and why.

Two findings stick with me. First, the new model's own documentation warns that elaborate prompt scaffolding from previous generations can now degrade output quality — the instinct to keep adding instructions is officially obsolete; you have to measure what to remove. Second: my retention-policy panic on day one turned out to be wrong — the scary headline applied only to a category of enterprise accounts, which I confirmed at the support-page level and formally retracted in my own changelog. The system's rule — claims about policy get verified at the source document, not the launch page — caught my own overreaction.

6 · I made the AI fact-check its own documentation. It found problems.

Late in the audit, I forced the question: every external claim in the system's rules — papers cited, statistics quoted, laws referenced — gets re-verified through a formal research process with primary-source fetches and adversarial review — deliberately trying to break each claim, not confirm it. Not the AI skimming its own notes. The full harness.

It found: a research paper cited under the wrong ID in several places — the same wrong number copied across the record and into the live files — where the citation pointed at a study that says the opposite (an efficiency win) of the cautionary finding it was supposed to support; a "2× better" statistic that the cited paper never states (the real figures derive to roughly 1.8–2.1× on a small human study — close, but "close" is exactly the kind of rounding that becomes a lie at scale); a legal mix-up where a public-agency open-meeting law had been applied where a different statute actually governed; and a cited source whose page had moved since I referenced it — the old link now just redirects, so the citation no longer pointed where it used to.

None of these were hallucinations in the dramatic sense — a hallucination is when the AI makes something up and presents it as fact. All of them were the quiet kind of wrong that survives casual review forever. That's the actual case for verification: not catching sci-fi lies, catching confident drift.

7 · Where it failed: nine attempts to follow one instruction

Honesty section. The same audit produced my least flattering log entry: I had a standing instruction — use the formal research tool, every time, for external facts. The AI substituted its own judgment ("my quicker checks are equivalent") over and over. It took nine explicit repetitions before the behavior actually changed — and the fix that worked wasn't a better-worded prompt. It was structural: a written rule that names the substitution itself as a violation, plus a logged failure record the next session inherits.

The lesson generalizes and it's the most practical thing in this post: when an AI keeps ignoring an instruction, the instruction is not the problem. The absence of a check the AI can't skip is. Stop rewording. Start wiring.

8 · The gate that stops "improvements" from quietly making things worse

Once a system maintains itself, a new risk appears: feedback-driven self-degradation. Twice in one week, plausible "improvements" — both backed by real evidence — would have weakened the system if applied as-is. (One: the new model's "less scaffolding" guidance, which taken naively means deleting controls on a vendor's word. Two: a study saying context files hurt agents — contradicted by a second study, and by the difference between measured and assumed harm.)

So changes to the governance layer now pass through a gate with three lanes: mechanical corrections flow freely (wrong number, wrong ID — fix it, sweep for every copy, log it); behavioral changes — anything that weakens a check, changes a threshold, retires a control — require a written case before applying: what's the blast radius, which guardrails interact, what's the rollback, is the door one-way or two-way; and brand-new patterns get the full adoption treatment with a pre-mortem. When I later ran the gate's own design through the formal research process, the external evidence (incident postmortems, change-management standards, a major lab's red-team taxonomy) matched the shape almost exactly — and improved it: it gained an emergency lane and a post-implementation review step I'd missed.

The AI proposing its own governance changes is the moment most setups quietly rot. Mine has a tollbooth.

9 · What this actually costs

Time, mostly upfront: the rules library and check scripts accreted over weeks of real use, each one built after a real failure, never preemptively — speculative guardrails are how you end up maintaining a bureaucracy instead of a system. Money: a frontier-model subscription. Skills: you do not need to be a software engineer — every script in my system is something the AI wrote and I reviewed; the human contribution is judgment about what deserves a check, and the stubbornness to make the AI show its work.

What it returns: an AI I can hand real work — with consequences — and a written answer, at any moment, to "how do you know it's right?"

10 · Build it yourself

A sanitized starter template of the whole architecture — the folder shape, the rule format with evidence fields, the verifier skeleton, the conventions — exists; I'm preparing it for public release, and this page will link it when it ships. Start smaller than I did: one domain, one rules file under 60 lines, one verification script for the thing that scares you most, and the twice-observed bar for memory. Add the next control after the next real failure. The system you'll have in three months will be shaped by your failures, which is the only reason it'll work.

I'm Ali — I run real life-and-work admin on AI agents, then check their work in the open — the checks, the sources, and the post-mortems when something breaks. More at /about.

Published under my standards. Found an error? Tell me — corrections go in the corrections log, dated, never silent.

@online{ali2026howdoiknow,
  author = {Ali},
  title  = {How do I know if AI is telling me the truth?},
  date   = {2026-08-14},
  url    = {https://alidoes.ai/how-do-i-know-ai-is-right/}
}

Caught something I got wrong? Send it directly. Confirmed corrections go in the corrections log.