alidoes.ai

All writing

Build log

I pointed AI at a professional's report. It found the carried-forward assumptions.

A credentialed analyst delivered a finished reserve study to an HOA board. An AI-assisted audit found old assumptions riding inside it — then caught its own reading error too.

Key findings

  • The tell for a carried-forward assumption is boring: the source field says “Previous Study”, a replacement cost equals a recent invoice, or a finished project's clock was never reset.
  • Every challenge had to pass a six-criterion rubric with a Ctrl-F evidence test: open the cited document and find the claim, or the challenge dies.
  • The audit also caught the AI itself: its PDF table extraction lost row association, so one claim was ruled unciteable until a human reread the original.

You can point AI at a professional's report and find real problems — if every challenge must survive an evidence test, and the AI's own reading gets checked the same way.

In May 2026, an HOA where I then served as board president received its updated reserve study: the long-range plan that says what the shared parts of a property will cost to replace, and when. A credentialed analyst prepared it. Formatted, professional, authoritative. And the most dangerous numbers in it weren't wrong. They were old. A professional's default is not a finding; it's a placeholder with a signature, and defaults are exactly what gets carried forward between study cycles.

The scale made checking worth it. Between the prior cycle and this one, the study's percent-funded figure moved from 25% to 47%, its fully funded target roughly doubled, and the total replacement cost it models grew about 60%. Volunteer boards adopt numbers like that with a nod, because nobody wants to argue with a professional. The study landed 2026-05-09; the site walk with the analyst was 2026-05-13. In the three days between, I built an apparatus that could argue on the record, with sources.

How a carried-forward assumption travels into a funding plan, and the audit gate that intercepts it A stale assumption's path, and the gate that stops it FAILURE PATH Prior study sets a value useful life, cost basis → New study copies it source: "Previous Study" → Funding plan adopted stale inputs, real dues caught: invoice booked as recurring cost, clocks never reset after finished projects AUDIT GATE 5-question audit capital vs recurring, life, cost, drift, layer split → 6-criterion rubric Ctrl-F test: open cited doc, find the claim, or it dies → Sourced challenge memo proposes only two inputs: useful life + today's cost
The path a carried-forward assumption travels (top) and the audit gate that intercepts it (bottom). Diagram source: this page; maps 1:1 to the 5-question framework and the 6-criterion rubric's Ctrl-F evidence test.

What does a carried-forward assumption look like?

It looks boring. The framework I captured on 2026-05-10 asks five questions of every component: Is this a capital addition or a recurring replacement? What useful life (the years a component is expected to last), from which source? What replacement cost? What did the analyst change between study cycles? Should one bundled line split into layers?

Those questions surfaced three specific tells. A source field reading "Previous Study" means the line hasn't been re-evaluated in years. A replacement cost exactly equal to a recent invoice means a one-time material upgrade was booked as a recurring cost. And a remaining useful life never reset after a completed project means a brand-new component entered the study already aged.

The useful-life question carries a source hierarchy worth stealing. The manufacturer's warranty is the floor, not the estimate; expected service life typically runs 30-50% past warranty; and an installer's workmanship warranty never sets useful life on its own. Bundling matters too: one line blending a 40-year structure with a 10-year coating gets a blended life that quietly shortens the structure. The audit proposed splitting those lines, and eight vendor evidence-request letters went into draft to turn verbal claims into written warranty terms.

How do you challenge a professional without just arguing?

With a rubric that can kill your own challenge. Every proposed revision had to clear six criteria: legal compliance, the prudent-person fiduciary standard, evidence, clean procedure, financial modeling at 10 and 30 years, and defensibility on the public record. The evidence criterion has teeth. Its test is literal: open the cited document, Ctrl-F for the claim, and if you can't find it the challenge dies, however plausible it sounds. Same discipline as the four gates on my wedding tracker: no source, no entry.

Two design choices kept the memos honest. Reserve software computes annual contribution as replacement cost divided by useful life, so a challenge proposes only those two inputs and lets the analyst's own tool produce everything downstream. Proposing output numbers just muddies the conversation. And a delta analysis priced each defensible revision per year: the conservative package came to about 5% of the recommended annual contribution, everything-accepted to about 12%. The delta file's own verdict: "real but not transformational." The bigger lever was which of the three funding plans the board chose, and the audit's job was to make that choice defensible instead of hopeful.

The apparatus also produced a do-not-challenge list before it produced challenges. The analyst's inflation and interest assumptions were industry-typical, and the three-year update cadence was correct. An audit that fights everything is advocacy with a spreadsheet.

Where did the AI itself fail?

In the funding-plan table. The AI's PDF text extraction lost row association (which label belongs to which row of a table). The extracted text showed a "Recommended funding plan" header with no provable link to any of the three plans beneath it. The rubric's own evidence criterion flagged that claim as unciteable: a human had to reread the original PDF before anyone cited it.

That catch matters more to me than any single revision. An AI that audits a professional but exempts its own reading isn't doing verification; it's laundering defaults into "verified." The system this page belongs to runs on the same premise: the checker gets checked.

What keeps the next study from rotting the same way?

Assumptions carry forward because nothing records when the clock restarts. Two artifacts close that loop. ul-reset-log.md is append-only (new entries only, nothing edited or deleted): one entry per completed project, naming the reserve lines it touches and the date their useful-life clock restarts. reconciliation-script.py reads a machine-readable header (YAML frontmatter) from each project ledger and cross-references the master component inventory. Reconciliation here means comparing records that should agree and flagging where they don't: completions missing reset entries, one-time capital items sitting in recurring reserves, double-count risks, orphan lines no project owns. It was designed to run quarterly and before each analyst engagement.

Its first drift report, dated 2026-05-18, is my favorite receipt in this whole system. 0 of 7 project ledgers had adopted the header schema (the agreed field layout) yet, and 3 inventory lines had no owning project. The first drift the reconciliation caught was its own adoption gap, and it reported that as a table of failures rather than a green check.

Details here are generalized to protect the people involved: no names, no dollar figures, no building identifiers. The mechanism is exact.

Your version of this probably isn't a reserve study. It's an inspection report, an appraisal, an actuarial letter. If you Ctrl-F'd its source fields tomorrow, how many would say some version of "Previous Study" — and which of your numbers is quietly aging inside one?

Method & data

Method: an HOA reserve-study challenge cycle — the 5-question component framework, the 6-criterion decision rubric, the per-revision delta analysis, and the reconciliation script's first drift report, generalized
Data: patterns and methods only; no names, dollar figures, addresses, case identifiers, or confidential content · Last checked: 2026-08-14

How this was made

AI-drafted, adversarially checked, human-directed. My AI assistant wrote this from the system's own records — the 2026-05-10 component-analysis framework, the challenge rubric and charter, the component delta analysis, `ul-reset-log.md`, `reconciliation-script.py`, and its 2026-05-18 drift report. A separate AI session then tried to break every claim against those records, and automated privacy and readability gates ran before publish. I direct this pipeline, own every boundary in it, and audit published pages on a rolling basis — if you find an error, tell me and it goes in the corrections log, dated, never silent.

I'm Ali — I run real life-and-work admin on AI agents, then check their work in the open. More at /about.

Published under my standards. Found an error? Tell me — corrections go in the corrections log, dated, never silent.

Cite this

@online{ali2026reserveaudit,
  author = {Ali},
  title  = {I pointed AI at a professional's report. It found the carried-forward assumptions.},
  date   = {2026-08-14},
  url    = {https://alidoes.ai/ai-audit-of-a-reserve-study/}
}

Caught something I got wrong? Send it directly. Confirmed corrections go in the corrections log.