← All writing

Build log

The AI swept 30 vendors. Could it account for all 30?

Aug 14, 2026

My AI swept roughly 30 turnover vendors and reported nothing missed — it had read the top of a ranked list and guessed about the rest. The audit that replaced guessing with counting found the winning bid in a missed thread.

Not at first. The first sweep read the inbox in relevance order and inferred the rest was noise. An exact-count audit caught two missed threads — one held the winning bid.

The job: source paint and floor vendors for a unit turnover at a small rental property. An AI ran read-only sweeps of the property mailbox and the voice line that forwards into it, covering roughly 30 solicited vendors, and scored them in a weighted spreadsheet, vendor-scorecard.xlsx. The claim I now treat as radioactive came out of that first sweep: "0 known misses." It sounds like a count. It is a feeling wearing a count's clothes — it means "I found nothing," and says nothing about what was never opened.

What did the first sweep get wrong?

It trusted the platform's sort order. On 2026-06-18 the sweep read the top 50 marketplace threads in most-relevant order, not date order, and the log later that day recorded "0 known misses." The audit's own write-up is blunt about the cause: the pass inferred the remainder from the ranking and never opened every direct email.

That line was true of every thread the AI had opened, and unverifiable about the inbox. Relevance ranking is the platform's opinion about what deserves attention. It is built to hold attention, not to guarantee coverage. Treating it as a completeness instrument was the actual bug, and the AI committed it politely and confidently.

What does an exact-count audit look like?

The 2026-06-19 self-audit re-ran the whole window as date-bounded queries reconciled to header counts. Reconcile — force your line-by-line list to equal an independent total, here the mailbox's own header count, so nothing can hide between the two. The audit enumerated 67 of 67 marketplace threads and 37 of 37 direct emails, gave every item a disposition (what it is, where it went), and ended at 0 unaccounted.

It caught two misses from the day before. One was cheap: a decline from a painter. The other was material: a formal written estimate that turned out to be the lowest licensed bid for the flooring trade. That vendor went on to win the actual award. The best bid in the project spent a day invisible because a ranked list decided it wasn't interesting.

A follow-up re-sweep on 2026-06-21 ran the same way: 2 of 2 new marketplace threads, 28 of 28 direct items, 0 unaccounted, and the scorecard recalculated with 0 formula errors. The method held on a quiet day too. That is what makes it a method.

Why open every quote-bearing thread?

Because counting threads is necessary and not sufficient. One thread's preview snippet cut off mid-sentence at a ballpark phrase; opened in full, it held a five-figure all-in price. A snippet-level sweep would have carried that vendor at a fraction of its real number. So the rule is two-part: reconcile to the count, then open every thread that might contain a price.

The audit also labels what it scraped, in a per-source confidence table. Quotes taken from the emails themselves are high confidence, and a license confirmed in the state's contractor registry is high confidence where found. Scraped star ratings are directional only, because review platforms block bots. And the wording that matters most in an owner-facing deck: "no license found" is a verify item, not proof of unlicensed. Five parallel research agents ran those registry cross-checks, and their output got confidence labels, not blanket trust.

The leftovers stayed visible too. A handful of phone numbers never matched any vendor. They are flagged for the owner to identify rather than silently dropped, because an honest sweep has a remainder column.

What got frozen into the playbook?

The reusable vendor-evidence playbook now encodes the discipline: ask every candidate for the same scope and terms, verify credentials through the current authoritative registry, and record unknowns as NOT CAPTURED. Its exact words: "a rating or model summary is not proof." The sweep rule froze in the QA log rather than the playbook: no completeness claim until the header reads N of N with 0 unaccounted.

This is the same species of failure I hit in my wedding vendor tracker: "looks complete" and "is complete" are different claims, and only a check can tell them apart. It is also why the broader system I run treats detection as the whole game — nothing stops an AI from inferring; everything guarantees the inference can't stay invisible.

Details here are generalized to protect the people involved: no vendor names, no prices, no license numbers.

So look at your own setup. Where does an AI read the top of a ranked list and report on the whole? And what independent number would its "nothing missed" have to reconcile against before you believed it?

I'm Ali — I run real life-and-work admin on AI agents, then check their work in the open. More at /about.

Published under my standards. Found an error? Tell me — corrections go in the corrections log, dated, never silent.

@online{ali2026vendorsweep,
  author = {Ali},
  title  = {The AI swept 30 vendors. Could it account for all 30?},
  date   = {2026-08-14},
  url    = {https://alidoes.ai/ai-vendor-sweep-exact-count/}
}

Caught something I got wrong? Send it directly. Confirmed corrections go in the corrections log.