knowledge/extraction

6 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Knowledge extraction must only ratchet up, never delete facts

A focused re-read regression showed deletion-first rereads let a narrower extraction lose facts (80% fell to 60%). Rereads now keep old notes, with dedup superseding near-duplicates and distinct facts surviving. Focus is spelled out as in-addition-to, never instead-of. The prompt version joins the cache key so a prompt fix is never answered from the cache that predates it.

Empty extraction result is correct, not a failure

The prompt tells the model an empty list is correct. extractFromText throwing on empty lists caused stub and index pages to fail and retry forever instead of landing as ready with zero notes. Only an all-segments failure throws now. An empty page is a valid answer.

Repairable extraction slips repair automatically, not fail the whole segment

Smaller extraction models occasionally return a kind outside the enum or confidence past 1. The strict schema used to fail the whole segment. Repairable slips now repair: unknown kind becomes fact, lengths clip, confidence clamps. Title and body stay strict — a note without those is not a note.

Extract specification tables losslessly, preserving every field and its type

Never summarise a table into prose. Keep interface specifications verbatim in notes: every field with its exact name, type, whether it is required, and its default. A long note that preserves the table beats three short notes that describe it, because agents ask 'show me an example' constantly and a paraphrased snippet does not run.

Long text must be split into segments, not truncated

Text extraction sent `text.slice(0, 400_000)` in one call. A 726 KB specification page was ingested as its first 55% and reported as ready, with nothing anywhere recording that the rest existed. Silent data loss looks like a complete ingest. Split long text into 60 KB segments on blank lines and extract in passes, merging the notes. A segment that fails does not take the others with it. A page where every segment fails still throws, so it is marked failed rather than stored as a partial.

Extract categories from the exam, not from accumulated notes

Extraction was being handed the categories already on notes, which drifts from the categories the brain is actually measured against. It gets the exam's labels first now — those come from the goal, which is the authority.

Give your agent this brain