Models are temporary, evidence is not

Context

AlanOS keeps a lot of my life in files: medical PDFs, transactions, mail, meeting transcripts, a spoken diary, my career history. Models read all of it and write summaries, readings and answers on top.

Models keep getting better, so the question I kept coming back to was: when the better model arrives, what should it be able to rebuild, and from what?

The answer became the first principle of the system. Models are temporary. Anything a model produced is a reading, and a reading can be thrown away and done again. The material it was read from cannot.

The model

Every domain holds two kinds of thing, in two separate folders.

Evidence is what happened, kept exactly as it arrived, in three shapes:

• A document is the original bytes, addressed by the hash of its content, so the address changes if a single byte does.
• A record is the one transcription of a document into text. It carries the document's hash and is the only thing downstream reads. A PDF is read once, at transcription; its information has not changed, so a better model has no reason to read it again.
• Reported evidence is something I am the source of, such as a symptom I describe. There is no document behind it, so it has one layer.

Knowledge is everything compiled from evidence: summaries, links, weekly briefings, the reading of a day. Delete it, compile again, and it comes back, ideally better.

Two rules hold the line. Evidence is immutable: a correction is a new item, and a store of evidence only grows. And evidence does not count until I confirm it. A confirmation binds to the content's hash, so an item edited afterwards is unconfirmed again, and unconfirmed evidence counts for nothing.

What I started noticing

Most of my "knowledge" was not knowledge. My career domain had a folder of about a hundred and seventy entries called knowledge. By the system's own definition, knowledge can be deleted and recompiled. So I asked what would happen if I did. Ten certificates would come back. Almost everything else would be gone for good.

I had written those entries from memory, which makes them reported evidence, the kind that can never be recompiled. They moved into the evidence layer, I confirmed each one, and the compiler now rebuilds the knowledge entries from them byte for byte. The move was done safely: compile into a side folder, compare, and never delete first.

Some evidence needs no transcription. A medical PDF needs a record, because a model should reason over text, not guess from an image. A bank's transaction data is already structured, so a transcription would only add room for mistakes. The rule changed to say so: machine-readable data can stand without a record, read by a deterministic, versioned parser.

Some evidence nobody composed. Mail fetched from a provider, a meeting transcript, a recorded working session: no person and no model wrote them, so confirming each one would be theatre. They are sealed instead, with their hash recorded the moment they arrive.

Anything a model writes from it is knowledge again.

The rule I would not bend

Nothing in an evidence folder is ever excluded from version control, in any domain. Keeping sensitive files out of the repository is the usual reflex, and it trades risk for possible loss. The repositories are private, deployed images are tested to exclude evidence, and no remote surface serves it. As I put it when I made the rule: we can manage risk, we can't manage something we lost.

Why this mattered

• Changing models is cheap. When I switch the model behind a domain, the evidence stays where it is and the knowledge is compiled again.
• Every answer has a floor. A claim traces back to confirmed or sealed evidence, or it does not count. That makes an answer checkable rather than merely plausible.
• Mistakes are recoverable. A bad compiler run or a wrong summary is undone by deleting knowledge and running again.

Trade-offs

• Confirming takes my time. Sealing removes the routine cases, but everything a person or a model composed is a queue I have to clear.
• The boundary takes judgement. As the career folder showed, whether a file is evidence is not always obvious.

What I would keep doing

Ask of every file: if I deleted this and ran the compiler again, would it come back? If yes, it is knowledge and it can be cheap. If no, it is evidence, and it deserves a hash, a confirmation or a seal, and a place in version control. Then let the models improve as fast as they like, because nothing they write is the only copy of anything.