Skip to content

πŸ€– vault-watch-agent

What it is for

The guide gives minimum versions, prices and security vulnerabilities to watch. This information changes constantly: a recommended version can become dangerous overnight. A collection without AI records the changes; the agent determines which ones make a page wrong and prepares the correction work.

What it does, step by step

PhaseWhat happensWho works
CollectionFetches the watch: the GitHub issue and its release notesProgram
TriageSorts each item: to correct, possibly to add, of no interestAI
LocationFinds the French pages concerned and groups what belongs togetherProgram
ProposalDrafts the corrections in parallel, with current text, new text and sourceAI
NumberingChecks that each quotation really exists in the pageProgram
Cross-checkRe-reads each correction with the whole guide and the web: keep, revise, rejectAI
ReviewPause: the author chooses the corrections to applyHuman
ApplicationApplies the corrections on a separate branchProgram
TranslationUpdates the corresponding English pagesAI
ChecksRuns the guide’s audits and tests the external linksProgram
PublicationPause: the author authorizes the pull requestHuman

The review screen

Each correction appears as a card: the current text, the proposed text, the severity and the source. Related corrections, such as a sentence and its footnote, are approved together.

vault-watch-agent review screen: ticked corrections, with the current text in red, the proposed text in green and the source.

What else it does

A watch follows sources that are already known, and it only corrects. Four other kinds of work go through the same review and the same pull request:

WorkWhat happens
Fixing a reported errorAn error noticed by a reader or by a check is verified against primary sources, then corrected
Checking the guideOn demand: dead links, consistency between pages, fidelity of the translation, dated claims to re-check
TrimmingWith every addition, the pages grow heavier. The agent lists what is repeated or out of date and proposes to shorten it, without removing a fact, a figure or a source
Looking for what is missingThe agent lists what a reader would look for and not find. Each lead that is kept is checked on the web, then written in the page where it belongs

A lead is a question to check, not a fact: the author chooses which ones to follow, and sees what the search will cost before starting it.

vault-watch-agent leads screen: three leads ticked, each with the passage of the guide it is about, and the estimated cost of the search.

Its safeguards

  • The model proposes, the code decides. The AI never writes to a file. A correction is refused if the text it quotes does not occur exactly once in the page.
  • Two human approvals. One after the report, the other before anything leaves the machine.
  • Evidence rather than memory. Proposals rely on release notes and on primary sources consulted on the web.
  • One source of rules. The agent loads the guide’s editorial rules on every run.

How good it is, measured

The agent was replayed on two watches processed by hand before it existed, then its proposals were compared with the human corrections. No proposal that reached the review was wrong, but it missed many: on the busiest watch, it found 2 corrections out of 15. The substantive change of that watch was found; 9 of the 13 missed corrections only concerned a date. On 18 November 2026, the replay of a more recent watch found the 11 merged corrections, with no error. That watch had been processed by the agent itself, then reviewed: this replay measures its consistency, not an independent truth. The full evaluation is public.

As of 20 November 2026, the agent has prepared 19 of the merged pull requests of this guide, all reviewed by the author before merging.

Its limits

  • Markdown in a Git repository. First written for this guide, the agent now keeps any Markdown guide described by a small configuration file. Other formats are not supported.
  • It does not write a new page. It adds to existing pages. When a subject needs a page of its own, it says so, and writing it remains the author’s work.
  • Its quality is only measured with Claude. This guide is kept with Claude models, a cloud service, for public content only. The agent can also use a local model, but the quality of its corrections is not measured there.
  • It remains an assistant. It never pushes directly to the main branch.

For developers

The project is written in Python with LangGraph for the orchestration. The model calls go through the Anthropic SDK, through Claude Code, or through any server compatible with the OpenAI API. The steps form a state graph, saved at every step in SQLite: a paused run can resume days later, from the web interface or the command line.

The French guide fits entirely in the model’s context thanks to prompt caching. There is therefore no vector RAG: at this size, reading everything costs less than building an index, and the model sees inconsistencies between pages.

GitHub repository: DamienBecherini/vault-watch-agent, under the MIT licence.

Give my opinion

Everything is optional: one answer is enough. This site sends nothing: you send it, the way you choose.

Your noteNo note
Your opinion is about
About what? (several choices possible)
What will be sent
Send by e-mail

Private, no account: opens your mail program with the message ready, for ia-on-prem@damien.becherini.fr.

Open a GitHub issue β†—

Public, with a GitHub account. You read the issue again before you publish it.

To paste it wherever you want.

Give at least a note, a subject or a message to be able to send.