# Per-site memory off switches only work if every reader asks the same authority

> Governing an agent's page memory took three enforcement points, a soft delete with a dry run, and a live test that found four defects the unit suites missed.

- Author: Chad Priest
- Published: 2026-08-27
- Canonical URL: https://blog.vodou.ai/per-site-memory-policy-enforcement/
- Tags: ai-agents, memory, architecture, privacy, observability

---

If your agent reads the browser tab, you already have a policy problem, whether or not you have written the policy. The extension sees a bank login page the same way it sees a Wikipedia article. The memory store does not know which facts came from a health portal. And the day a user asks "forget everything you learned from that site", you find out how many code paths actually read that data, because each one needs to honor the answer.

I shipped the governance layer for Vodou's page memory across two days in August. This is what it took, including the parts the tests said were fine and were not.

## Off, suggest, collect: the mode lives in the gateway, not the extension

The capability is small to describe. Every host gets a mode: off, suggest, or collect. Banks, health portals, tax sites and sign-in hosts default to off, which means the extension never reads the tab there until the user flips it. Everything else collects. Precedence is one line: a user rule beats the sensitive default, which beats the global default, and a rule covers subdomains.

The part that matters for your stack is where the mode is decided. It is not in the extension. The gateway is the authority, exposed as `GET/PUT /api/page-match/site-mode` and `GET /api/page-match/site-modes` in `MCP-servers/Vodou-Console/src/page-site-mode.ts`. The extension asks before it does anything with a page. Typing suggestions ask the mode first and send nothing until it answers. That ordering is the whole design: the client is not trusted to remember the policy, it is required to fetch it.

**Diagram: One authority, three enforcement points**

Three read paths for page memory, each gated by one site-mode lookup in the gateway before any daemon call

```text
  [Tab URL] --> [site-mode lookup]
  [site-mode lookup] --> [page-match]
  [site-mode lookup] --> [page_probe]
  [site-mode lookup] --> [note / link]
  [page-match] --> [memory daemon (fixed)]
  [page_probe] --> [memory daemon (fixed)]
  [note / link] --> [memory daemon (fixed)]

  notes:
    site-mode lookup: user rule > sensitive default > global
    page-match: off = empty, no daemon ask
    page_probe: off = no green icon
    note / link: 403 unless collect

  Every path that touches page memory consults the same mode before it reaches the store.
```

Three paths read page memory: the match endpoint that fills the side panel, the probe that colors the toolbar icon, and the note and link routes that write a fact with a page attached. All three now check the mode. Off means the match returns empty without asking the daemon at all, the probe returns no icon, and a write returns 403 unless the mode is collect. Those are three separate checks because they are three separate readers, and I tested them separately: four tests for the mode resolution, four for enforcement, in `MCP-servers/Vodou-Console/src/__tests__/page-site-mode.test.ts`.

## Forget a host: soft delete, dry run first, confirm within eight seconds

The second half of governance is retraction. `mem forget --host <h>` sets `invalid_at` on every fact whose source host is the host or a subdomain of it. It does not delete rows. Library documents saved from that host are counted and reported, not removed, because a saved article is something the user chose deliberately and I did not want one command to take two kinds of data.

The panel exposes it as "Forget site…" with a two-click shape: the first click runs a dry run and shows the count, the second click within 8 seconds confirms. `--undo` clears the `invalid_at` stamps back. I verified the loop live on w3schools.com: forget hid one fact, the count read 1, undo brought it back, the count read 0.

**Diagram (beforeafter)**

Before: a page fact could only be removed by a hard delete with no preview. After: forget by host is a soft invalidation with a dry run, a count, a confirm window and an undo

```text
  BEFORE: Before P4
    - no per-host retraction
    - delete meant a hard row delete
    - no preview of what would go

  AFTER: After P4
    - forget --host sets invalid_at, subdomains included
    - first click = dry run with count
    - second click within 8 s = confirm
    - --undo reverses it; documents counted, not removed
```

Soft delete has a cost that a hard delete does not: every reader must filter on `invalid_at`. That is the same shape as the site-mode problem. A governance decision is only as good as the number of readers that honor it.

## The unit suites were green and the live page found four defects

Before P4 landed I loaded build .74 unpacked and ran it against real pages: ChatGPT with the consent gate open in DevTools, a Wikipedia clip, a saved article, the Ctrl+Shift+B regression. Every unit suite had passed. Four things were wrong anyway.

A fact I had stored 30 minutes earlier displayed as "18h" old. The by_page query returns `COALESCE(valid_at, created_at)`, and a fact that came out of a daily log has its `valid_at` set to the log day at local midnight. The age math was correct and the input was a day-granular instant. Day-granular instants now render as today, yesterday, or N days.

"Send selection to Vodou memory" sent no URL. The two Wikipedia clips I made landed with an empty `source_url`, which means they could never be found from that page again. The selection was the only content the handler passed along. It now sends the tab's URL and title as well.

The lane the "Add this page to Vodou Library" action actually uses, `add_text`, never wrote `memory_sources.source_url` and never stamped its chunks. Only `add_url` did. Two entry points to the same table, one of which stamped provenance. I mirrored the behavior and stamped the one existing document by hand so it matched.

And once that article was stamped, its 111 chunks flooded the "From this page" list as raw `# Title\n\nSource:` fragments and buried the four facts I had clipped. The fact tiers now exclude `doc:%` sources and a separate `docs_by_page` lists each document once.

None of these four are exotic. Each one is a reader or a writer that did not participate in the provenance contract the others followed. The tests were green because each test exercised the path that did participate.

## The icon took four rejected drafts in one evening

The smaller build story is the toolbar icon, which turns green when the current page has memory from it. The first draft was a text badge; Chrome's fixed badge overlay ate most of a 16px icon. The second was a corner dot. The third was a pulse, which reads as an alert that never resolves. The fourth, the one that shipped, redraws the icon on an OffscreenCanvas from our own artwork, hue-shifting only the brand-blue pixels so the white eyes survive. A flat `source-atop` fill had painted over them.

The decision about which signal drives the color was harder than the drawing. Counting site-tier matches swallowed the icon on every page of a site. Counting title-only semantic hits dotted every Google results page. The icon goes green for exact facts and documents saved from this page, nothing broader. The probe result is cached for 30 seconds per page key, so a burst of tab switches is one round trip. That logic is in `MCP-servers/Vodou-Console/src/vbb/page-probe.ts` with five tests.

## The invariant: readers of a governed column equal checks of its policy

Here is the failure class, stated so you can check it rather than nod at it.

For any column that a policy governs (`source_host` under a site mode, `invalid_at` under a soft delete), the set of code paths that read or write that column must equal the set of code paths that consult the policy. If there are N readers and N minus one policy checks, the policy is decorative on exactly one path, and that path is the one the user will hit.

The four defects in .74 all violate a sibling of this: two writers of `memory_sources`, one stamping provenance. Three readers of page memory, before P4, zero consulting a mode. The property is countable.

## Run this against your own store before you trust your off switch

Take five minutes and count. First, find every place that reads the governed column, and every place that reads the policy. Substitute your own names.

```bash
# readers of the governed axis
grep -rn "source_host\|source_url" src/ --include='*.ts' --include='*.py' | grep -v test | wc -l
# places that consult the policy
grep -rn "siteMode\|site_mode\|isHostAllowed" src/ --include='*.ts' --include='*.py' | grep -v test | wc -l
```

Passing looks like the second number being close to the first, and a quick read of the diff between the two lists showing only helpers. Failing looks like `14` and `2`, which is roughly what I would have seen before P4.

Second, check whether your writers agree on provenance:

```sql
SELECT source_kind, COUNT(*) AS rows,
       SUM(CASE WHEN source_url IS NULL OR source_url = '' THEN 1 ELSE 0 END) AS no_url
FROM memory_sources
GROUP BY source_kind;
```

Passing is a `no_url` of 0 for every kind that came from a page. Failing is one kind with rows and no URLs, which tells you which writer skipped the stamp. That query is how I would have caught `add_text` in seconds instead of on a live Wikipedia tab.

Third, if you soft delete, prove every reader filters:

```sql
UPDATE memories SET invalid_at = datetime('now') WHERE source_host = 'example.com';
```

Then hit each of your read endpoints for that host with curl and look for a nonzero count. Any endpoint that still returns those rows is a reader that did not get the memo. Undo with `SET invalid_at = NULL` on the same rows.

## What the memory-pattern guides leave out

The published specs describe memory in tiers. The [agent memory pattern spec](https://geodocs.dev/ai-agents/agent-memory-pattern-spec) names working, episodic, semantic and procedural, and its privacy rule is to redact PII at write time. The [engineering playbook](https://engineering-playbook.vercel.app/agentic/agent-memory) splits the same space into episodic, semantic and procedural. Both are right and neither says where the fact came from. A site mode is a rule about the source, not the tier, and write-time redaction cannot fire on a page the extension should never have read in the first place. The off mode has to sit in front of the read, which is a different enforcement point than the redactor.

Retraction is under-described too. The [dependency-guided rollback paper](https://arxiv.org/html/2608.10502) makes the point that deleting a faulty memory leaves everything derived from it active. My `forget --host` has the same limit: it invalidates the source facts and does nothing about a fill plan or an answer that already used them. That is a real gap, not a footnote, and the paper is the clearest statement of it I have found.

## Still open: a chunk that names its own page can undo a manual link

One limitation is live. `mem page-link` stamps a memory with a page by writing the column directly, and sync's `COALESCE` keeps that value on re-index. But a chunk whose own text carries a `page:` token reverts to that token's page on the next sync, overriding the manual link. Two writers of `source_url` again, with a precedence rule that favors the wrong one for this case. I noted it in the P2 commit and did not fix it in P4. If you count your writers with the query above, that is the kind of pair it will show you.

---

Source: [Per-site memory off switches only work if every reader asks the same authority](https://blog.vodou.ai/per-site-memory-policy-enforcement/) by Chad Priest, from Building Vodou in Public.
