Prose Is Not a Control

Two of my tools passed every test they had. Then they copied a private passage from my journal into two new files, under headings that invited it to become a permanent claim about me.

Nothing was broken. Both tools did exactly what they were written to do.

The rule that should have stopped them was a sentence in AGENTS.md. It said, plainly, that some parts of my life are lived, not modelled. Neither tool reads AGENTS.md.

That is the whole lesson, and it took me a bad afternoon to learn it. A rule in prose is not a control. It is a wish with good posture.

A sign is not a guard

I spend a lot of my working life around fulfilment: sortation, transport, delivery, the machines and people that move parcels. Nobody on a sort floor protects hands from a conveyor with a laminated sign. The sign is there. It matters. But the thing that keeps fingers out of the nip point is a fixed guard and an interlock that stops the belt when the guard opens.

Safety people have a name for this. The hierarchy of controls ranks how you manage a hazard, from most effective to least:

1. Eliminate the hazard.
2. Substitute something safer.
3. Engineer a barrier between the person and the hazard.
4. Use administrative controls: procedures, training, signs.
5. Hand out protective equipment.

A paragraph in a prompt file is an administrative control. It sits on the fourth rung, just above safety glasses. Lean people would call the better version poka-yoke: design the step so the mistake cannot happen, like a plug that only fits one way.

On logistics sites, physical guardrails and barriers usually do more for safety than signage. Signs wear away over time, or someone has messed with them, and sometimes that only shows up in an audit. On older sites especially, it is not uncommon to find a sign missing altogether. The focus on safety is still high, and the physical guards are still in place. The sign wears away. The guard stays.

The odd part is that leaders who would never accept a sign as the only guard on a conveyor will happily accept a paragraph as the only guard on an agent that can send email.

What I changed

I did not write a sterner paragraph. I moved the rule down the ladder.

The fix is a small library function that the journal tools have to call before they copy anything into a derived file. If a passage belongs to a part of my life I have marked as out of scope, the function refuses the copy.

It does not hide anything. The passage stays in the journal, where I wrote it. The digest still tells me how many passages were held back and points me at the entry. I get pointed at my own words instead of having them lifted out of context.

Then I wrote evals that fail if the guard goes soft. The first version of the guard passed my review and failed its own test. It missed a sentence that carried the whole meaning of the passage and did not contain a single word on my list.

People write messily. A keyword list works on tidy prose and fails on the human kind. The second version checks relationships as well as topics, and it is deliberately broad.

That breadth has a cost, and I chose it on purpose. If the guard over-matches, one sentence I could have quoted stays in my journal, where I can still read it. If it under-matches, a private moment ends up in a machine-readable file that outlives the day that made sense of it. Those are not equal risks.

The rungs, for agents

Here is how I now read my own setup, top rung first.

Eliminate. The capability does not exist. If a tool must not send email, it has no mailer and no credential. This site is built that way. Adding an edition does nothing until I run the deploy myself. There is no path from a draft to the public page that skips me.

Engineer. Where a capability has to exist, the guard sits in the tool path, one function call before the bad action, with a test that goes red when someone narrows it later. Someone, in practice, is often me, tidying a rule I have forgotten the reason for.

Administer. Prose still earns its place. AGENTS.md is how people agree on the why, and it is where I find out what to test. It just does not stop anything by itself.

Protect. The last line is a person reviewing the output. It catches things. It is also the rung that fails first when you are tired.

What this means at work

Most organisations now have an AI policy. Many of them are good documents. A document is still the fourth rung.

When I read one now, I ask three questions of every line that says never:

• Which rung actually enforces this?
• Who owns that control?
• What test or audit would show us it has stopped working?

If the answer to the first question is "the policy", the control is a sign on the conveyor. That can be fine for low harm. It is not fine for sending, spending, publishing, or moving private data.

Where prose still wins

I am not against writing things down. I bang on about writing things down. Prose is how a team learns why a boundary exists, and a guard with no written reason gets deleted by the next person who finds it annoying.

There is also a failure on the other side. A guard that refuses everything is not a guard. It is an off switch, and people route around off switches. My evals check both directions: the private passages must be refused, and ordinary sentences must still get through.

So write the rule. Then ask where it lives. If it lives only in the prose, you have written a wish.

Which never in your setup would hurt most if it failed tomorrow, and which rung is it sitting on today?