The patch that caught nothing
An audit had just handed me a list of holes in the safety config around an autonomous agent, the kind that runs while I’m asleep. The first one looked trivial to close, so I closed it. My fix blocked nothing.
The hole was a delete that slipped past the deny-list inside a shell wrapper, bash -c 'rm -rf .'. I added a deny rule with a wildcard, Bash( rm -rf ), and it looked responsible sitting in the config. Then, because I’d spent the previous week learning to distrust things that look responsible, I tested how the matcher actually treats it. The patterns anchor at the start of the string, so a leading * matches nothing.
// looks like a guardrail. enforces nothing.
"deny": ["Bash(* rm -rf *)"]
// patterns anchor at the start, so a leading * matches zero commands,
// and `bash -c 'rm -rf .'` still walks straight through.My patch parsed, sat in the config looking like a gate, and would have blocked exactly zero commands. I had written a wish and filed it as a rule, which was the entire thing the audit had just been about. I had reproduced the bug inside the fix for the bug.
Finding the holes, it turned out, was the easy half. Every patch I reached for wanted to fail the same way: on an assumption about the mechanism that I had not checked.
A fix is a claim too
The audit’s rule was simple. Before you call something a gap, check it isn’t a decision. The fixes taught me the other half of it. Before you call something a fix, check it actually does the thing.
A patch rests on claims. This pattern matches that input. This hook runs on that event. This flag changes that behaviour. Each one fails quietly when it’s wrong: a deny rule that matches nothing still parses, and a hook wired to the wrong event still loads. The config reads as more governed than before, you get the small hit of having done something safe, and none of that is evidence that the input you’re scared of would now get caught.
The next patch I reached for already existed. I went to add a pre-push check and found the repo already had one, wired in and working, scoped on purpose to the part that mattered. Had I added mine, I’d have shipped a second, slightly different gate doing the same job worse, then spent a fortnight confused about which one fired. Read the mechanism before you extend it.
The patch after that was a one-line change: remove a trailing || true I was sure was swallowing a failure. I removed it, re-ran, got the identical result. The || true wasn’t on the line that decided anything; the real exit code came from elsewhere. A no-op dressed as a fix is worse than no fix, because now it’s ticked off the list and nobody looks again.
The bypass that turned out to be a decision
One of the guards printed its own escape hatch in the error message: here is how to get around me. I read it as a leak and moved to delete it. Then I found the dated note explaining the override was deliberate, left in for the one authorised case where a human genuinely needs it.
The audit had drilled me on checking whether a missing control was a choice. This was the same trap from the other side: an existing control I was about to rip out because I hadn’t read why it was there. “Is this an oversight or a decision” cuts both ways, and the fixing direction is the dangerous one, because deleting feels like tidying up.
The fixes had their own bugs
Before any of this landed, I did the thing the whole week had been shouting at me to do. I had something that wasn’t me try to break it: two other models, pointed at my patches as adversarial reviewers, with one instruction. Assume the fix is wrong, find how.
They earned it on the first pass:
- The mid-session kill switch, whose entire job is to halt the agent the instant I drop a stop-file, blocked file edits and shell commands but not the agent’s ability to spawn a sub-agent. A halted agent could open a fresh one and keep going through it.
- The delete-blocker I’d just written false-flagged
git rm, which is reversible and index-only. In a walk-away run that means a job wedged at 3am for no reason. - The stop-file check looked only at the main checkout, blind to the parallel worktrees the agent actually runs in, so the kill switch wouldn’t have fired in the place it most needed to.
Every one of those was a real bug in a control I’d written specifically to be careful. The control existed, it was wired in, and it would have passed any “do we have a kill switch?” checkbox on any maturity model you like. It just didn’t work, and I couldn’t see it, because I wrote it and wanted it to be right.
Microsoft is framing the same problem from the runtime-security end: its 2026 Agent Governance Toolkit ships an automated kill switch among its controls. Installing one was never my problem. Getting mine to fire where the agent could actually act was.
The knob that lied
The last item wasn’t a guardrail, it was a setting, and it gave the cleanest version of the lesson. The agent kept running its context window up to the ceiling before doing anything about it. Annoying, and I was sure I’d fixed it months ago by setting an override.
I had. The override was sitting in the config file, with a sensible value, doing nothing. The tool reads that variable from the shell environment, not from that file, and the file’s copy still gets handed to subprocesses, which is just enough to make a quick check print the value and convince you it’s live. It looked set. It was ignored. The real fix was one line in the shell profile instead. The docs I found, plus a couple of blog posts, each described the setting differently and disagreed about what the number even meant. The only thing that settled it was watching what the running system did.
The config said one thing, the docs said another, my own quick check said a third, and the live behaviour was the only one telling the truth.
What I’d tell the next person
The audit half of this is well covered now: don’t call something a gap until you’ve checked it isn’t a choice. The fixing half deserves equal billing, because it’s where I quietly torched my own confidence. Four things I keep coming back to:
- A fix is a claim. Before you tick it off, name the exact input you’re afraid of and watch the gate catch it. Green on good input proves the gate runs, not that it guards.
- Read the mechanism before you touch it. The gate you’re about to build might already exist, the line you’re about to delete might do nothing, and the rule you just wrote might match nothing.
- A control that exists is not a control that works. Most of all the ones you wrote yourself and trust without watching them fail.
- Have something that isn’t you try to break each fix before it ships. A second model told “assume this is wrong” beats your own review, because you are the worst-placed person alive to find the bug you just wrote.
None of this is clever. It is the same small move on repeat: trust the running system over the description, even when the description is your own patch. The audit was the part I was good at. The fixes were where I found out what I’d actually built.
The audit prompt that started all this is here, and the post about pointing it at my own project is here. This is what came after the list of holes: the half where you find out whether you can actually close one.
Comments & Reactions
Got a thought, a war story, or a “well, actually”? Sign in with GitHub and jump in.
Loading comments…