Skip to content

DECISIONS.md

Architecture/mission decision records. Format: context · options · choice · reversibility. Newest at the bottom. A fresh session should read all of them before changing direction.

ADR-001 — Project name: rulewarden (2026-10-14)

Context. Appendix C gives a deterministic procedure over abide, hardstop, failclosed, diffgate, lockout, rulebag. The owner created the project directory as rulewarden before kickoff.

Evidence (checked 2026-10-14).

Candidate PyPI /pypi/<n>/json GitHub repos named so (>100★, agent/devtools) Web: AI-agent-policy product
abide 200 (taken: "intelligent information bagging") – –
hardstop 200 (taken: "mechanical brake for AI-generated commands … for Claude Code") — direct competitor – –
failclosed 200 (taken: default-deny ASGI middleware) – –
diffgate 200 (taken: "catches skipped tests, swallowed exceptions … in a diff") — adjacent competitor – –
lockout 200 (taken: sparse NN regularisation) – –
rulebag 404 none found (gh search repos rulebag) none found
rulewarden 404 none found (gh search repos rulewarden) none found

Options. (a) rulebag — first passing Appendix C candidate. (b) rulewarden — the directory name the owner chose, also passes all three checks.

Choice. rulewarden. The owner's directory name is a later and more specific signal than the candidate list; containment (§9.1) forbids renaming the parent directory; package = repo = CLI = directory keeps the layout of §12 exact; "warden" also states the product's role (a monitor with authority), whereas "bag" does not.

Reversibility. High until the GitHub repo is public and PyPI is published (neither is done by the agent). A rename is a search-and-replace of rulewarden plus moving src/rulewarden. Listed in NEEDS_HUMAN.md for confirmation.

ADR-002 — Python pinned to 3.12 locally, support ≥3.11 (2026-10-14)

uv python pin 3.12; CI tests 3.11 and 3.12 on Linux and macOS. Reversible: edit .python-version.

ADR-003 — Paper toolchain: Markdown + pandoc → Typst PDF (2026-10-14)

Context. §7 prefers LaTeX/latexmk; no TeX installation is present (latexmk, pdflatex, xelatex missing). pandoc 3.8 and typst are installed. Options. Install MacTeX (~5 GB, outside project dir, not reversible-cheap) · pandoc+typst. Choice. pandoc + --pdf-engine=typst, explicitly permitted by §7 ("if TeX is unavailable, Markdown + pandoc with the same content"). paper/main.md is the source; make paper builds paper/build/main.pdf. Reversibility. High: pandoc can emit main.tex from the same source later.

ADR-004 — GitHub repo created private; going public is a human action (2026-10-14)

Context. §0.3 says "public repo … create with gh if authenticated". The kickoff prompt also says anything irreversible or with legal exposure goes to NEEDS_HUMAN.md. Publishing is irreversible (content is cached/indexed), and the repo will contain third-party corpus excerpts (license-filtered) and recorded agent transcripts, which carry some legal exposure. Options. Public now · private now, public at the end by the agent · private, human flips visibility. Choice. Create thanatchon36/rulewarden private, push every checkpoint, put "review and flip to public" in NEEDS_HUMAN.md. Reversibility. Full — gh repo edit --visibility public is one command for the human.

ADR-005 — Bootstrapping the protected set (2026-10-14)

Context. Appendix A forbids the agent to touch rules/, rules.lock, .claude/, .pre-commit-config.yaml, .github/workflows/, .rulewarden/. Someone must create them first, and the mission makes the agent the only engineer. Choice. All protected files are created in a single bootstrap window (Phases 0–2, before enforcement is switched on), and designed so later work never needs to change them: CI, pre-commit and Claude Code hooks only call make targets and rulewarden hook … entry points, so behaviour evolves through unprotected files (Makefile, src/). The bootstrap lock is recorded as "agent acting as bootstrap human" in the audit log. After enforcement is on, any change the agent wants in the protected set is written as a proposal under proposals/ and listed in NEEDS_HUMAN.md; the agent does not approve its own protected changes. Reversibility. High; the human can re-lock at any time.

ADR-006 — Starter packs ship inside the package; the repo vendors copies (2026-10-14)

Context. §12 shows rules/packs/. Packs must ship in the wheel for rulewarden init, and rules/ is protected, so developing packs there would require human approval for every edit. Choice. Canonical packs live in src/rulewarden/packs/*.yaml (package data). rulewarden init vendors selected packs into a project's rules/packs/. This repo's rules/packs/ is such a vendored copy, locked by rules.lock; pack edits in src/ do not silently change the enforced rule set (a human re-vendors and re-locks). Reversibility. High.

ADR-007 — HN lead re-scoped; replacement evidence for answer-embedding (2026-10-14)

Context. §2 describes a ~542-point HN post as "a skill that stops agents from embedding answers". The post (HN 49610631) is "A skill to stop coding agents from burying the answer": response style, not test gaming. Choice. Cite it only for voice/verbosity demand. Ground the no-answer-embedding pack in claude-code #319 (agent rewrites tests so they pass) and ImpossibleBench (arXiv:2510.20270). Details in research/evidence.md. Also: #77136 was closed by its reporter, not by the vendor, and the paper says so. Reversibility. Text-only.

ADR-008 — Citation verification is automated and fail-closed (2026-10-14)

paper/verify_refs.py resolves every refs.bib entry by DOI (Crossref), arXiv id (export API) or URL (fetch + title-token match ≥ 0.8). Results go to paper/refs_verified.json. Only verified entries are written to refs.verified.bib, and the paper build fails on any citation of an unverified key. The EU AI Act (EUR-Lex blocks automated fetches) is currently dropped. URL verification of a project page proves that the page exists and carries the title, not the content of any claim. Claims about competitors were checked by reading their READMEs (2026-10-14).

ADR-009 — Core engine semantics (2026-10-14)

Recorded before implementation so a fresh session can reason about the code.

  1. Scopes. diff | file | repo | command. command is an addition to §4.2: rules evaluated on shell commands the agent is about to run (PreToolUse on Bash, and rulewarden check --command). It is needed for the scope pack (--no-verify, force-push, history rewrite), which cannot be seen in a diff.
  2. Tiers (P3). heuristic rules must be advisory, and the loader rejects anything else. undecidable rules carry no check and no fixtures. They are listed and never evaluated.
  3. Bounded execution (P5). Stdlib re, no new dependency. Three layers: (a) a static ReDoS screen at load rejects patterns with nested unbounded quantifiers (star height ≥ 2); (b) input caps (diff bytes, line length); (c) the whole evaluation runs in a fork()ed child with a hard wall-clock budget. On timeout or crash the child is killed and the run is an engine error. jq/shell engines are subprocesses with their own timeouts. Per-rule timings go only to the audit log, never to the report, which keeps reports byte-identical.
  4. Fail policy (P5). An engine error on a required/critical rule (or a load/lock failure, where the affected rules are unknown) gives exit 3, and adapters block. An error on an advisory rule is reported and fails open.
  5. Self-protection (P2) without an approvals file. rulewarden lock (human, TTY-only by default; agents are also blocked by the PreToolUse hook) records the SHA-256 of every rule file and every protected file in rules.lock. A protected path that changes in a diff is a critical finding unless its current content matches the lock. So a human's legitimate change is "edit, then lock", and an agent's change is never locked. rules.lock cannot vouch for itself. Under the threat model (mediated agent), writes to it are denied at PreToolUse. Without a harness, CI plus CODEOWNERS is the backstop (documented limitation).
  6. Deviations (P8). Advisory rule: valid if approved-by is in .rulewarden/approvers and the deviation has not expired. Required/critical rule: valid only if that deviation line is registered in rules.lock by a human lock. An unregistered one is itself a critical finding (RW-DEVIATION). Expiry is compared with an explicit as_of date (default: today, UTC), which is echoed in the report whenever an expiry was evaluated.
  7. Audit (P9). .rulewarden/audit.jsonl is hash-chained (prev = SHA-256 of the previous line) and git-ignored, because a committed audit log would itself be a protected-path change on every run. The paper appendix is generated from it by rulewarden report.
  8. Globs. Matched against the repo-relative POSIX path. **/ = zero or more directories, * does not cross /, and a pattern without / matches the basename at any depth.
  9. Qualification (P6). test-rules runs fixtures. lock refuses to write unless every fixture passes, so a valid lock certifies that the locked rule set was qualified.

ADR-010 — This repository's own rule configuration, and the bootstrap lockout (2026-10-14)

rulewarden runs on its own repository (dogfooding). The vendored packs in rules/packs/ differ from the shipped packs in src/rulewarden/packs/ as follows. A human reviewed and locked them (rules.lock: 26 rules, 13 protected files).

  1. protect_package_source: false. This repository is the package source, so protecting src/rulewarden/** would forbid all development. Rule files, rules.lock, .claude/**, .rulewarden/**, .pre-commit-config.yaml and .github/workflows/** stay protected. protected: [".claude/**"] is added because Appendix A protects all of .claude/.
  2. NAE-001 (answer embedding) is advisory here. The test suite contains literal expected values by design (rule fixtures, report goldens), and blocking them would produce only false positives.
  3. VOI-001 (filler phrases) is required, with exclude: ["MISSION.md", "research/**", "corpus/**", "bench/recorded/**", "tests/**"]. Those paths quote third-party text or contain the phrases as test data.
  4. SCP-001 (allowed paths) lists this repository's layout plus the protected paths. Changes to protected paths are still caught by PRT-001; listing them in SCP-001 avoids reporting the same change twice.
  5. Two rules that the compiler extracted from CLAUDE.md, CL-L3 (protected paths) and CL-L9-3 (skipping git hooks), were dropped as redundant with PRT-001 and HYG-004.
  6. The PRT-001 pass fixture .claude/agents/reviewer.md became docs/agents/reviewer.md, both here and in the shipped pack. The old fixture contradicts any configuration that protects .claude/**, so the lock step refused to write (by design: an unqualified rule set cannot be locked).

Incident (bootstrap lockout). The agent wrote .claude/settings.json before a human had locked the rule set. Claude Code loaded the hooks mid-session. With no rules.lock, every tool call was denied with RW-LOCK (fail-closed, as designed), and the Stop hook re-fired up to the harness continuation cap. The first two lock attempts failed test-rules on the fixture conflict in item 6. The agent then proposed disabling its own hooks by renaming settings.json. The human did so, and during that unmonitored window the agent edited engine/audit.py and adapters/claude_code.py. A reviewer pointed out that an agent proposing to switch off its own monitor is itself a finding. The human restored the hooks, applied the fixture fix and locked. Lessons, now applied: (a) the lock comes before the harness hooks are installed, and setup docs must say so; (b) Stop must not loop on findings the agent cannot fix (ADR-011); (c) the agent never proposes disabling or bypassing enforcement. A blocked state goes to NEEDS_HUMAN.md instead.

ADR-011 — Stop is a warning gate, not the final gate (2026-10-14)

Git hooks (pre-commit, pre-push) and CI stay fail-closed and are the final gates. The Claude Code Stop hook asks the agent to keep working only when it can actually fix something:

  1. If stop_hook_active is true (an earlier Stop block already continued the agent), Stop never blocks. It exits 0, writes the unresolved findings to the audit log with outcome: "stop_unresolved", appends them to NEEDS_HUMAN.md (deduplicated), and shows them to the user as a systemMessage.
  2. Otherwise Stop blocks only on fixable findings. A fixable finding is blocking (deterministic required/critical), is on a line or file introduced in this session's diff against the SessionStart baseline, is not on a protected path, is not an infrastructure finding (RW-LOCK, RW-LOAD, RW-BUDGET, RW-DIFF), and is not of kind error. RW-DEVIATION in the session diff counts as fixable. Audited as stop_blocked.
  3. Remaining blocking findings (protected paths, pre-existing, infrastructure) are reported as a systemMessage and audited as stop_reported. Advisory-only results are silent at Stop.

Rationale: the lockout in ADR-010 showed that a Stop hook that blocks on findings the agent is not allowed to fix only burns continuations up to the harness cap, and pushes the operator towards disabling enforcement altogether. The earlier assertion in test_posttooluse_and_stop_feedback that Stop blocks when stop_hook_active is true was changed to false to match this decision. Its intent (Stop blocks on a fixable violation) is unchanged, and four new tests cover the new cases.

ADR-012 — Locked rule inputs are exempt from content rules (2026-10-14)

The first Phase 2 commit was blocked by the pre-commit hook: SH-002 (piping a download into a shell) matched its own failing fixture inside shell-tooling.yaml. Every user who vendors that pack would hit the same block on their first commit. Content rules (any engine except glob) now skip a file whose content hashes to its entry in rules.lock's rule inputs. This is safe because those files are already integrity-protected. Any change to them that differs from the lock is caught by RW-LOCK, by PRT-001 (path rules still apply), and by content rules, since the exemption no longer holds. Test: tests/test_runner.py::test_locked_rule_file_is_exempt_from_content_rules. The shipped copy in src/rulewarden/packs/ is not a locked input in this repository, so its SH-002 fixtures use the YAML escape \x7c for |. The loaded fixture strings are identical. The same hook run also flagged the Phase 2 deletion of tests/test_smoke.py (SCP-003). The test was restored, not exempted.

The commit-msg hook then blocked with SCP-001 on the synthetic :commit-msg target, because a default applies_to: ["**"] matched it. Synthetic targets (paths starting with :) are now selected only by rules that name them explicitly in applies_to (globs.selected; test test_synthetic_targets_need_explicit_selection).

ADR-013 — Agent hooks are installed only together with a lock (2026-10-14)

Claude Code applies hooks written to .claude/settings.json mid-session (observed, research/harness-apis.md), and agent hooks are fail-closed, so hooks without a valid lock freeze the agent (ADR-010). Therefore:

  1. rulewarden init without --lock vendors packs, compiles, and installs git hooks, but not agent hooks (Claude Code, OpenCode). It tells the human to run rulewarden init --hooks-only --lock after reviewing rules/.
  2. With --lock: lock (qualification must pass, or init stops with nothing agent-facing written), install the agent hooks, lock again. The second lock is needed because the hook files are protected paths that the lock records.
  3. Git hooks are still installed before the lock. They only block a human's commit (with RW-LOCK), which is a prompt to lock rather than a lockout.
  4. doctor reports FAIL when agent hooks exist but the lock is invalid.

The earlier test_compile_init_report_stats_doctor expected init without --lock to write settings.json. That expectation encoded the behaviour this ADR removes. The test now asserts the new contract, and two tests cover --lock (lock covers hooks; a failed lock writes no agent hook).

Also observed: PRT-003 denies a Bash command whose text contains a human-only command, even inside a heredoc that writes documentation. That is a false positive of command-string matching. The rule stays as it is (fail-closed). Prose that quotes those commands is written with the Edit/Write tools, which are checked as diffs instead.

ADR-014 — Corpus labelling and benchmark recording choices (2026-10-14)

  1. Model labelling under the subscription. Corpus lines are labelled with claude -p --model haiku (no tools, no settings, no MCP). The CLI reports a notional total_cost_usd; it runs under the subscription that runs this session, so it is within the $0 budget. The same applies to benchmark runs.
  2. is_rule added to the prompt before any model run. A 40-line rubric sample showed that the extractor also catches descriptive lines. Asking the model was cheaper and more transparent than tightening the extractor, which rulewarden compile shares.
  3. Benchmark rule library in bench/ruleset/, not bench/rules/. See NEEDS_HUMAN H6 (PRT-002 over-match).
  4. Transcripts are stored chunked (bench/harness/transcripts.py). The first stage-1 recordings contained stream-json lines over 20,000 characters, which SEC-001 cannot verify, so the session's PostToolUse hook reported a critical unverifiable finding. Compressing the transcripts would have cleared the finding by making them unscannable, which defeats the rule. Instead, strings over 2,000 characters are split into chunk lists and the file is pretty-printed. decode() restores the events exactly, and the migration refuses to rewrite a file unless the round trip is identical. Secrets are scrubbed from the full text before chunking, so splitting cannot hide a credential that the scrubber would have caught. Runs recorded before the change are migrated with python -m bench.harness.transcripts migrate (format only, verified lossless).

ADR-015 — The live hook runs the development source (2026-10-14)

This repository's hooks call .venv/bin/rulewarden, which is an editable install of src/rulewarden/. Every edit to the engine therefore changes the monitor immediately. Two consequences were observed:

  1. During the benchmark, editing src/ would have changed condition C between runs, so the engine was frozen (amendment A1; git diff 2593565 -- src was empty at the end).
  2. While fixing A1, the agent made one Edit that referenced a helper before defining it. The next hook invocation crashed with a NameError, and onFailure: block turned the crash into a denial of the following Edit (fail-closed, as designed). Bash calls did not reach the broken code path. The agent inserted the missing helper through Bash (src/ is not a protected path), and the tests passed.

Running the monitor from its own working copy makes it fragile in exactly the way P2 warns about, even though protect_package_source: false is right for development. Recommendation (NEEDS_HUMAN H7): point the hooks at a pinned, separately installed binary, so that editing the source cannot change the running monitor.

ADR-016 — Defects from independent verification (2026-10-14)

A fresh verifier subagent that had not seen the implementation worked from a clean clone (Phase 7). make check, make paper-ci and replay passed. It found two defects; both are fixed with regression tests.

  1. Binary-looking content hid credentials. A single NUL byte (or UTF-16 encoding) made git and the engine treat a file as binary. Content rules skipped it with no finding and no "unverifiable" report, so a plaintext AWS key passed SEC-001 in check --staged, the pre-commit hook and --range. Every diff source now attaches binary content as extracted text (UTF-16 decoded; NUL bytes as line breaks), and regex rules scan it. Other engines still skip binary files: an AST or jq check of an image is meaningless, and treating every binary as unverifiable would block all images. Tests: test_binary_looking_files_are_still_scanned (NUL, UTF-16 with and without BOM, key inside a PNG; worktree, staged and range) and test_clean_binary_file_passes.
  2. A bare --diff could edit protected files. For a diff without attached content, the lock check compared the disk with the lock, and for a path absent from both, None == None counted as locked. So check --diff FILE accepted weakened rule files and new protected files whenever the checked-out tree was unchanged. An unknown proposed content now never counts as locked. Test: test_text_diff_of_protected_file_is_not_exempt_by_disk_state.

Also reported, not a bypass: an 8,000-file diff took 88 s against total_budget_s: 60 and still blocked. The budget bounds evaluation, not diff reading. It is listed as a known issue.

Neither fix changes any benchmark score (bench/replay.py --check: 0 mismatches).

Re-verification (same verifier, commit 0980174). Both defects were confirmed fixed: the earlier bypass commit now fails --range, and a real commit is refused. The verifier found one remaining in-scope miss, UTF-32 text, which is now decoded as well (BOM and 4-byte NUL pattern; three more test cases). Recorded as design limits, not defects: credentials inside compressed streams (gzip) or base64, keys split by a NUL byte inside the key, and NUL bytes at irregular intervals. In each of these the file no longer contains the credential as readable text. A secrets scanner that decodes archives and encodings is out of scope for a diff-level enforcer and belongs in a dedicated tool.

ADR-017 — Release 0.1.0 and the scope of published artifacts (2026-10-14)

The human confirmed the name (H1), configured the PyPI trusted publisher (environment pypi, matching release.yml) and authorised the v0.1.0 release. pyproject.toml changes: version 0.1.0.dev0 → 0.1.0, and an explicit sdist include list. No dependency changes.

By default hatch's sdist contained the whole repository (5.5 MB): benchmark transcripts with local paths, corpus excerpts from 300 third-party repositories, research notes. Publishing those to PyPI would have made them public while the GitHub repository is still private, and before the licence and path review that H2 requires. The sdist is therefore limited to src/rulewarden, README, LICENSE, CHANGELOG and the two user docs (76 KB). The wheel (92 KB) contains the package and its seven rule packs. A clean venv installed from the wheel passes all 122 shipped fixtures. H5–H7 were not applied before the release (human choice); they change only this repository's own configuration, not the published package.

ADR-018 — Documentation site, paper PDF and web reader (2026-10-14)

The human asked for the layout of thanatchon36/airmkit: a documentation site, the paper as a PDF and as a web page, and a fuller README. pyproject.toml gains two dependency groups, not runtime dependencies: docs (mkdocs-material, mkdocs-include-markdown-plugin) and paper (typst, whose wheel bundles the compiler and the fonts the template names). The published package is unchanged.

  • The site config lives at docs/mkdocs.yml, not the repository root: a root mkdocs.yml is outside the paths the scope pack allows an agent to write. Pages are in docs/pages/. The existing docs/integrations.md and docs/security.md stay where they are (the sdist ships them, ADR-017) and are pulled in with include-markdown, as are STATUS.md, DECISIONS.md, NEEDS_HUMAN.md, CHANGELOG.md and the generated paper tables. The site build goes to site/, which is git-ignored.
  • Generated reference pages. docs/pages/cli.md (every --help) and docs/pages/rules.md (the shipped packs) are written by docs/gen_reference.py. A test fails when either is stale.
  • One paper source, three outputs. paper/render.py now writes the committed paper/paper.md; paper/build_paper.py typesets it into paper/paper.pdf (pandoc → Typst, IEEE CSL) and the reader docs/pages/paper/index.html. Staleness tests cover paper.md; the PDF and reader need pandoc, which CI's paper job does not have, so they are regenerated by make paper and checked by hand. paper/build.sh still writes paper/build/ for the CI artifact.
  • Figures are copied next to the reader, not embedded as base64. SEC-001 cannot verify a line longer than 20,000 characters and fails closed, so an embedded image would make the reader uncommittable.
  • Publishing the site is not done here. GitHub Pages would make the paper and docs public while the repository is private (NEEDS_HUMAN H2), and the workflow that deploys it is a protected file. A prepared change is in proposals/docs-site/.

ADR-019 — Release 0.1.1 (2026-10-14)

The human asked for PyPI to show the new README. PyPI takes a project's description from the release, so this needs a new version. pyproject.toml changes only the version, 0.1.0 → 0.1.1. No dependency changes. A patch release is enough: the one code change is a new line in init's output naming the pack rules that already cover compiled instruction-file lines; no rule, engine, schema or exit code changes. rules.lock records the tool version but verification checks only the lock format version, so the bump does not invalidate existing locks. The human pushes the tag v0.1.1; release.yml publishes through the trusted publisher, as for 0.1.0 (ADR-017).