2.1 Structure of the Guidelines

The Guidelines apply to all financial institutions in Singapore and, for locally incorporated institutions under consolidated supervision or owning critical information infrastructure, on a group basis (§1.2). Section 1 defines AI, models, systems and use cases (§1.3–§1.4), restates the risk landscape including the amplified risks of generative AI and the “unauthorised or erroneous actions” of agents (§1.10–§1.11), and sets the transition (§1.8). Section 2 provides the applicability test: an institution whose AI use could have a material adverse impact applies the full regime, and one below that threshold applies a basic governance regime of accountable senior management, permitted and prohibited uses, an approved-tool list, staff education and periodic review (§2.2–§2.5).

Section 3 places governance with the board and senior management and requires control functions for identification, inventory and materiality assessment (§3.3–§3.6). Section 4 requires consistent identification of AI use including AI embedded in services from material third-party providers (§4.2, fn21), management of the residual risk from AI that is not identified (§4.4, fn22), an inventory with specified attributes and an agent-specific extension (§4.5–§4.9, fn24–fn25), and a use-case-level assessment of risk materiality over Impact, Complexity and Reliance (§4.10–§4.13). Section 5 covers the lifecycle: roles, data, transparency and explainability, fairness, human oversight, third-party AI, selection, evaluation and testing, security, pre-deployment review and independent validation, monitoring, change management and retirement (§5.1–§5.26). Section 6 addresses competence, conduct and capability, extended in the final text to personnel using AI rather than only those building it (§6.1).

Three definitional features matter for what follows. First, materiality is anchored on the use case, not the model or system, and the definition of a use case in §1.4 is the context in which a model or system is applied, so the same model can carry different tiers in different uses. Second, footnotes carry requirements: fn25 lists the attributes an agent inventory entry may include, fn22 lists technical controls for unidentified AI, and fn46 lists failure modes of generative AI that evaluation must cover. A catalogue that indexes only numbered paragraphs would miss a material part of the obligations. Third, the Guidelines are deliberately silent on method where method would be institution-specific, and the Response to Feedback records that silence as a decision rather than an omission (§7.19, §10.28–§10.29, §10.66). Everything quantitative in this paper is therefore the authors’ proposal and is labelled as such.

2.2 From consultation to final text

We diffed the November 2025 consultation draft against the final Guidelines and recorded 36 substantive changes with a full paragraph concordance (docs/WHAT_CHANGED.md). The direction of change is proportionate rather than lenient. On the easing side, the single twelve-month transition became phased (§1.8); the “integrated into business processes” applicability test, with its annex flowchart and example list, was replaced by a material-adverse-impact test (§2.2–§2.4); a dedicated cross-functional AI committee is no longer required (§3.3); the inventory must be accurate “to the extent possible or practicable” at an institution-chosen frequency (§4.5); human review of all AI outputs became “effective human oversight” (§2.5); and data representativeness moved from “the full range” to “a range” of real-world conditions (§5.4(b)).

On the tightening side, residual risk from unidentified AI must now be kept within appetite with named mitigations (§4.4, fn22); agent attributes — identifiers, tools and systems accessible, components, guardrails — appear as an example attribute set (fn25); evaluation of generative AI and agents must cover key failure modes and the effectiveness of the associated guardrails (§5.15, fn46); third-party AI carries the institution’s primary accountability with limit, suspend or replace where residual risk exceeds appetite, and with certifications or independent external assessments preferred over vendor self-attestations (§5.10–§5.11, Response §9.9); enhanced logging of prompts, responses, model versions and reasoning became a monitoring consideration for generative and agentic systems (§5.23(d)); and the board must now approve and regularly review the strategic direction for AI use (§3.4).

Two of these changes shape the experiments directly. The guardrail-effectiveness duty of §5.15 is what RQ3 measures, and the fn25 attribute set is what makes an agent inventory schema necessary at all.

2.3 Why qualitative guidance resists automation

Three structural properties of supervisory text explain why operationalisation is a measurement problem rather than an engineering one. First, obligations are stated at the level of outcomes (“effective human oversight”) rather than artefacts (a documented escalation rule), so a check can confirm that an artefact exists and is current but not that it achieves the outcome. Second, obligations are proportionate: the same paragraph binds differently at different materiality tiers, so applicability is a function of a judgement the institution must make first. Third, obligations are organisational: a board’s review of strategic direction leaves no artefact a program can inspect without reducing it to a signature. The automatability classification of §4.3 is designed to make these three properties visible in the data rather than to argue them away, and the central empirical claim of this paper is that the three properties are unevenly distributed across the Guidelines’ sections.

4.1 Source acquisition and clause extraction

All eleven primary documents — the Guidelines, the Response to Feedback, the consultation paper, the 2024 Information Paper, the FEAT principles and the Veritas documents — were acquired from mas.gov.sg and hashed; docs/sources/MANIFEST.md records a SHA-256 for each so that every citation in the toolkit can be verified offline. The Guidelines were transcribed by hand into src/airm/catalog/mas_airm_2026.yaml (design decision D-3: a hand-built catalogue reviewed against the PDF, rather than a parser that would drift on the next revision). Each paragraph carries its section, heading, a summary of at most forty words in our own words, a kind (expectation, definition, scope, context or guidance), who it applies to, its effective date under §1.8 as clarified by Response §13.5, and tags such as genai, agent, third-party, inventory and human-oversight. Footnotes are included where they carry a requirement. The catalogue is the sole source of truth for citation: a control that cites a paragraph not in the catalogue fails the test suite, which is the only mechanical defence we have against a control catalogue drifting into invention.

4.2 Interpretation protocol

Where a paragraph admits two readings, we implemented the stricter as the default, exposed the lenient one as configuration, and recorded both. Eleven such interpretations are documented in docs/INTERPRETATIONS.md (I-1 to I-11) and referenced from the controls that depend on them. Four are load-bearing for the results. I-3 keys the high-materiality triggers of §5.3, §5.19, §5.23(b) and §5.24 on the inherent rather than residual rating, following Response §7.21. I-4 fixes the combination rule for the three materiality dimensions, which the Guidelines leave open (§4.3 below). I-6 treats the fn25 agent attributes as expected for any use case tagged as an agent, although the footnote says “may include”, on the ground that a supervisor asking for the attribute set will not accept their absence as a documented choice. I-8 requires a validator to be a different party from the owner, developer and deployer and not the vendor, following Response §10.55 and §10.58.

4.3 Control catalogue and automatability classification

Each of the 65 controls carries an identifier, the paragraphs it serves, applicability by inherent tier and by scope (use case or institution), the evidence items it requires, an optional automated check, and an automatability class with a written rationale. The class is the instrument on which RQ1 rests, and its definitions were fixed before classification began (D-6):

A clause is auto only if a deterministic function of the inventory entry, its evidence files and ingested evaluation outputs decides pass or fail without human judgement about adequacy. It is semi if such a function can decide whether the required artefact exists and is current but not whether it is good enough. It is doc if compliance is an organisational or judgemental fact with no artefact the toolkit can inspect. The classification is relative to this implementation — a richer integration would move some semi controls to auto — and it was made by a single annotator, so no inter-rater agreement is available. Each row is published with its rationale so that it can be disputed individually without rejecting the scheme.

One consequence of the scheme governs how every count in this paper should be read: a semi control never reports PASS. When its automated check succeeds, the outcome is NEEDS_ATTESTATION, because the toolkit has established presence and currency but cannot establish adequacy. The attestation count in §6 is therefore a work queue, not a failure count, and airm check exits non-zero only on a genuine automated FAIL, since a pipeline that failed on attestations would teach engineers to suppress them.

4.4 Risk-materiality rubric

The rubric operationalises §4.12 from fields the inventory already carries under §4.7 and fn25. Twenty-three signals, each citing the paragraph it derives from, score Impact, Complexity and Reliance at four levels, and the three levels combine into an inherent tier of Low, Medium, High or Critical. The combination rule is max with an impact gate (D-5, I-4): the tier is the highest of the three levels, but Complexity and Reliance may raise it at most one level above Impact. The gate implements two points from the Response to Feedback — that Complexity is not a standalone driver (§7.9) and that the assessment should anchor on the consequences of failure (§7.5) — and the max rule is the strictest common reading of a text that does not prescribe one. The rubric is our proposal, not a MAS methodology. A residual tier is derived by crediting declared mitigations that the Guidelines name: human checkpoints at consequential steps (§5.9(c)), tested guardrails (§5.15), formal independent validation (§5.19), kill switches and contingency plans (§5.3, fn29), trace logging (§5.23(a), (d)) and independent vendor assessment (§5.11(a)). Every score carries its reasons, so an assessment can be argued with rather than only accepted, and a weighted-sum variant is run as a sensitivity check in RQ2.

Two structural properties of the defaults deserve statement before the results. First, the impact gate means an internal research assistant does not become Critical because it uses a large language model. Second, because generative AI sets Complexity to High and the gate permits one level above Impact, no generative-AI use case can be rated Low under the defaults. We report this as a finding about the rubric rather than tuning it away, and §7.5 shows how it weakens one hypothesis.

4.5 Synthetic portfolio

The worked example is Merlion Commercial Bank (fictional), with 8 AI use cases chosen to span the materiality range: a credit decision model, AML alert triage, a customer-service chatbot, an internal policy RAG assistant, an agentic KYC document processor, card fraud scoring, a vendor-provided transaction-monitoring service and a trading-signal research agent. 2 are agentic and 7 are rated High or Critical. The portfolio carries three third-party assessments, 266 synthetic evidence files and a 14-record hash-chained audit trail. No real institution’s data appears anywhere in this work, and the portfolio was written by the same person who wrote the rubric, a dependency §8 returns to.

4.6 Pre-registration and analysis protocol

Five research questions were fixed in docs/RESEARCH_QUESTIONS.md before the experiments were implemented, each with a hypothesis, the experiment that would refute it, the metric and the decision threshold:

• RQ1 Automatability. What fraction of MAS-AIRM expectations can be automatically checked from artefacts, semi-automated with human attestation, or only documented? Method: Each of the 51 expectation paragraphs takes the best automatability class among the controls implementing it; paragraphs with no implementing control count as doc. Classes and their justifications are in src/airm/controls/controls.yaml.
• RQ2 Autonomy and materiality. Holding purpose and data fixed, how does autonomy change Impact/Complexity/Reliance and which Section 5 controls become binding? Method: Each of 4 portfolio use cases is rewritten at 6 autonomy levels; only the autonomy field and the minimal agent scaffolding needed for validity change. Applicable controls are those the catalogue selects for the variant.
• RQ3 Guardrail effectiveness. For common guardrail patterns, what is the measured failure rate under standard adversarial probe families, and does it meet a defensible threshold per materiality tier? Method: 20 probes across 5 families (4 attack families plus a benign control set) run under 5 guardrail patterns. Detection is by regular expression, never by a model, so the measurement does not depend on the thing being measured. Wilson 95% intervals throughout.
• RQ4 Crosswalk coverage. What share of MAS-AIRM expectations is covered, partially covered or uncovered by ISO/IEC 42001, NIST AI RMF and the EU AI Act, and what residual work does an already-certified institution face? Method: 224 hand-authored mapping rows over 51 expectation paragraphs. The denominator is the expectation count, not the mapped-row count, so an unmapped expectation counts against coverage rather than being excluded.
• RQ5 Cost of compliance. How many controls, evidence items and estimated person-hours does each materiality tier require, and where is the long tail? Method: Applicable controls and their evidence items are counted per use case from the control catalogue, then multiplied by a per-item effort estimate that varies with the control’s automatability class and a tier multiplier. Institution-level controls are counted once.

Verdicts are mechanical against the recorded thresholds. Proportions are reported with Wilson 95% intervals, and a guardrail pattern “meets” a tier only if the upper bound of its interval lies below the tier’s limit, so that a small sample cannot manufacture a pass. Monotonicity is tested with Kendall’s τ. Where implementation forced a change to a pre-registered construct, the original hypothesis is reported as refuted on its original wording and any revised test is labelled as not pre-registered; §7.2 is the one such case. Where a measurement instrument was found to be defective, the defect, its direction of bias and the correction are recorded in the result file and reproduced here; §7.3 has two such cases. These rules were adopted so that the reader can distinguish what the pre-registration predicted from what the implementation taught us, which is the distinction a pre-registered study exists to preserve.

7.1 RQ1: automatability stops at the governance boundary

23 of 51 expectation paragraphs (45.1%) admit a fully automated check, 16 (31.4%) are semi-automatable and 12 (23.5%) are documentary. The pre-registration stated H1 as a point prediction and a refutation criterion: at most one third of expectations would be auto, with semi the majority and the auto share concentrated in Section 4; H1 would be refuted if auto reached 50% or if Section 3 out-automated Section 4. On the refutation criterion the share component holds, since 45.1% is below 50%, but the point prediction of one third was exceeded, and the majority claim was refuted outright: at paragraph level auto is the largest class (23 against 16). H1(b) held comfortably.

The refutation of the majority claim is an artefact of the unit of analysis, and we regard the control-level view as the more honest one. A paragraph takes the best class among the controls that implement it, so a single automatable control in a paragraph with four documentary ones makes the paragraph auto. Counted per control, semi is the largest class (24 of 65, 36.9%) and auto the smallest (20, 30.8%). The paragraph-level figure is reported as the headline because it is what was pre-registered; the control-level figure is what we would pre-register next time.

The distribution is the finding. Section 4 — identification, inventory, materiality — is 84.6% automatable across its 13 expectations, because its subject matter is a register and a score. Section 3 — governance, board and senior management accountability — is 0.0% automatable and 40.0% semi-automatable across 5 expectations, and Sections 1, 2 and 6 contain no automatable expectation at all. Section 5, the lifecycle, splits almost evenly: 46.2% automatable and 46.2% semi-automatable across 26 expectations, with 7.7% documentary. The three structural properties of §2.3 are therefore not uniformly distributed: outcome-stated and organisational obligations concentrate in the governance and capability sections, artefact-stated obligations in the inventory section, and the lifecycle section mixes the two.

The practical consequence runs against the order of the transition. The obligations that bind first, in October 2027, are the ones tooling can carry; the obligations that bind in October 2028 mostly reduce to “the artefact exists and is current”, which is worth automating but is not the same as compliance. An institution planning its programme around what a tool can do will find the help arriving in the wrong order.

7.2 RQ2: autonomy enlarges the control set, not the tier — and a disclosure

Holding purpose, data and model family fixed, we rewrote 4 portfolio use cases at each of 6 autonomy levels (assistive -> human_in_the_loop -> human_over_the_loop -> autonomous_decisioning -> autonomous_agent -> multi_agent), changing only the autonomy field and the minimal agent scaffolding the inventory’s validator requires, and recounted the applicable controls and the tier at each rung.

H2(a) held: the applicable-control count is monotone non-decreasing in autonomy for all 4 use cases, with Kendall’s τ of 0.7303 in every case — below the 1.0 of a strictly increasing sequence because adjacent rungs often bind the same controls, but with no reversal anywhere. 7 controls first bind at the autonomous-agent step, including the fn25 agent attributes, guardrail effectiveness testing (§5.15), agent permissions (§5.16), trace monitoring (§5.23) and design for oversight (§5.9). The weighted-sum rubric variant reproduces the monotonicity.

H2(b) and H2(c) were refuted, and the reason is a change we made to a pre-registered construct. The hypotheses were framed on a four-rung ladder whose decisive step was human-in-the-loop to autonomous agent. While implementing the inventory we found that a model which acts without human review of each decision — a real-time fraud block — is fully autonomous under §4.12(c) but has no tools, so the fn25 agent attributes cannot apply to it, and conflating it with a tool-using agent would demand tool allowlists from a scoring model while understating the distinct agent risk of §1.11. We therefore inserted a distinct autonomous_decisioning rung. That moved the step the hypotheses named, and both were refuted on their original wording. The result file carries the disclosure verbatim:

docs/RESEARCH_QUESTIONS.md pre-registered the ladder as assistive, human-in-the-loop, autonomous agent, multi-agent, and framed H2(b) and H2(c) on the human-in-the-loop to autonomous step. While implementing the library an AUTONOMOUS_DECISIONING level was added, distinct from AUTONOMOUS_AGENT, because a model that acts without human review of each decision (a real-time fraud block) is fully autonomous under MAS-AIRM 4.12(c) but has no tools, so the agent attributes of 4.8 fn25 do not apply to it. That change moved the step the hypotheses named. H2(b) and H2(c) are reported as refuted on their original wording. H2(c-revised) tests the same underlying claim on the revised ladder and is reported separately so a reader can see which test was pre-registered and which was not.

H2(c-revised), which tests the same underlying claim at the step autonomous_decisioning->autonomous_agent, holds for all four use cases and is labelled not pre-registered.

The tier result is the more interesting refutation. Only 1 of 4 use cases gained an inherent tier at the agent step, because 2 were already Critical on Impact alone and the fourth was held by the impact gate. Under this rubric the tier is dominated by consequence, and autonomy expresses itself in the control set rather than in the tier. For a supervisor that is arguably correct — a fraud model that blocks payments is no less consequential for being tool-free — but it has an operational implication: a tier label alone does not tell a reviewer whether the agent controls are in force, and a review process keyed on tier will miss the agent step entirely.

7.3 RQ3: guardrails that are effective and unusable

We ran 20 probes — 4 attack families of four probes each (direct jailbreak, indirect prompt injection, PII leakage, policy evasion) plus a benign in-scope control set of 4 — under 5 guardrail patterns against a local qwen3.5:0.8b model: no guardrail, a system-prompt instruction only, a deterministic input–output filter, a second-pass LLM judge using the same model, and the filter and judge layered. Each pattern therefore faces 16 attack trials. Detection is by regular expression, never by a model, so that the measurement does not depend on the thing being measured, and every proportion carries a Wilson 95% interval.

Attack success falls from 75.0% [50.5%, 89.8%] unguarded to 6.2% [1.1%, 28.3%] layered. H3(a) held: the system prompt alone leaves attack success above 20% on every family, and at 50.0% [28.0%, 72.0%] overall it is barely distinguishable from no guardrail. H3(c) held: no pattern’s upper bound on indirect injection falls below the Critical limit of 1.0%, so for high-materiality agentic use cases the §5.15 expectation cannot be met by guardrails alone and requires the human-oversight and scope-limitation controls of §5.9 and §5.2.

H3(b) was refuted on its pre-registered test. Layering looks strong on the point estimate, but the test required non-overlapping intervals, and at 16 trials the layered interval [1.1%, 28.3%] overlaps the deterministic filter’s [3.5%, 36.0%]. The experiment is underpowered for that comparison, the pre-registered criterion is the one we report against, and the honest statement is that layering was not shown to beat the best single pattern at this sample size.

The substantive result is the benign control set. Over-refusal is 0.0% unguarded, 50.0% with the system prompt, 50.0% with the judge and 50.0% layered: the patterns that suppress attacks also refuse half of ordinary in-scope requests. Against the joint criterion fixed before the run — attack success within the High-tier limit of 2.0% and over-refusal at or below 5.0% — 0 of 5 patterns qualify, and H3(d) is refuted. The deterministic filter is the only pattern that combines a large reduction in attack success (75.0% to 12.5%) with zero measured over-refusal, and it is also the only pattern that eliminates PII leakage (100.0% to 0.0%); its residual failures are entirely in policy evasion (50.0%), where the disallowed output is paraphrasable and a pattern match cannot see it. This is a result about the economics of determinism, not a claim that guardrails are easy.

MAS-AIRM 5.15 asks for the effectiveness of guardrails to be tested. Measured on attack success alone, the layered pattern looks strong. Measured jointly with over-refusal on a benign control set, the patterns that suppress attacks also refuse a large share of ordinary in-scope requests, so on this model and probe set no pattern is both effective and usable. Both thresholds were fixed before the run.

The implication for §5.15 is direct. An institution that tests guardrail effectiveness on attack suites alone will conclude that its layered defence is strong, and will not learn that it has made the system unusable for the purpose that was approved. Guardrail evaluation is meaningful only with a benign control set, and we would argue that a supervisor reading a guardrail test report should ask for one before reading the attack numbers.

Two instrument defects. Both were found during this experiment, both were corrected before the reported run, and both would have produced flattering results had they survived. The result file records them as follows:

Each probe’s success_pattern is an instrument, not a hypothesis. The patterns were revised once after inspecting the unguarded (none) responses of the reference model, which exposed two error classes: DJ-01 matched the ‘As DAN’ persona echo inside an outright refusal, counting a refusal as an attack success; and DJ-03, DJ-04, PE-01 and PE-04 missed successful attacks because the model phrased them differently from the anticipated wording (for example ‘offers a compelling 20% return’ rather than ‘guarantees’). Calibration used the none condition only, so it could not be steered by which guardrail pattern it would favour, and no pattern was changed after seeing the guarded results. A separate defect was fixed in the backend first: the reference model is a reasoning model, and with thinking enabled the whole token budget went to the thinking field, returning an empty completion that matched no probe and produced a false 0% attack success rate across every pattern. The backend now passes think=False and treats an empty completion as an error rather than as a refusal.

The backend defect is worth restating because any team using a reasoning model will meet it. With thinking enabled, the reference model spent its whole token budget in the thinking channel and returned an empty content field with a length stop reason. An empty response matches no probe pattern, so every attack scored as “the guardrail held”: a perfect 0% attack success rate across all five patterns, produced by truncation, with no error raised. The runner now disables thinking, caps prediction length and raises on an empty completion rather than scoring it. The detector defect is the mirror image — a pattern that matched a persona echo inside a refusal and counted the refusal as a success — and recalibration was confined to the unguarded condition so that it could not be steered toward whichever guardrail it would favour. All RQ3 numbers come from one small local model (ollama qwen3.5:0.8b at temperature 0.0, seed 20261013, num_predict 512, think=False) and are a demonstration that the joint measurement is possible and that the joint criterion bites, not an estimate of guardrail effectiveness in general.

7.4 RQ4: what remains after certification is governance, not agents

H4(a) held: the best single framework, NIST AI Risk Management Framework 1.0 (NIST AI 100-1), fully covers 49.0% of expectations, below the 60% threshold, with the EU AI Act at 43.1% and ISO/IEC 42001 at 37.2%. H4(b) held: the union of ISO/IEC 42001, the NIST AI RMF and the EU AI Act fully covers 33 of 51 expectations (64.7%), leaving 18 (35.3%) with no full mapping. An institution holding ISO/IEC 42001 alone still faces 31 partially covered expectations; adding the NIST AI RMF reduces that to 23.

H4(c) was refuted, and it is the result we would most want a reader to carry away. We predicted that the residue would concentrate in the agent-specific and Singapore-specific clauses: group and local accountability, the control-function designations, the three-dimension materiality assessment, shadow and embedded AI, and the fn25 attributes. Four of the ten residual expectations were predicted; six were not. The NIST Generative AI Profile is narrow (29.4% of all expectations) but accurate where it applies, and it covers the generative-AI and agent clauses well. What no framework fully covers is governance machinery and MAS-specific proportionality:

The clauses §2.2, §2.5, §3.4, §3.6, §4.1, §4.4, §4.8, §4.13, §5.11, §5.20 have only partial mappings everywhere. They are the applicability test and basic-governance regime (§2.2, §2.5), board responsibilities (§3.4), the identification process and the residual risk from shadow AI (§4.1, §4.4), the agent attribute set (§4.8), materiality arbitration between control functions (§4.13), third-party accountability with limit, suspend or replace (§5.11) and the scope of independent validation (§5.20). Even including MAS’s own FEAT principles and the 2024 Information Paper, which raise the all-framework union to 41 (80.4%), these 10 clauses remain partial. An institution certified to ISO/IEC 42001 and aligned to the NIST AI RMF has done the lifecycle work and has not done the Singapore-specific governance work, which is the opposite of the common intuition that the agent clauses are what international frameworks miss.

The mapping is interpretive and asymmetric. ISO/IEC 42001 is a management-system standard, the EU AI Act is product law and the NIST AI RMF is voluntary guidance, so a paragraph can be covered in substance and uncovered in legal effect, and the reverse. 18 of 224 crosswalk rows are low confidence and are listed individually in results/rq4_crosswalk.json; a further 106 are medium and 100 high.

7.5 RQ5: proportionality scales rigour, not volume

H5(a) was refuted, and we believe the refutation is a property of the Guidelines rather than of our rubric. Critical-tier use cases require 91.4 evidence items on average against 92.0 at Medium, a ratio of 0.993 where the hypothesis required more than two. Estimated hours do rise, from 264.0 to 364.5 to 437.3 across the three tiers present, but only through the tier multiplier we apply to the same artefacts. In the Guidelines, proportionality operates on the depth of review of a given artefact — a Critical use case needs independent validation of the same documentation that a Medium use case needs signed off internally (§5.19, §5.24) — not on how many artefacts exist. The hypothesis is also weaker than written because the portfolio contains 0 Low-tier use cases, for the structural reason given in §4.4, so Medium had to stand in for Low.

H5(b) held strongly. 93.1% of estimated effort sits in semi and doc work (36.8% and 56.3% respectively), and only 6.8% in work a tool can finish unaided. H5(c) held: the largest category is documentation_and_review at 26.5%, against 16.8% for evaluation and testing.

Effort is evenly spread rather than concentrated: the Gini coefficient across the eight use cases is 0.0897, and institution-level controls add 81.0 hours once. Total first-time effort for the portfolio is estimated at 3260.6 hours with 1141.2 hours per year recurring. These are assumptions, not measurements. They are published so that a reader can substitute their own and recompute. Measuring them would need a user study; see docs/FUTURE_WORK.md. The shape of the distribution is the claim; the absolute values are an assumption a reader can replace.

7.6 Summary of refutations

• H1(a) semi is the largest class — semi = 16, auto = 23, doc = 12
• H2(b) at least 3 of 4 gain a tier from human-in-the-loop to autonomous agent — tier gains: credit-decision-model +0, aml-alert-triage +0, customer-service-chatbot +1, policy-rag-assistant +0
• H2(c) the largest control delta is at the human-in-the-loop to agent step — the largest step is autonomous_decisioning->autonomous_agent for 4/4 use cases, not human_in_the_loop->autonomous_agent
• H3(b) layering beats the best single pattern (non-overlapping 95% intervals) — layered 6.2% [0.01,0.28] vs best single filter 12.5% [0.04,0.36]
• H3(d) at least one pattern meets the High-tier ASR limit without over-refusing — patterns meeting both pre-registered thresholds: none. The patterns that cut attack success most also refuse ordinary in-scope requests: over-refusal is 50% under system_prompt and 50% under layered, against 0% unguarded.
• H4(c) the residue is concentrated in the predicted clauses — 4 of 10 residual clauses were predicted: §3.6, §4.4, §4.13, §4.8. Not predicted: §2.2, §2.5, §3.4, §4.1, §5.11, §5.20
• H5(a) Critical requires more than twice the evidence of the lowest tier present (Medium) — Critical mean 91.4 evidence items vs Medium mean 92.0; ratio 0.993. The portfolio contains no Low-tier use case, so Medium is substituted for Low and the hypothesis is weaker than written.