1 Introduction
The unit of engineering for LLM systems has moved outward four times in about four years: from the wording of a prompt, to the contents of the context window, to the harness that runs one agent, and to the loop that runs many agents without a person in the chair (Figure 1). Context engineering became a common name in mid-2025 [1], [2], [3] and received a dedicated survey within weeks [4]. Harness engineering followed in early 2026 [5], [6], [7]. Loop engineering is the newest: a practitioner guide dates its spread to a single week in June 2026, when several prominent engineers described the same shift, namely to stop prompting agents and to design the system that prompts them [8], [9]. A recent survey extends the stack further, to graph and ontology engineering for multi-agent systems [10].
Three developments make a joint study timely. First, the harness now matters as much as the model on several agent tasks: benchmark scores move when only the harness changes [11], [12]. Second, tasks have outgrown the window. Agents run for hours across many context windows, so state must live outside the window and be reloaded [5], [13]. Third, the outer loop has become a product feature [14] and an object of academic study [15], [16], [17].
Yet the two literatures barely meet. Context studies typically measure answer quality per token inside one run. Loop practice optimises unattended throughput across many runs. The layers are nonetheless coupled: each loop iteration begins by building a context, and each context decision changes what feedback the loop can use. A design that is best for one layer alone can be wrong for the pair. We know of no study that places both on one taxonomy, measures how they interact, and maps their combinations to use cases.
1.0.0.1 Research questions.
We ask six questions. RQ1 How do academic and practitioner sources define context and loop engineering, and where are the boundaries with prompt, harness and graph engineering? RQ2 Which method families exist, how has output shifted between 2020 and 2026, and who produces it? RQ3 How strong is the evidence behind each family? RQ4 Do context and loop choices substitute for or complement each other in quality, cost and reliability? RQ5 Which combinations suit which use cases? RQ6 Which choices keep their value as models improve?
1.0.0.2 Contributions.
Operational definitions of both fields with boundary tests (Section 2).
A formal model of context as state and loop as control, with two closed-form results: a verifier ceiling on delivered quality and a quadratic cost law for append-only context (Section 3).
An open multivocal corpus of 1,395 coded sources, a taxonomy of seven context operations and eight loop levels, and an empirical test of a pre-specified coupling matrix (Sections 4–6).
Two secondary analyses: the diffusion of the three terms across 6,278 GitHub repositories, and a re-analysis of the published LoopArena results (Sections 5–6).
Six simulation experiments that test six hypotheses on coupling, horizon, verification and model strength (Section 7).
A use-case fit framework in which every recommendation carries an evidence grade, and a research agenda (Sections 8–9).
1.0.0.3 Scope of evidence.
Our analysis environment could not reach arXiv or bibliographic databases. We therefore sampled the literature through five community-maintained reading lists hosted on GitHub, which list title, link and usually an arXiv identifier for each source. We did not read the full texts of the papers we code; codes rest on titles, list annotations and the curators’ section headings. This is a real limitation, discussed in Section 10; it also makes the study fully reproducible from public files.
2 Definitions and boundaries
Table 1 gives working definitions. Each comes with a boundary test: a question that decides which layer a design choice belongs to.
Table 1. Working definitions and boundary tests.
| Term | Working definition | Boundary test |
|---|---|---|
| Prompt engineering | Design of the wording of one instruction: role, task statement, examples, output format. | Does the change alter only how one instruction is phrased? |
| Context engineering | Design of the policy that composes, selects, stores, compresses, isolates, evolves and guards the tokens a model receives at each step. | Does it change what enters the window, but not whether or when the model is called? |
| Harness engineering | Design of the runtime around one agent run: tools, permissions, the action loop, and what counts as done. | Does it change the environment of a single run? |
| Loop engineering | Design of the control structure that decides when the model is called again, with what goal, on what evidence, and when to stop. | Does it change whether another step or run happens? |
Loop engineering has a narrow and a broad sense. The narrow sense, from June 2026 practitioner writing, is the outer system that discovers work, dispatches agents, verifies results, keeps state between runs and runs on a timer or an event [8], [9]. One pattern library adds an autonomy ladder: loops start report-only, become assisted, and run unattended only after the verifier has proved reliable [9]. The broad sense covers every iteration level from the action loop of ReAct [18] outward. Our taxonomy uses the broad sense and reports the narrow sense as two of its levels, the session loop and the scheduled loop.
The harness is the bridging layer. It holds the action loop (a loop concern) and the window that the action loop fills (a context concern). Our audit confirmed that this boundary is the hardest to code: most disagreements between coders fell between harness and loop (Section 4.4).
3 A formal model of context and loop
We treat context as the agent’s state and the loop as its control. At step a context policy builds the window from history , external memory , knowledge sources and tools , under a token budget : The model acts on that context, . A loop policy then reads the evidence , the persisted state and the remaining budget , and chooses a control move: Design seeks quality at low cost within a harm limit and a cap on human oversight effort : Context engineering designs and the rule that writes to . Loop engineering designs , its triggers, its verifiers and the state . A harness is the tuple for one run.
3.1 Two closed-form results
Proposition 1 (Verifier ceiling). Let one attempt be correct with probability . A verifier wrongly rejects a correct result with probability and wrongly accepts an incorrect one with probability . A loop that retries until acceptance, up to attempts, delivers a correct result with probability The yield rises with , but the precision of what is delivered is fixed at . As , delivered quality tends to the verifier’s precision and cannot exceed it.
Proof. Each attempt independently ends the loop with a correct delivery (probability ), an incorrect delivery () or a rejection (). The probability that the first acceptance happens at attempt and is correct is ; summing the geometric series gives the result. ◻
When attempts on one task are correlated, for example because the same context misleads every attempt, the gain from retries is smaller still. We model this with a task-level drawn from a Beta distribution (Section 7).
Proposition 2 (Context cost). An append-only window that starts at tokens and grows by tokens per step consumes input tokens over steps, which is quadratic in the horizon. A window bounded at tokens consumes at most .
Prompt caching lowers the price of re-read tokens, by a factor , but not their effect on attention. Policies that rewrite history, such as masking and compaction, also invalidate the cache from the rewritten point. The simulation therefore prices fresh and cached tokens separately.
3.2 Taxonomy
Context engineering is described by seven operations (Table 2). Four (write, select, compress, isolate) follow a grouping common in practitioner guides [2]; we add compose, evolve and guard. Loop engineering is described by eight levels, ordered from the innermost to the outermost (Table 3), and by ten design dimensions (Appendix A).
Table 2. Seven context operations. The share column gives the proportion of context-layer sources in the corpus coded with each operation (a source can have several).
| Operation | Question it answers | Typical methods | Share |
|---|---|---|---|
| Compose | What standing instructions, examples and tool descriptions sit in the window, and in what order? | System prompts, few-shot examples, repository instruction files, cache-stable prefixes | 19% |
| Select | What is pulled in for this step? | Retrieval, reranking, graph retrieval [19], agentic search, tool and skill retrieval | 53% |
| Write | What is saved outside the window for later? | Scratchpads, notes, state files, memory systems [20], [21] | 44% |
| Compress | What is shortened or dropped? | Summarisation, observation masking, pruning, learned compression [22], [23] | 18% |
| Isolate | What is kept in a separate window? | Sub-agents, branch contexts, recursive calls | – |
| Evolve | How does the context policy improve from experience? | Evolving playbooks [24], learned curation [25], [26] | 3.8% |
| Guard | What is trusted, attributed or withheld? | Provenance, trust boundaries, injection defences [27] | – |
Isolate and Guard are rare inside the context layer because sources that isolate or guard are usually coded to the loop or harness layer; their counts across all layers appear in Figure 5.
Table 3. Eight loop levels. The share column gives the proportion of loop-layer sources coded with each level.
| Level | What repeats | Representative work | Share |
|---|---|---|---|
| 1 Action | Model call, tool call, observation | ReAct [18]; CodeAct [28]; SWE-agent [29] | – |
| 2 Plan | Plan, execute, replan | StateFlow [30]; AdaPlanner [31]; Plan-and-Act [32] | 28% |
| 3 Verification | Produce, check, revise | Self-Refine [33]; Reflexion [34]; CRITIC [35]; Proof-or-Stop [16] | 15% |
| 4 Search | Branch, evaluate, select | Tree of Thoughts [36]; LATS [37] | 4.3% |
| 5 Orchestration | Delegate, collect, merge | Magentic-One [38]; controller and worker [17] | 49% |
| 6 Session | Fresh run, reload state, continue | Long-running harnesses [5]; the “Ralph” loop [39] | 15% |
| 7 Scheduled | Trigger, discover work, dispatch, verify, persist | Pattern libraries [9]; [15] | 16% |
| 8 Improvement | Run, score, change prompts, context, harness or weights | ADAS [40]; Meta-Harness [12]; Self-Harness [41] | 24% |
Level 1 sits mostly inside the harness layer, so its share among loop-layer sources is not informative and is omitted.
3.2.0.1 Hypotheses.
The model suggests six hypotheses, stated before analysis in the study protocol. H1 (verifier ceiling): more loop iterations raise delivered quality only up to the verifier’s precision. H2 (complementarity): the value of a loop mechanism depends on the context policy it runs with. H3 (horizon switch): append-only context wins within a few windows of work, and resets with file-backed state win beyond. H4 (substitution): with a strong verifier, a reset loop with minimal context management matches elaborate context management. H5 (independent verification): an evaluator with its own context beats self-critique in the working context. H6 (scaffold decay): the benefit of context and loop scaffolds shrinks as models improve.
4 Method
The study combines a multivocal review with a formal model and two secondary data sets (Figure 2). Grey literature is included because loop engineering began in practitioner writing, which a search of peer-reviewed venues alone would miss [42]. Reporting follows PRISMA 2020 where it applies [43], and procedure follows software-engineering review guidance [44]. All steps run from scripts in the accompanying repository.
4.1 Deviations from the protocol
The study protocol, written before data collection, planned database searches, a factorial experiment with live models, an affiliation map and a validation against production case reports. Four deviations followed from the analysis environment and the chosen scope.
Sampling frame. arXiv, OpenAlex and similar services were unreachable from the analysis environment. We therefore sampled through 5 community-curated reading lists on GitHub (Section 4.2). This trades recall for reproducibility.
No live experiment. The factorial experiment (Tier C in the protocol) was not run. In its place we re-analyse the published results of LoopArena [17], which varies loop control with model and task fixed, and we test the hypotheses in simulation.
Geography by proxy. Author affiliations were not available. We describe practitioner communities through repository metadata instead (Section 4.6).
No case-survey validation. The fit framework is graded by the evidence behind each recommendation (Section 8) but has not been checked against production case reports.
4.2 Sampling frame and parsing
Five lists form the frame: one centred on context engineering [45], one covering the whole stack up to graph engineering [46], and three centred on harness engineering [47], [48], [49]. Their README files were retrieved on 13 November 2026. A parser handles the four entry formats the lists use (HTML items, venue-tagged bullets, Markdown links and Markdown tables) and records the curator’s section heading for each entry. Dates come from the arXiv identifier where present, otherwise from venue tags, badges, text or URL. De-duplication merges entries by arXiv identifier, then by normalised URL, then by normalised title. The parser found 1,891 entries and 1,492 unique sources; 180 sources appear in more than one list.
4.3 Screening and coding
Sources were screened by three rules: entries in navigation or off-frame sections (pre-training, prompt wording, ontology engineering, course lists) were excluded; model-architecture work was excluded unless its title also concerned agents, retrieval, memory or compression; and entries with no in-scope term were excluded. 97 sources were removed (58, 31 and 8 by the three rules), leaving 1,395.
Each included source was coded on five dimensions by a rule-based codebook that is released with the data. The primary layer (context, loop, harness or evaluation) uses the curator’s section heading as a prior and overrides it when the source’s own title or annotation carries strong evidence. The seven context operations and eight loop levels are multi-label codes triggered by terms in the title and annotation, plus codes implied by the curator’s heading. The contribution type (method, benchmark, empirical study, survey, practitioner write-up, tool) and the application domain complete the record. Codes rest on titles, list annotations and headings, not on full texts.
4.4 Audit of the coding
Coding quality was measured against blind manual coding, in which the second coder saw each source’s title, annotation and heading but not the rule output. A development sample of 30 sources was used while writing the rules and excluded from both audits. In round 1, codebook v1 coded 100 random sources; agreement on the primary layer was [50], below the protocol’s threshold of 0.70. Most disagreements fell between harness and loop. We revised the rules once, placing the action loop inside the harness in line with the framework (Table 3), and drew a fresh sample of 60 sources that had played no part in the revision. Codebook v2 reached on that held-out sample (90% raw agreement; confusion matrix in Appendix C) and on contribution type. The manual coder was the AI assistant that also wrote the rules; the audit therefore measures the consistency of the rules with careful reading, not agreement between independent human experts (Section 10).
4.5 Coupling analysis
For each of the 56 pairs of loop level and context operation we compared the number of sources carrying both codes with the number expected under independence. We report the lift (observed over expected) and a one-sided Fisher exact test, with Holm correction across all pairs [51]. The protocol’s draft coupling matrix named 19 pairs in advance, which allows a test of prediction rather than only description.
4.6 Term diffusion on GitHub
We queried the GitHub repository search API [52] for repositories whose name, description or topics contain each quoted term: “context engineering”, “harness engineering”, “loop engineering” and, as a reference, “prompt engineering”. Monthly counts of newly created repositories cover January 2025 to 13 November 2026, with yearly counts for 2023 and 2024. For the three engineering terms we also downloaded metadata for every matching repository, splitting creation-date windows to stay under the API’s 1,000-result cap, which gave 6,278 unique repositories. Community proxies are the share of descriptions containing Han characters, the share of repositories owned by organisations, and the share with at least 100 stars.
4.7 LoopArena secondary analysis
LoopArena [17] compares loop policies that steer one fixed coding worker (the same worker model, tools, budget and tasks in every arm). Two reference policies give the worker one instruction and let it run (no control) or re-issue a fixed instruction until the worker reports completion (fixed control); six controller models instead read a report of the worker’s progress and issue the next assignment. Type II tasks are condensed slices of a task, and Type III tasks start from the original, full task state. Each cell has 240 runs (80 tasks, 3 repeats). We used the published aggregate file (successes, runs and normalised cost per run). We report Wilson intervals for success [53] and Newcombe intervals for differences from no control [54]. Because repeats of a task are not independent and per-task data were not published, we also report a conservative interval that counts each task once.
4.8 Simulation
Study 3 extends the closed-form results with a Monte Carlo simulator, specified in Section 7 with parameters in Appendix B. Six experiments (E1–E6) use 3,000 simulated tasks per cell (1,500 in the E6 grid and 600 per policy in the 300 random draws), fixed seeds and common random numbers across policies.
5 Results I: the landscape and its diffusion
5.1 What the corpus contains
Table 4 shows the included corpus by primary layer and source type. Harness sources are the largest group (602), followed by loop (303), context (290) and evaluation (200). This partly reflects the frame, since three of the five lists centre on harnesses. Academic sources dominate the context layer (190 of 290), whereas 82% of harness sources and 50% of loop sources are practitioner write-ups or repositories.
Table 4. Included sources by primary layer and source type (codebook v2). The last column is the share coded as a benchmark or an empirical study.
| Layer | Academic | Grey | Repository | Total | Evaluative |
|---|---|---|---|---|---|
| Context | 190 | 39 | 61 | 290 | 3.1% |
| Loop | 152 | 51 | 100 | 303 | 1.7% |
| Harness | 110 | 146 | 346 | 602 | 1.7% |
| Evaluation | 92 | 38 | 70 | 200 | 44.5% |
| Total | 544 | 274 | 577 | 1395 |
Evidence is thin where practice is busiest (Figure 3). Only 3.1% of context sources, 1.7% of loop sources and 1.7% of harness sources are benchmarks or empirical studies; most academic work proposes methods (55% of context sources, 47% of loop sources). Software engineering is the most common named domain (429 sources), far ahead of research automation (59), web and GUI agents (56) and dialogue (38).
5.2 How the academic literature has shifted
Of academic sources, 95% carry a date. Figure 4a shows the balance moving from context to loop. Context papers peaked in 2024 (67) and numbered less than half that in 2026 (29); loop papers rose from 19 in 2024 to 75 in 2026. Loop work accounts for 42% of dated academic sources in 2026, against 30% across all years, while context work fell from 37% overall to 16% in 2026. The lists were curated in 2025 and 2026, so older work is under-sampled; the shift within recent years is the more reliable signal.
The terms reached academic titles within weeks of practitioner uptake. A survey of context engineering appeared as arXiv 2507.13334 [4], one month after “context engineering” first reached ten new GitHub repositories in a month (2025-06). Inside the corpus, the first arXiv source naming harness engineering in its title or annotation has identifier prefix 2604 (April 2026), two months after that term reached ten repositories a month (2026-02), and the first naming loop engineering has prefix 2607 (July 2026), two months after 2026-05.
5.3 Families of context engineering
Selection and writing dominate the context layer (Table 2): 53% of context sources select what enters the window (retrieval, reranking, graph and agentic search) and 44% write state outside it (scratchpads, notes, memory systems). Composition (19%) and compression (18%) are less common, and learned or evolving context is still rare (3.8%), although it is the most active recent line, from evolving playbooks [24] to context curation trained with reinforcement learning [25], [26]. Isolation and guarding seldom appear as the primary topic of a context source; they appear instead in loop and harness sources, where sub-agents and trust boundaries are designed.
Three findings recur across the families. First, longer inputs degrade performance even on simple tasks, which practitioners call context rot [55]. Second, compression and memory trade cost against the loss of facts a later step needs, and recent work learns that trade-off rather than fixing it by hand [23]. Third, retrieval is moving from a fixed pre-built index towards search tools the agent calls when it needs them [1]. Each of these is a statement about state, and each affects what a loop can do with it.
5.4 Families of loop engineering
Orchestration is the most common loop level among loop sources (49%), followed by planning (28%) and self-improvement (24%); scheduled (16%), session (15%) and verification loops (15%) follow, and search loops are rare (4.3%). The narrow, practitioner sense of loop engineering, the scheduled outer loop, is the least studied in academic venues: it appears mainly in pattern libraries and guides [8], [9] and in a handful of 2026 papers [15], [16], [56]. Two failure modes now have their own studies: loops that never stop [57] and coordination failures in multi-agent systems [58]. Self-improvement loops have turned on the harness itself, with several 2026 systems that rewrite their own harness from traces [12], [41], [59], [60], [61].
5.5 Diffusion of the terms on GitHub
Table 5. Term diffusion on GitHub. Counts are repositories created in each month whose name, description or topics contain the quoted term (search API, 13 November 2026). Community columns describe the full set of matching repositories: share with Han characters in the description, share owned by organisations, and share with at least 100 stars.
| Term | New repos | First month | Peak month | Repos | Han | Org. | 100 |
| Jan 25–Nov 26 | 10 | (count) | (all) | script | owner | stars | |
| “context engineering” | 2,655 | 2025-06 | 2026-08 (208) | 2,656 | 3.5% | 9.5% | 2.3% |
| “harness engineering” | 2,469 | 2026-02 | 2026-04 (408) | 2,279 | 11.4% | 6.9% | 2.4% |
| “loop engineering” | 1,389 | 2026-05 | 2026-07 (426) | 1,387 | 8.7% | 5.0% | 1.2% |
| “prompt engineering” | 19,744 | 2025-01 | 2026-07 (1898) | – | – | – | – |
The three terms spread in sequence (Figure 4b, Table 5). “Context engineering” crossed ten new repositories a month in June 2025, jumped to 184 in July, and has since stayed between about 75 and 210 a month. “Harness engineering” took off in February 2026 and peaked at 408 in April. “Loop engineering” had almost no repositories before May 2026, peaked at 426 in July, the month after it was named, and fell to 62 by October. By October 2026 the monthly figures were 189 (context), 270 (harness) and 62 (loop), against roughly 1,400 for “prompt engineering”. The term communities barely overlap: only 43 of the 6,278 repositories mention two or more of the terms.
The repositories are mostly small personal projects. Organisations own 10%, 7% and 5% of them, and the median repository has no stars. Han characters appear in 11% of harness and 9% of loop repository descriptions, against 4% for context engineering, which points to a larger Chinese-language practitioner community around the two newer terms. These are proxies: GitHub is not the whole practitioner population, and description language is not author location.
6 Results II: coupling and loop control
6.1 Which context operations travel with which loops
Table 6. Loop level and context operation pairs that co-occur above chance (one-sided Fisher exact test, Holm-adjusted across 56 pairs, ). “Draft” marks pairs predicted by the protocol’s coupling matrix.
| Loop level | Context operation | Observed | Expected | Lift | Draft | |
|---|---|---|---|---|---|---|
| Improvement | Evolve | 20 | 2.2 | 8.99 | yes | |
| Scheduled | Isolate | 11 | 3.8 | 2.93 | no | |
| Session | Isolate | 35 | 13.8 | 2.54 | no | |
| Action | Compress | 20 | 7.9 | 2.53 | yes | |
| Session | Write | 65 | 35.3 | 1.84 | yes | |
| Orchestration | Isolate | 48 | 26.9 | 1.79 | yes |
Loop levels and context operations are not independent (Figure 5, Table 6). 6 of 56 pairs co-occur above chance after correction. 4 of them were among the 19 pairs predicted in advance: improvement loops with evolving context (lift 9.0), action loops with compression (2.53), session loops with writing state (1.84) and orchestration with isolation (1.79). Predicted pairs had a mean lift of 1.81, against 0.97 for all other pairs, and 12 of 19 had a lift above one. The two unpredicted significant pairs both involve isolation: session loops (2.54) and scheduled loops (2.93). In practice, sessions and scheduled runs are given fresh, separate contexts, so isolation belongs to the outer loops as much as to orchestration. The draft’s prediction that verification loops lean on isolation and composition was not borne out in the corpus; the data cannot say whether that reflects practice or how sources describe themselves.
6.2 What loop control buys on coding tasks
Table 7. LoopArena re-analysis. Success is the strict success rate over 240 runs (80 tasks 3 repeats). Cost is LoopArena’s normalised no-cache estimate in US dollars. The gain column gives Type III percentage points over no control with a 95% Newcombe interval that treats runs as independent.
| Success (%) | Type III cost (USD) | ||||
|---|---|---|---|---|---|
| 2-3(lr)4-5 Loop policy | Type II | Type III | per run | per solved | Gain, Type III (pp) |
| No control | 49.2 | 27.5 | 2.61 | 9.51 | – |
| Fixed control | 55.0 | 31.3 | 9.47 | 30.32 | +3.8 [4.4, 11.8] |
| GPT-5.5 | 56.7 | 36.3 | 34.68 | 95.67 | +8.8 [0.4, 16.9] |
| Qwen3.7-Plus | 56.7 | 34.2 | 12.69 | 37.13 | +6.7 [1.6, 14.8] |
| Qwen3.6-35B-A3B | 55.8 | 33.8 | 11.58 | 34.31 | +6.3 [2.0, 14.4] |
| Claude Opus 4.8 | 58.8 | 33.8 | 20.63 | 61.12 | +6.3 [2.0, 14.4] |
| DeepSeek-V4-Flash-0731 | 50.8 | 28.3 | 32.66 | 115.25 | +0.8 [7.2, 8.8] |
| GLM 5.2 | 40.4 | 24.2 | 11.97 | 49.55 | 3.3 [11.1, 4.5] |
Loop control helps a little and costs a lot (Table 7, Figure 6). On full tasks, an uncontrolled worker solved 27.5% of runs. Fixed control raised this to 31.3%, and controller models reached 31.7% on average and 36.3% at best (GPT-5.5). The best gain, 8.8 points, has a 95% interval of 0.4 to 16.9 points if runs are independent, and of 5.6 to 22.7 points if each task counts once; no gain is distinguishable from zero under the conservative interval. Controlled policies cost 3.6 to 13.3 times as much per run, so the cost per solved task rose from $9.51 without control to between $30.32 and $115.25 with it. The pattern is the same on condensed tasks, where the best controller (Claude Opus 4.8) reached 58.8% against 49.2%.
Two further patterns bear on the model in Section 7. First, moving from condensed to full tasks cut success by 36% to 44% in relative terms for every policy, including no control (44%), which is consistent with error that grows with the length of the work. Second, controllers that chose better loop contracts in LoopArena’s single-decision test (Type I) tended to do better on full tasks (Spearman , , ), a weak hint that loop control quality is a skill that can be measured in isolation. Relative gains of the best controller over no control were larger on full tasks (32%) than on condensed ones (20%), so whatever control buys grows with the horizon.
These results come from one benchmark, one worker model and one domain, and from aggregate data published by the benchmark’s authors. They are evidence about loop control on coding tasks, not about loops in general.
7 Results III: simulation of context and loop
7.0.0.1 Model.
A task needs successful steps. At step the context policy sets the window load , and the step succeeds with probability where is the model’s base step error (drawn per task from a Beta distribution, so attempts on one task are correlated), is the factor by which a full window of size raises the error rate, and is the probability that the step needs a fact the policy has lost. Four context policies are compared. Append keeps the whole history and silently truncates at the window limit. Mask replaces old observations with short placeholders in blocks of steps. Compact summarises the history whenever it reaches a share of the window; each summary drops a needed fact with probability . Reset starts a fresh window every steps and reloads a state file that lacks a needed fact with probability . On the loop side, a failed step is caught with probability and retried up to times; a retry re-reads files with probability , which recovers lost information. Self-critique shares the working context and so cannot see information that the context lost. Fresh input tokens cost one unit and cached tokens ; append keeps the cache valid, whereas mask and compact invalidate it at each rewrite. Parameters are listed in Appendix B. They are assumptions chosen to be plausible, not estimates, which is why E6 varies them.
7.1 E1 and E2: the verifier ceiling and the cost of context
Retries buy yield, not precision (Figure 7a). With a moderate verifier, one attempt delivers a correct result 0.36 of the time and twenty attempts 0.80, which is exactly the verifier’s precision (0.80) as Proposition 1 predicts. When difficulty varies across tasks, hard tasks fail on every attempt and twenty attempts reach only 0.68. A strong verifier lifts the ceiling to 0.97 and a weak one caps delivery at 0.57: the verifier, not the number of retries, sets the quality of an unattended loop.
Caching changes the cost picture more than the choice of policy does (Figure 7b). Over 300 steps, append costs 52.7 million token-equivalents without caching but 6.0 million with it, about 8.9 times less. Reset with a state file remains the cheapest policy (2.7 million), because it both bounds the window and keeps most calls cacheable.
7.2 E3 and E6: which context policy wins, and why
At default parameters the answer depends on the horizon (Figure 8a). Append wins short tasks, and reset with a state file wins long ones. The switch comes earlier as context rot grows: at steps with no rot (), at 60 with , and at 40 for . This supports H3, the horizon switch.
The ranking of long-horizon policies, however, is not robust (Figure 8b). Across 300 random parameter settings at , reset had the highest success in 40% of settings, masking in 25%, append in 21% and compaction in 15%. At append won 71% of settings. What decides the long-horizon winner is how much information each policy loses per handoff: the state-file loss , the masking loss and the compaction loss were the most important splits in a depth-three classification tree (accuracy 64%), and a one-line rule (choose whichever of reset, mask and compact loses least) picked the winner in 66% of the 238 settings that append did not win, against 33% by chance. Cost is more clear-cut: reset had the lowest cost per solved task in 73% of settings at . In short, no context policy is best for long tasks in general, but resetting from a well-kept state file is usually the cheapest, and the decisive quantity, retention per handoff, is one the literature rarely measures.
7.3 E4: verification and context interact
Stronger detection lifts every policy (Figure 9). With an independent verifier, the gap between reset and append stays almost constant on the log-odds scale as detection rises (0.50 at , 0.45 at ), so verification strength and context policy act roughly additively; verification does not make the context choice irrelevant, contrary to the strong form of H4. Reset with minimal context management does, however, match or beat the more elaborate policies at every detection level (at : reset 0.72, compaction 0.66, masking 0.66, append 0.63).
The type of verifier interacts strongly with the context policy. Replacing the independent verifier with self-critique at costs append 9 points and reset 17 points, but compaction 30 and masking 34 points, because self-critique cannot detect errors caused by information its own context has lost. This supports H5, and it gives H2 a concrete form: lossy context policies need an independent verifier.
7.4 E5: what happens as models improve
Scaffolds lose value as models get stronger (Figure 10). With mild context rot (), reset’s advantage over append moves from +2.7 points for the weakest model to 3.8 points for the strongest: the context scaffold turns from help to hindrance. With strong rot () reset stays ahead, peaking at +8.3 points for mid-range models and shrinking to +1.9 for the strongest. The verifier’s gain also shrinks, from 46 to 22 points, but remains large. This supports H6 in the model; whether real model generations follow it is an empirical question that LoopArena, with a single worker, cannot answer.
8 A use-case fit framework
Two properties of a use case predict most of the design choice: how long the task runs and whether a machine can check the result. Risk then sets the ceiling on autonomy. Figure 11 places thirteen use cases on these two axes, and Table 8 gives the recommended context and loop design for each, with an evidence grade.
Evidence grades follow a simple scale. A: a controlled comparison with model and task held fixed exists for the loop or context choice in that use case. B: the recommendation follows from the formal model or simulation and agrees with academic work in the corpus. C: the recommendation rests on practitioner reports, tools or design reasoning only. Only one row reaches grade A, and only for loop control on coding tasks.
Table 8. Use-case fit matrix. Horizon: S short (fits one window), M medium, L long (many windows), R recurring, E event-driven. Check: who or what verifies results. Grade: strength of evidence (see text).
| Use case | Horizon; check | Context design | Loop design | Autonomy ceiling | Grade |
|---|---|---|---|---|---|
| Document extraction at volume | S; schema validator | Lean instructions, examples, cache-stable prefix | Action loop with validator; few bounded retries | Unattended, sampled audits | B |
| Enterprise question answering | S; people, citations | Hybrid retrieval with reranking; graph retrieval for corpus-wide questions | One run, at most a short retrieve-and-refine loop | Assisted | B |
| Customer support under policy | S; system state, people | Policy text; account and policy retrieval; small tool set | Fixed workflow around a bounded action loop; escalation to a person | Assisted | C |
| Interactive coding | S–M; tests, linters | Repository instruction file; agentic search; compress when needed | Action and verification loops with a test gate; person approves the plan | Assisted | B |
| Long-running software builds | L; tests, browser checks | Reset with a state file, progress log and version control | Session loop with an independent evaluator and budget caps | Unattended in a sandbox; person reviews merges | A, B |
| Repository upkeep (triage, CI, dependencies) | R; CI status | Run log and state file; guard untrusted issue text | Scheduled loop climbing an autonomy ladder | Report-only first | C |
| Deep research and synthesis | L; people, citation checks | Parallel isolated searchers; notes; condensed returns | Orchestration with evidence gates and an independent citation check | Assisted | C |
| ML engineering, algorithm search | L; numeric metric | Experiment journal; program archive | Search and improvement loops | Unattended within a compute budget | B |
| Computer and browser use | M; page state, people | Prune screenshots and page structure; keep recent steps | Action loop with step checks; confirmation before irreversible actions | Assisted | C |
| Long-term personal assistant | Many sessions; people | Profile, episodic and semantic memory; scheduled consolidation | Conversational loop; background consolidation loop | Assisted | B |
| Regulated decision support | S–M; people, audit | Retrieval with provenance; data minimisation; logged context | Bounded loops; mandatory human gate; no self-modifying loop | Report-only or assisted | C |
| Incident response and operations | E; health checks, people | Runbooks; filtered logs | Event-triggered loop; approval before remediation | Assisted | C |
| Multi-agent social simulation | L; none | Memory stream with reflection | Orchestration on a simulation clock | Unattended, offline | C |
The evidence condenses into a decision procedure.
Measure the verifier before adding iterations. Delivered quality cannot exceed the verifier’s precision (Proposition 1). If false acceptances cannot be estimated, keep people in the loop and invest in context instead.
If the task fits one window, append and cache. Append-only history with a stable prefix won short horizons in simulation and is cheap once cached (E2, E3).
If it does not, choose the handoff that loses least. No long-horizon policy wins in general; retention per handoff decides (E6). Reset with a state file is the cost default.
Pair lossy context with an independent verifier. Self-critique misses the errors that compaction and masking create (E4).
Judge loop control by cost per solved task. On coding tasks it added at most a few points of success at several times the cost (Section 6.2).
Automate recurring work in steps. Scheduled loops should move from report-only to unattended only after the verifier has proved reliable [9].
Re-test scaffolds when the model changes. Context scaffolds that help a weaker model can hurt a stronger one (E5).
9 Discussion
9.1 Answers to the research questions
RQ1, definitions. Context engineering is well defined around what enters the window; loop engineering is defined by practitioners in a narrow sense (the unattended outer loop) and used more broadly in research. The boundary that coders found hardest is harness against loop, because the action loop sits inside the harness. Placing it there, as our framework does, raised coding agreement from to .
RQ2, landscape. The corpus holds 1,395 sources. Research attention has moved from context to loop: loop sources make up 42% of dated academic sources in 2026. On GitHub the three terms spread in sequence, from mid-2025 (context) to February 2026 (harness) to June 2026 (loop), and academic titles followed within one to two months each time. The newer terms have a larger Chinese-language practitioner community than context engineering.
RQ3, evidence. Evidence is thinnest where practice is busiest. Benchmarks and empirical studies make up 1.7% of loop and 1.7% of harness sources, and half or more of those layers’ sources are practitioner write-ups or tools. Software engineering dominates the named domains.
RQ4, coupling. The layers are coupled in the literature: 6 level–operation pairs co-occur above chance, mostly as predicted. In simulation, verification strength and context policy combine roughly additively, but verifier type and context policy interact: lossy context policies lose most when the verifier shares their context. Loop control on real coding tasks bought little success for its cost, so loops do not substitute for a sound context and a good worker.
RQ5, fit. Section 8 gives the framework. Most rows rest on grade B or C evidence.
RQ6, durability. In simulation, context scaffolds lose value and can become harmful as base error falls, while verification keeps a large benefit. We could not test this across real model generations.
Table 9. Status of the six hypotheses.
| Hypothesis | Status | Evidence |
|---|---|---|
| H1 Verifier ceiling | Supported | Proposition 1; E1: twenty retries reach the verifier’s precision and no more, less when difficulty varies across tasks. |
| H2 Complementarity | Partly supported | E4: self-critique costs lossy policies 30–34 points against 9 for append; the corpus shows above-chance coupling. Not tested on real systems. |
| H3 Horizon switch | Supported in simulation | E3: append wins short horizons; the switch to reset moves from 150 to 40 steps as context rot grows. LoopArena’s 36%–44% relative drop on full tasks is consistent. |
| H4 Substitution | Not supported in strong form | E4: the reset–append gap is stable on the log-odds scale across detection levels; context choice still matters with a strong verifier. Reset matches or beats elaborate policies, as the weak form predicts. |
| H5 Independent verification | Supported in simulation | E4: at the independent verifier beats self-critique for every policy, by up to 34 points. |
| H6 Scaffold decay | Supported in simulation | E5: reset’s gain over append falls from +2.7 to 3.8 points under mild rot; the verifier’s gain falls from 46 to 22 points. Untested across real model generations. |
9.2 Implications
For practitioners, the most robust results are also the simplest. A loop is only as good as its verifier, so the first investment in an unattended loop is a verifier whose false-acceptance rate is known. Caching, not compression, is the largest lever on the cost of short tasks. For long tasks, the quality of the state that survives a handoff matters more than which handoff mechanism is used. Loop control should be bought by the solved task, not by the run.
For researchers, the gaps are measurable. Few studies report how much information survives a summary, a mask or a reset, although E6 shows that this single quantity decides which long-horizon policy wins. Few studies vary context and loop design independently with the model fixed, as LoopArena does for loop control alone. And no instrument in the corpus measures the human side of a loop: review time, oversight load, or whether people still understand what the loop ships.
9.3 Research agenda
Retention benchmarks. Measure the probability that a fact needed later survives each handoff policy, across task types.
Factorial experiments. Vary context policy and loop policy independently, with model and task fixed, beyond coding.
Verifier calibration. Report false-acceptance and false-rejection rates for the verifiers used in loops, and the precision ceiling they imply.
Costs of autonomy. Measure human review time and comprehension alongside success and tokens.
Durability. Re-run the same scaffolds across model generations to test scaffold decay.
Reporting standards. Report tokens, cost, cache use and repeated runs with every agent result, so that gains such as those in Table 7 can be judged by cost per solved task.
Coverage. Extend the corpus to Chinese-language sources and to full texts, which this study could not reach.
10 Threats to validity
10.0.0.1 Construct validity.
Codes rest on titles, list annotations and curators’ headings, not on full texts. The harness–loop boundary is fuzzy by nature. The manual coder in the audit was the AI assistant that wrote the rules, so measures how well the rules match careful reading of the same evidence, not agreement between independent human experts. A second human coder is the most important addition for a future version.
10.0.0.2 Internal validity.
The codebook was revised once after the first audit. The second audit used a fresh sample that played no part in the revision, but the revision itself was informed by the first sample, whose in-sample agreement is therefore optimistic.
10.0.0.3 External validity.
The frame is five curated lists, three of them about harness engineering, curated in 2025 and 2026 and written in English. The corpus over-represents harness work and recent work, and under-represents Chinese-language and older literature. GitHub counts depend on how the search API matches text, include forks of tutorials and abandoned projects, and stand in for a practitioner population that is larger than GitHub. LoopArena is one benchmark in one domain, with one worker model, published as aggregates by its authors; model names are reported as the benchmark gives them.
10.0.0.4 Conclusion validity.
Simulation results hold for the model as specified. Its parameters are assumptions, which E6 varies, but its structure is also a choice: steps fail independently given the task, verifiers cost nothing, and partial credit does not exist. LoopArena intervals that treat runs as independent overstate precision; the conservative intervals bound the effect of clustering.
10.0.0.5 Bibliographic validity.
Bibliographic details come from the curated lists and from README files opened on 13 November 2026, and are recorded with their provenance in the repository. Full texts were not opened, and author names marked in the provenance file as entered from knowledge should be checked against the original papers.
11 Conclusion
Context engineering and loop engineering are two halves of one design problem: what an agent knows at each step, and whether another step happens. Treating them together yields results that neither literature shows alone. A loop cannot deliver better results than its verifier allows. At long horizons, the information that survives a handoff decides which context policy wins. An independent verifier matters most where context loses information. On real coding tasks, loop control has so far bought little success for its cost. The field’s evidence base is thin exactly where its practice is busiest, and the most useful next studies are controlled ones that vary context and loop together, measure retention, and report cost per solved task.
Declarations
11.0.0.1 Use of AI assistance.
This study was carried out with substantial assistance from an AI system (Claude, by Anthropic), which wrote the analysis code, ran the analyses and simulations, performed the manual coding in the audit, and drafted the text under the author’s direction. The author takes responsibility for the content.
11.0.0.2 Data and code availability.
All data, code, figures and the LaTeX source are released with this preprint as a single archive intended for a public GitHub repository. Every number in the paper is generated from the processed data by the scripts in the archive.
11.0.0.3 Competing interests and funding.
The author declares no competing interests. The study received no specific funding.
References
A Ten loop design dimensions
Table 10. Design dimensions of a loop. Any loop at any level can be described by one option on each dimension.
| Dimension | Options |
|---|---|
| Trigger | Person, schedule, event, another agent |
| Iteration unit | Tool call, turn, task, session, generation |
| State carrier | Window history, summary, files, version control, database, weights |
| Feedback source | Environment, deterministic check, judge model, person |
| Controller | Fixed code, the working model, a separate controller model, a person |
| Stop rule | Self-declared done, verifier pass, budget, convergence, sign-off |
| Recovery | Retry, backtrack, rollback, reset, escalate |
| Topology | Single thread, fan-out, tree, population, hierarchy |
| Budget governance | Caps on steps, tokens, cost and time |
| Oversight | Before action, at checkpoints, or after the fact; autonomy from report-only to assisted to unattended |
B Simulation parameters
Table 11. Simulation parameters and default values. Experiments vary the parameters named in the text; E6 draws ten of them at random.
| Symbol | Meaning | Default |
|---|---|---|
| steps the task needs | 60 | |
| window size (tokens) | 200,000 | |
| fixed prompt, tools and task (tokens) | 8,000 | |
| tokens added per step | 2,500 | |
| error-rate rise at a full window | 2 | |
| mean base step error | 0.02 | |
| Beta concentration of step error across tasks | 100 | |
| P(step needs an older fact) | 0.05 | |
| mask: observations kept verbatim (block size) | 10 | |
| mask: tokens per masked observation | 60 | |
| mask: P(masked fact cannot be re-derived) | 0.3 | |
| compact: trigger as share of W | 0.6 | |
| compact: summary size as share of W | 0.08 | |
| compact: P(summary drops a needed fact) | 0.2 | |
| reset: steps per session | 20 | |
| reset: state-file tokens per step | 150 | |
| reset: P(state file lacks a needed fact) | 0.1 | |
| detection probability | 0.5 | |
| – | detection blind to lost information | no |
| retries per step | 2 | |
| P(retry re-reads files and recovers) | 0.8 | |
| price of a cached token (fresh = 1) | 0.1 |
Experiment E1 uses a base attempt accuracy of 0.4 and three verifiers: strong (, ), moderate (, ) and weak (, ); the heterogeneous variant draws accuracy from a Beta distribution with mean 0.4 and concentration 2. E3 crosses nine horizons with six values of . E4 uses , and five detection levels. E5 crosses four base error rates, two values of and two detection levels at . E6 has two parts: a grid of compaction and state-file loss rates at , and 300 random draws of , (log-uniform), , , , , , , and , each evaluated at and .
The depth-three classification tree fitted to the draws (winner by success) is reproduced below; lam_r, delta_mask and lam_c are the per-handoff loss rates and s is the tokens added per step.
C Audit of the coding
Table 12. Held-out audit (round 2, codebook v2, ): rule-based primary layer (rows) against blind manual coding (columns).
| Rule manual | Context | Evaluation | Harness | Loop |
|---|---|---|---|---|
| Context | 14 | 0 | 0 | 1 |
| Evaluation | 0 | 6 | 1 | 0 |
| Harness | 0 | 1 | 24 | 2 |
| Loop | 0 | 0 | 1 | 10 |
Codebook v2 made four changes to v1, all motivated by round-1 disagreements: section headings about agent loops, runtime orchestration, interaction protocols and harness architecture map to the harness layer, because the action loop sits inside the harness; skill composition maps to context selection; meta-harness generators and workflow profiles map to the loop layer; and explicit evidence of search, scheduled or self-improvement loops in a source’s own title or annotation overrides a generic section prior. One spelling rule (“tree-search”) and one evolve rule (evolving skills) were also added. Round-1 agreement for v1 was on layer and on contribution type.