Loop Engineering: From Manual Prompting to Autonomous Orchestration — The Next Abstraction Layer for AI Agents
In June 2026, Boris Cherny, Head of Claude Code at Anthropic, said something at Meta @Scale that propagated through the AI engineering community almost immediately:
"I don't prompt Claude anymore. I have loops running that prompt Claude and figure out what to do. My job is to write loops."
This wasn't a prediction. It was a description of how he works. The implication is uncomfortable: if you're still manually, turn-by-turn prompting an agent, you might be using the right tool at the wrong level of abstraction.
What Is Loop Engineering
The most concise definition comes from Addy Osmani (VP of Chrome at Google, June 2026):
"Loop engineering is replacing yourself as the person who prompts the agent. You design the system that does it instead."
Operationally, loop engineering means replacing manual per-turn agent prompting with an autonomous orchestration system — a "loop" that:
- Discovers work (polls a task queue, listens to git diffs, scans issues)
- Dispatches tasks to agents (builds context, injects tools, delivers goals)
- Verifies results (runs tests, checks constraints, evaluates output quality)
- Decides next steps (continue, retry, escalate to human, or terminate)
The entire cycle runs without a human in the loop at each iteration.
This is architecturally distinct from the agent itself — an agent is the executor inside a loop; loop engineering is the discipline of designing the loop.
Where It Sits in History
Loop engineering sits at the end of an evolution chain:
| Layer | Discipline | Core Question | Era |
|---|---|---|---|
| 1 | Prompt Engineering | How should I word this to get a good single response? | 2022–2024 |
| 2 | Context Engineering | What information should I put into the context window at each step? | 2025 |
| 3 | Harness Engineering | How do I build a reliable runtime environment for agents? | Early 2026 |
| 4 | Loop Engineering | How do I build a system that drives agents autonomously? | June 2026 onward |
Each layer contains the previous one rather than replacing it. A good loop still needs carefully crafted prompts, thoughtful context delivery, and reliable harness infrastructure.
Cherny's framing of this shift: "Loops are as big as the step from source code to agents was." That's a strong claim — it's a quote, not an empirical finding — and the rest of this article addresses it honestly.
Loop Anatomy: Five Required Components
"A loop is a task with a check. A task without a check is just hope."
— DEV Community practitioner guide, June 2026
A production-grade loop requires five components. Missing any one produces a predictable failure mode:
| Component | Role | Failure Mode When Missing |
|---|---|---|
| Trigger | What event starts the loop? (schedule, git push, PR opened, external webhook) | Loop never runs, or fires at the wrong moment |
| Inputs | What context does the agent receive on launch? (task description, relevant files, history) | Agent runs on wrong assumptions; output drifts from target |
| Action | What is the agent authorized to do? (write files, call APIs, commit code, invoke sub-agents) | Loop cannot perform actual work |
| Check | How is output validated? (run tests, lint, typecheck, LLM-as-judge) | Loop always "succeeds" even when output is wrong |
| Stop | When does the loop terminate? (goal met, max retries reached, human escalation triggered) | Loop runs indefinitely, consuming API budget until exhausted |
Trigger and Stop are the two components most commonly underspecified — yet they bookend everything else. A loop without a reliable exit condition is an uncontrolled cost center.
The Evaluation Trap: pass@k vs pass^k
When agent behavior is non-deterministic, conventional testing intuitions break. The Anthropic engineering blog (Demystifying Evals for AI Agents) defines two critical metrics:
- pass@k: probability that at least one of k attempts is correct
- pass^k: probability that all k attempts are correct
They're identical at k=1. As k grows, they diverge completely:
| k (attempts) | pass@k (at least one success) | pass^k (all succeed) | Engineering implication |
|---|---|---|---|
| 1 | p | p | Identical |
| 5 | 1-(1-p)⁵ | p⁵ | Diverging |
| 10 | → near 100% | → near 0% | Opposite signals |
(Assuming independent per-attempt success probability p = 0.7)
This divergence surfaces a fundamental design choice:
- If you're designing a loop, you need pass^k — each run must be reliable because downstream decisions depend on the output. An agent that fails 3 out of 10 times produces silent errors that accumulate through the pipeline.
- If you're evaluating capability headroom, pass@k is appropriate — it tells you whether the model can do this, not whether it will do it reliably in production.
Conflating the two is the most common loop evaluation error: a high pass@k score deployed into a pass^k-demanding production environment produces confident, plausible, frequently wrong output.
The Security Cost of Autonomy
Letting a loop run fully autonomously introduces two threat categories that barely exist in manual prompting scenarios:
Threat 1: Agentic Denial of Service / "Denial of Wallet"
Attackers inject logical paradoxes or self-replicating subtasks into a loop, causing API call volume to explode and exhaust budget in minutes.
This isn't theoretical. Sysdig documented a real incident on AWS Bedrock where a compromised loop system generated $46,000 per day in API charges (a technique since labeled "LLMjacking"). A separate incident involved a leaked Google Gemini API key accumulating $82,000 in 48 hours (March 2026). OWASP LLM Top 10 2025 classifies this as LLM10 (Unbounded Consumption).
Threat 2: Indirect Prompt Injection
Loops retrieve content from external environments (web pages, codebases, emails, databases) and process it. That content can carry hidden malicious instructions that get executed when the agent processes them — moving the attack surface from "your prompt" to "your retrieved data."
CVE-2025-32711 (EchoLeak) is a confirmed instance: attackers crafted malicious email content that triggered Microsoft 365 Copilot to perform unauthorized actions, fully exploiting indirect prompt injection. OWASP LLM01 (Prompt Injection) ranks this as the top LLM risk.
| Threat Type | Attack Vector | Known Incidents | Defense Principle |
|---|---|---|---|
| Agentic DoS / Denial of Wallet | Logical paradoxes, self-replicating subtasks | AWS Bedrock $46K/day; Gemini key $82K/48h | Hard token/cost caps; max iteration count; budget circuit breaker |
| Indirect Prompt Injection | External content retrieved during execution (emails, web, docs) | CVE-2025-32711 (EchoLeak, M365 Copilot) | Input isolation; least-privilege tool calls; output content marking |
Both threats share a root cause: a loop raises the trust level of external content from "read-only" to "executable." In manual prompting, a human in the loop serves as an implicit filter; a fully autonomous loop removes that filter.
Legitimate Skepticism: Is This Just Rebranded Cron?
Loop engineering faces a pointed critique: this is just Airflow, Temporal, or a cron job with an LLM substituted in. Swap the scheduler for a loop and the task definition for a prompt — the underlying mechanism hasn't fundamentally changed.
This critique has real force. The core orchestration mechanics — triggers, state management, error handling, retry strategies — are identical to what workflow engineers have been building for a decade. Commentators including those at StationX have directly labeled it "rebranded automation."
The critique remains unrefuted in any empirical sense. The loop engineering field currently lacks:
- Any peer-reviewed paper formally defining the concept
- Controlled comparisons showing "loop engineering thinking" produces measurably better systems than "workflow orchestration thinking"
- A broadly accepted formal definition (as of June 28, 2026, the term is approximately three weeks old)
Where the critique falls short is in conflating mechanics with the conceptual frame. Even if the underlying machinery is similar, the frame changes what questions you're required to answer. Building a loop with agents at the center forces you to directly confront a set of problems that traditional workflow engineering doesn't surface:
- Non-deterministic evaluation (pass@k vs pass^k, and which one you're actually optimizing)
- Indirect prompt injection as a primary attack surface, not a niche edge case
- A cost model measured in tokens rather than time or compute
- Accountability mechanisms for autonomous decisions that cause harm
This doesn't make loop engineering an independent discipline — it may simply be a specialization of harness engineering. But it's also not pure renaming.
Open Questions
Four questions emerged from research that have no satisfying answers yet — they mark the genuine frontier:
-
What's the formal boundary between loop engineering and Airflow/Temporal/LangGraph? Is there a workable criterion beyond "has an LLM in it"?
-
How do pass@k and pass^k recompose in hierarchical multi-agent systems? When sub-agent non-determinism compounds across layers, do these metrics still apply, or do they need to be rebuilt?
-
When a fully autonomous loop causes data loss or a security incident, where does accountability lie? Which elements (circuit breakers, audit logs, human escalation gates) are engineering best practices versus compliance requirements?
-
When the "model" is itself a multi-agent loop, how does test-time scaling strategy guidance change? arXiv 2512.02008 established that no single TTS strategy universally dominates (optimal depends on problem difficulty × model type × compute budget) — but that was scoped to single LLMs on math and science benchmarks. What happens when you're scaling loops?
Conclusion
Loop engineering is real — but it is currently in a practice-before-theory state. Boris Cherny's observation describes a real shift in how AI engineers work: from driving agents manually to designing systems that drive themselves. Addy Osmani gave it a name and a preliminary framework. But the field hasn't had enough time to be thoroughly stress-tested.
My position: this is a direction worth tracking, not one worth fully committing to yet. Specifically:
- Do: Start identifying which manual prompt-response cycles in your current work can be driven autonomously. Build one small loop with hard exit conditions and a cost ceiling. Validate pass^k before expanding scope.
- Don't: "Loopify" every agent interaction, or let loops run fully autonomously without reliable check and stop conditions. That's the path to denial of wallet and accumulated silent errors.
- Watch: pass^k, not pass@k; the design quality of your Stop condition and Check condition; the trust boundary on external inputs.
Cherny says his job is to write loops. Working at that level is real leverage. But a poorly designed loop costs an order of magnitude more than a poorly designed prompt.
References
- Boris Cherny (Anthropic) — Meta @Scale conference remarks, June 2026; widely cited in secondary sources
- Cobus Greyling — Loop Engineering — The core of Loop Engineering (Medium, June 2026)
- Addy Osmani — Loop Engineering (addyosmani.com, June 7, 2026)
- Ben Dickson — Demystifying loop engineering: Get more from AI agents, avoid loopmaxxing (TechTalks, June 22, 2026)
- Anthropic Engineering — Demystifying Evals for AI Agents
- arXiv 2512.02008 — The Art of Scaling Test-Time Compute for Large Language Models (December 2025)
- OWASP — LLM Top 10 2025: LLM01 Prompt Injection / LLM10 Unbounded Consumption
- CVE-2025-32711 — EchoLeak: Indirect Prompt Injection via Email Content in Microsoft 365 Copilot
- Sysdig — LLMjacking on AWS Bedrock ($46,000/day incident documentation, 2026)
- truongpx396 — The Agentic Loop: A Practical Field Guide (DEV Community, June 25, 2026)
- arXiv 2510.23883v2 — Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges