Harness Engineering and Loop Engineering: Building the Skeleton and Gait of AI Agents
Why the Model Is No Longer the Bottleneck
By mid-2025, mainstream LLMs had converged in capability. Claude Sonnet 4, GPT-4.1, and DeepSeek V4 perform similarly on pure reasoning, code generation, and tool-use benchmarks. Yet in real-world usage, the experience gap between AI agent products is enormous — some can autonomously complete hours-long complex tasks, others fall apart in three steps.
Where's the difference? Not in the model, but in the two layers of engineering around it:
- Harness Engineering: the static scaffolding — everything wrapping the model
- Loop Engineering: the dynamic control flow — how the model stays coherent across many calls
Analogy: the model is the CPU, the Harness is the motherboard + I/O + drivers, the Loop is the OS scheduler. No matter how powerful the CPU, a bad motherboard or chaotic scheduling means the system won't run.
Part 1: Harness Engineering — Building a Good Skeleton
What Is a Harness
A harness is all the static infrastructure around a single model call:
- Tool definitions and execution environment
- Context assembly (system prompt, files, memory, retrieved docs)
- Output parsing and format constraints
- Permission gates and safety checks
- Error handling and retry logic
Anthropic has repeatedly emphasized across their 2025 engineering blog posts: the model is the raw engine; the harness is everything else that makes it useful.
Core Elements
1. Tool Design
Tools are the harness component that most directly impacts model performance. Anthropic's "Writing Tools for AI Agents" provides key principles:
- Write tool descriptions for the model, not for humans — describe parameters and return values in language the model can understand
- Return values should be structured so the model can extract the information it needs for the next step
- Error messages should be actionable — not "failed" but "permission denied, you need to run chmod first"
- Tool count isn't the problem; tool description quality is
2. Context Assembly
Context is a finite resource. The Manus team's "Context Engineering for AI Agents" distills real-world lessons:
- KV-cache stability: avoid frequently modifying the front of the system prompt, or every call recalculates attention for the entire context
- Tool masking, not removal: instead of removing unused tools from context, mark them unavailable — removal breaks the cache, masking doesn't
- Filesystem as memory: write intermediate results to files rather than stuffing them into context; read them back when needed
- Keep errors in context: don't clear failed tool call results — the model needs them to adjust its strategy
3. Safety and Permissions
The harness must place a safety layer between the model and the real world:
- Tiered command approval: low-risk auto-approved, high-risk requires human confirmation
- File access sandboxing: restrict readable/writable paths
- Network isolation: restrict APIs and domains the model can reach
- Output filtering: prevent leaking sensitive information
4. Output Parsing
Model output isn't plain text — it may contain tool calls, code blocks, structured data. The harness needs:
- Robust parsers: handle malformed output, truncation, mixed formats
- Type validation: check tool parameter types
- Fallback strategy: how to recover when parsing fails
Harness Design Principles
Anthropic's "Building Effective Agents" distinguishes "workflows" from "agents":
- Workflows: predefined paths where the model executes in fixed steps — controllable, predictable
- Agents: the model autonomously decides its path — flexible but unpredictable
Key principle: start with workflows, introduce agentic loops only when flexibility demands it. Don't jump straight to fully autonomous agents — solve the problem with a fixed pipeline first, then gradually loosen the model's degrees of freedom.
Part 2: Loop Engineering — Giving the Model a Good Gait
What Is a Loop
A loop is the agent's dynamic control flow — how the model stays coherent across multiple calls:
observe → think → act (tool call) → get result → repeat
Until a stop condition is met or a step limit is reached.
Core Elements
1. Stop Conditions
The most overlooked yet most important design decision:
- Task completion: how does the model know it's done? Needs clear success criteria
- Step limit: a safety net against infinite loops
- User confirmation: pause before critical steps for human approval
- Cost threshold: stop when token consumption exceeds budget
OpenAI's "A Practical Guide to Building Agents" stresses: every agent must have explicit termination conditions, or it will consume indefinitely.
2. State and Context Management
A loop spans multiple calls — how does state persist between iterations:
- Short-term memory: current conversation context — grows with iterations, needs compression
- Long-term memory: cross-session persistent memory — user preferences, environment info
- Context compression: when context approaches the window limit, how to summarize without losing critical information
- Checkpoints: periodically save state, support rollback
3. Feedback Quality
The shape of tool return values directly determines the quality of the next iteration:
- Return values should contain all information the model needs to decide the next step
- Error messages should be actionable — not just "failed" but "failure reason + suggested fix steps"
- Large outputs should be summarized or paginated, not dumped into context all at once
4. Error and Retry Handling
Model tool calls will fail. The loop needs to decide:
- Retry: try again with the same parameters (network hiccup?)
- Adjust: try a different approach (permission denied? fix permissions first)
- Degrade: complete with a simpler method (complex tool failed? manual steps as fallback)
- Abort: give up and report failure (unrecoverable error)
5. Pacing
- One big step vs. many small ones: complex tasks are more controllable when broken into small steps
- Parallel vs. serial: independent subtasks can be parallel, dependent ones must be serial
- Batching: combine multiple similar operations into one call
Classic Loop Patterns
ReAct Pattern
The foundational agentic loop, from Yao et al.:
Thought → Action → Observation → Thought → ...
The model alternates between reasoning (Thought) and acting (Action), observing results before deciding the next step. This is the basis of most AI agents today.
Source: ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., ICLR 2023)
Reflexion Pattern
Adds self-reflection to ReAct: after failure, don't just retry — first summarize why it failed, then re-attempt with the reflection in context.
Source: Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al.)
Plan-Execute Pattern
First plan a complete sequence of steps, then execute them one by one. Works well when the task structure is clear but has many steps. Plans can be dynamically adjusted during execution.
Multi-Agent Orchestration
Multiple agents collaborate, each handling different subtasks. Anthropic's multi-agent research system is a canonical example: an orchestrator agent splits tasks, worker agents execute in parallel.
Source: Anthropic — How We Built Our Multi-Agent Research System
But the Cognition team warns: don't rush to multi-agent. A single-threaded agent sharing full context often beats multiple agents passing fragmented context between them.
Part 3: The Relationship Between Harness and Loop
| Dimension | Harness Engineering | Loop Engineering |
|---|---|---|
| Nature | Static scaffolding | Dynamic control flow |
| Scope | Single model call | Across many calls |
| Core question | What goes in, what can be done, what comes out | When to stop, how to learn, how to recover |
| Metaphor | The room and its tools | How to walk around the room |
| Design goal | Make each call correct | Make many calls coherent |
They are complementary: a good harness ensures each call has correct inputs and safe outputs; a good loop ensures many calls form a coherent workflow. Without a harness, the model takes dangerous or meaningless actions; without a loop, the model loses direction in complex tasks.
Examples in Real Systems
- Claude Code: Harness = tool definitions (Bash/Edit/Read/Write) + context assembly + permission approval; Loop = ReAct pattern + context compression + checkpoint rollback
- Manus: Harness = tool set + filesystem memory + KV-cache management; Loop = observe-think-act + tool masking + error retention
- Hermes Agent: Harness = tool registry + skill loading + security scanning; Loop = delegate_task subagents + context compression + cron scheduling
Part 4: Curated Reference Materials
Foundational Papers
| Paper | Core Contribution | Link |
|---|---|---|
| ReAct (Yao et al., 2023) | Foundational reasoning+acting loop paradigm | arxiv.org/abs/2210.03629 |
| Reflexion (Shinn et al., 2023) | Self-reflection loop, summarize-then-retry | arxiv.org/abs/2303.11366 |
| CoALA (Sumers et al., 2024) | Cognitive architecture taxonomy for language agents | arxiv.org/abs/2309.02427 |
| AutoGen (Microsoft) | Multi-agent conversation framework | arxiv.org/abs/2308.08155 |
Engineering Practice Blogs
| Article | Highlight | Link |
|---|---|---|
| Anthropic — Building Effective Agents | Workflows vs agents, when to use which | anthropic.com |
| Anthropic — Effective Context Engineering | Context as finite budget, just-in-time retrieval | anthropic.com |
| Anthropic — Writing Tools for AI Agents | Tool descriptions for the model, actionable errors | anthropic.com |
| Manus — Context Engineering Lessons | KV-cache stability, tool masking, filesystem memory | manus.im |
| Cognition — Don't Build Multi-Agents | Single-threaded shared context > multi-agent fragments | cognition.ai |
| Chip Huyen — Agents | Systematic overview: planning, tool use, failure modes | huyenchip.com |
| Lilian Weng — LLM Powered Autonomous Agents | Best single-article map: planning+memory+tools | lilianweng.github.io |
Framework and Tool Documentation
| Doc | Use Case | Link |
|---|---|---|
| Google — Agents Whitepaper | Model+tools+orchestration reference architecture | kaggle.com/whitepaper-agents |
| OpenAI — A Practical Guide to Building Agents | Loop, guardrails, handoffs, stop logic | cdn.openai.com |
| LangGraph | State machine/graph model agent loops, persistent checkpoints | langchain-ai.github.io/langgraph |
| Model Context Protocol | Tool-plane standard | modelcontextprotocol.io |
Design Pattern Overview
Andrew Ng's four agentic design patterns remain the most concise taxonomy:
- Reflection: self-critique and improve
- Tool Use: call external tools to extend capabilities
- Planning: decompose tasks into sub-steps
- Multi-Agent: multiple agents collaborating
Source: Andrew Ng / DeepLearning.AI — Agentic Design Patterns
Part 5: Practical Recommendations
If you're building your own AI agent
- Start with workflows, then add agents — validate core logic with fixed pipelines, then gradually introduce model autonomy
- Tool descriptions are your leverage — time spent on tool descriptions pays off more than switching to a bigger model
- Context is a budget — every token has a cost; compression, summarization, and filesystem memory are essential skills
- Stop conditions before everything else — an agent without termination conditions is a loop that burns money infinitely
- Errors are the best teacher — keep failed tool calls in context; the model will adjust its strategy from them
If you're choosing an agent framework
- Need full control: write your own Harness + Loop, referencing the materials above
- Need rapid prototyping: LangGraph (state machine model) or Claude Agent SDK
- Need multi-agent: AutoGen or Anthropic's multi-agent orchestration pattern
- Need tool ecosystem: MCP protocol + compatible tools
Conclusion
Harness engineering and loop engineering are not two separate domains but two faces of the same problem: how to make LLMs work reliably in the real world.
While everyone chases bigger models, what truly separates the winners is the engineering around the model. Why is Claude Code good? Not because the Claude model is that much stronger, but because Anthropic invested heavily in Harness (tool design, context management, permission control) and Loop (ReAct cycle, context compression, checkpoints).
The model is the engine. The Harness is the chassis. The Loop is the transmission. When all three match, the car runs fast and steady.