Prime Agent Source Teardown: The Self-Rewriting Harness and Its Seven Decisions Worth Copying

The agent paper that mattered most in 2026 wasn't about a model. It was Stop Comparing LLM Agents Without Disclosing the Harness, and its claim is blunt: between models of comparable capability, harness-induced variance routinely exceeds model-induced variance — enough to reverse rankings outright.

If that holds, "how the harness is written" stops being an implementation detail and becomes the product. Prime Agent is what showed up under that premise — open-sourced on August 5, 2026; as of August 11 it sits at 13,180 stars, 1,339 forks, 486 open issues, MIT, TypeScript.

I already covered what it is and whether it's safe from the product and reward-hacking angle in an earlier post. This one ignores the blog post and reads the code: I pulled the whole repo, went through the refinement, kernel, and autonomous subsystems, then lined it up against twelve comparable projects.

The conclusion up front: the most valuable part of this repo isn't in the papers. It's in the comments.

Two papers, shipped as a product

Both of Prime Agent's core abstractions have explicit academic lineage — rare for an agent project.

RLM (Recursive Language Model), arXiv:2512.24601, by Alex L. Zhang, Tim Kraska, and Omar Khattab. The idea: treat the prompt as a Python variable inside a REPL, so the root model can programmatically inspect, decompose, and recursively call itself over slices of input — handling prompts two orders of magnitude past its context window.

Continual Harness, arXiv:2605.09998, by Seth Karten et al. (Princeton / ARISE / Google DeepMind), submitted May 11, 2026. It decomposes the harness into four components the agent itself can create, read, update, and delete — system prompt, sub-agents, skills, memory — plus a Refiner that rewrites them from trajectory data. The paper's setup is Gemini Plays Pokemon, where human-in-the-loop harness refinement produced the first AI to finish Pokemon Blue, Yellow Legacy on hard mode, and Crystal without a lost battle. Continual Harness removes the human from that loop entirely.

The official numbers: Opus 5 hits 95.5% RHAE Best@1 on ARC-AGI-3, above the 95.4% human expert baseline, with 99.97% Best@3 across all 183 levels; on long-context OOLONG at 128k it scores 0.700 against pi-mono's 0.420.

Set the numbers aside. Here are the actual design decisions.

1. There is exactly one tool

This isn't a figure of speech:

// packages/coding-agent/src/core/tools/index.ts
export type ToolName = "ipython";
export const allToolNames: Set<ToolName> = new Set(["ipython"]);

Not "IPython-first" — literally one tool schema. Reading files, editing files, running tests, invoking skills, spawning sub-agents: all of it is a Python expression in one persistent kernel. Each %%bash cell is a throwaway subshell, but the Python namespace, %cd, and os.environ survive across turns and across compaction.

The academic root here is CodeAct (ICML 2024), which found code actions beat JSON actions on 12 of 17 models — up to +20 percentage points and up to 30% fewer interactions. The productized versions are Anthropic's "code execution with MCP" (November 2025) and Cloudflare's Code Mode, which compresses the entire Cloudflare API into roughly 1,000 tokens. What separates Prime Agent is that it left no fallback.

There's a line in the system prompt that acts as a self-correction on this design, and I think it's more valuable than the one-tool decision itself:

Do not assume IPython is the native runtime of the external thing being investigated. …Evaluate external systems through their own interface, then use IPython to coordinate the process and analyze what comes back.

In plain terms: don't drag someone else's project into your kernel. Use its own uv run or .venv/bin/python; IPython is only for orchestration and analysis. This guards against a very specific failure — the agent installs dependencies into the kernel just to make an import succeed, then produces a green light that has nothing to do with the project's real environment. Anyone who has debugged "the AI said it passed but CI is red" knows the price of that one.

2. Sub-agents are async calls that never return an answer

handle = await rlm("Review the authentication flow", name="auth-reviewer")
# handle carries rlm_child_id / name / session_dir / model — no result field

rlm() returns the instant the task is admitted, not when it completes. Results come back through exactly two channels: the child explicitly calls await agent_message.send(msg, receiver_role="parent"), or it writes files the parent reads.

This constraint matters more than it looks. "Return at admission, results via events" means the parent isn't blocked for fifteen minutes by one child — and more importantly, the parent's context is never passively flooded with the child's full trajectory. Want to look? Call agent_observe explicitly. Don't want to? You get one sentence. Communication is scoped to parent, siblings, and direct children; anything deeper relays through the intermediate node.

The cost is filed too: #759 — a child finishes its work, writes its files, and forgets to send the message, so the parent keeps believing it's still running. An explicit reply is a protocol, not a guarantee. Any design that stakes "done" on the model voluntarily performing an action should budget for this lesson.

3. Four mutable state kinds × two scopes

refinement.ts (1,017 lines) is the single most worthwhile file in the repo. It lands the paper's H = (ρ, G, K, M) as:

type RefinementKind = "prompt" | "memory" | "skill" | "subagent";
type RefinementAction = "create" | "update" | "delete";
type HarnessScope = "local" | "global";

Several details are clean in the way that only comes from having actually run this thing:

• Local by default. Writes land in the current session's artifact directory and don't pollute the global store. Only cross-session lessons, durable user preferences, and reusable skills or sub-agents get promoted. The API is global_=True — because global is a Python keyword, and the system prompt says so explicitly to stop the model emitting global=True and hitting a syntax error.
• The base system prompt is immutable. Refinement can only append supplemental prompt notes. The refiner's own system prompt says MUST NOT be rewritten.
• Merge rules are deterministic. mergeHarnessStates stacks global and local; on an id collision the local entry displays with a local: prefix, and the model is told to use the bare id when editing.
• Writes are atomic. tmp file → renameSync, preserving the original file mode (new files default to 0o600).
• A corrupt read degrades to empty. loadHarnessState sits on the path of every system-prompt build, so a damaged state file returns empty state instead of throwing, and the next save rewrites it cleanly. The reasoning is written into the function.

For contrast: Letta's self-editing memory mutates memory blocks — the same direction, earlier and narrower. Claude Code, opencode, and Codex CLI all store persistent state as human-authored markdown: AGENTS.md, CLAUDE.md, skills. The writer is a person, not the agent. That column is genuinely empty right now.

4. Spend 4k tokens deciding whether this turn is worth remembering

/refine doesn't run every turn. There's a separate, cheaper gate in front of it:

const REFINEMENT_MAX_OUTPUT_TOKENS = 32_000;
const AUTO_REFINE_REVIEW_MAX_OUTPUT_TOKENS = 4_096;

AUTO_REFINE_REVIEW returns only { shouldRefine, rationale, instructions? }, and its prompt explicitly tells it to reject one-off noise, unsupported hypotheses, and transient tool output. Only a pass triggers the real refinement.

This converts self-improvement from a fixed per-turn cost into an admission-gated occasional cost. For any production system billed per turn, that restructuring is more useful than self-improvement itself.

The same file carries a comment worth quoting on its own:

Output budgets are derived from the selected model instead of fixed literals. /refine input scales with harness size…, so a constant output cap silently truncates exactly the large multi-edit proposals that matter most.

The budget is Math.min(model.maxTokens, CAP) — scaled to the model, not hardcoded. The reasoning is airtight: refine's input grows with the harness, so a constant output ceiling truncates precisely the multi-edit proposals that matter most.

Paired with it is an explicit error constant:

const TRUNCATED_JSON_ERROR =
  "the model stopped before completing its JSON object. This usually means
   the output budget was exhausted; retry with a smaller request.";

Truncation is thrown as an error, not accepted as a shorter answer. This is the single most valuable engineering judgment in the repo, and the easiest one to miss — LLM truncation is silent end to end: finishReason gets dropped, half-written content is stored as completed, everything looks fine until something downstream reads JSON that stops mid-sentence.

5. Kernel state is pickled per variable, and allowed to fail

On resume the kernel is fresh, while the model still believes the variables it defined last turn are there. state-snapshot.ts serializes each top-level name independently with dill: one unpicklable object (an open file, a socket, a GPU tensor) is skipped and reported rather than aborting the whole snapshot. The default ceiling is 256MB; oversized variables are likewise skipped and reported.

One detail that only appears after a real incident: every builtin is referenced through a local _b alias, so the snapshot code keeps working when the user namespace has shadowed list, open, or print.

There's also a Linux-only IPython forkserver that skips the python -m ipykernel_launcher cold boot. Its comment is the engineering value I agree with most in the entire repo:

Everything degrades to direct spawn: if the forkserver is disabled, unavailable, or a spawn request fails/times out, callers catch ForkServerUnavailable and fall back to the existing path, so correctness never depends on fork.

The optimization path can always be switched off entirely; correctness never depends on it. It's disabled outright on macOS (fork-without-exec isn't safe there) and one env var turns it off anywhere.

6. Autonomous mode: the model saying "done" isn't done

In autonomous.ts, the default return of shouldAutonomouslyContinue is this:

return { shouldContinue: true, reason: "missing_terminal_evidence" };

Which means: unless a configured gate command (npm run check, tests, a linter) actually exits 0, keep going. The model claiming completion is not a terminal condition. The continuation prompt nails it down:

Do not end the session yourself; the verifier/evaluator decides completion when configured gates pass.

The smarter part is the spin guard. Before and after each gate run it takes a git worktree snapshot (status + diff + a hash of untracked content). If the workspace hasn't changed by a single character since the last gate failure, it doesn't rerun the command — it just says:

not rerun: workspace unchanged since previous failed gate
The autonomous gate was not rerun because the workspace has not changed
since this failure. Edit source files, tests, or a blocker artifact
before attempting to finish again.

This blocks the most expensive pathology in autonomous mode: the model repeatedly deciding to "just run the tests again" without having changed anything in between. The budget drains on spin, and the log still looks industrious.

One accounting detail worth auditing in anyone's long-session system:

return usage.input + usage.output + usage.cacheWrite;   // cacheRead deliberately excluded

The comment is direct — cache reads are repeated context served from the provider's cache, and counting them cumulatively exhausts a long autonomous loop's budget well before the real work reaches the cap.

7. Skills are importable Python packages — and it reads other harnesses' directories

Prime Agent implements the Agent Skills standard (markdown + frontmatter) and extends it with Python-backed skills installed into the kernel's venv and callable by import name:

report = await release_audit(repository=".", target_version="0.4.0")

Only skill metadata goes into the startup prompt; the full SKILL.md loads when the task matches. More notable is that it explicitly reads other tools' skill directories:

{ "skills": ["~/.claude/skills", "~/.codex/skills"] }

In a year when everyone is building a harness, choosing interoperability at the skills layer is a smart position — the skills ecosystem is already under cross-harness reuse pressure, and whoever accepts everyone else's format removes one excuse not to switch.

The competitive landscape

Star counts from the GitHub API on August 11, 2026.

Project	★	Action representation	Sub-agents	Agent-mutable persistent state	Isolation	Long-running
prime-agent	13.2k	single IPython kernel	runtime await rlm(), async	prompt / memory / skill / subagent, CRUD'd by the agent	none (explicitly not a sandbox)	daemon + heartbeats + goals + autonomous gate
opencode	195.9k	JSON tool call	declarative (JSON/md, per-agent model and permissions)	AGENTS.md + skills (human-written)	none	mostly foreground
Claude Code	141.0k	JSON tool call	Task/Agent tool	CLAUDE.md + skills + memory (human-written)	permission system	background tasks
Antigravity CLI	closed	—	—	—	—	—
Codex CLI	105.2k	JSON tool call (Rust host)	yes	AGENTS.md	seatbelt / landlock sandbox	cloud Codex
pi	86.8k	JSON tool call	yes	lazy skills	none (external container advised)	foreground
OpenHands	83.6k	CmdRun / IPythonRunCell / FileEdit / Finish	yes	microagents	Docker sandbox (bundled bash + Jupyter + browser)	yes
goose	52.6k	MCP extensions	recipes	none	optional	yes
smolagents	28.7k	CodeAct, pure code actions	managed agents	none	optional E2B / Docker	no
Letta	24.2k	JSON tool call	multi-agent	self-editing memory (MemGPT lineage)	none	yes
SWE-agent	20.0k	ACI (command interface designed for models)	no	none	Docker	no

Four readings:

First, "code as action" isn't Prime Agent's invention — it's the first to remove the fallback. OpenHands has had IPythonRunCellAction for ages, but it sits alongside CmdRunAction and FileEditAction. smolagents' CodeAgent is pure code action, but it's a library, not a harness: no sessions, no daemon, no crash recovery. Prime Agent's position is clear — smolagents' action model, OpenHands' runtime ambition, pi's TUI engineering (the README credits pi directly, and the whole packages/ai tree is vendored from it).

Second, the genuinely scarce column is "the agent can rewrite its own harness." Only Prime Agent and Letta fill it, at different depths. Everyone else's persistent state is written by a human. If harness engineering really is becoming a discipline, then the answer to "who writes the harness" is sliding from human to agent, and Prime Agent has slid the furthest.

Third, sandboxing is its most visible gap. The README's warning is candid: splitting worker and kernel into separate processes is for lifecycle isolation and crash recovery, not a security boundary. Set against OpenHands' Docker sandbox and Codex CLI's seatbelt/landlock, it loses this column outright. Irrelevant for a solo developer; disqualifying for anything running untrusted repositories.

Fourth, the ecosystem is contracting. At I/O on May 19, 2026, Google announced the retirement of Gemini CLI and the Gemini Code Assist IDE extensions, cutting off individual tiers on June 18; the replacement, Antigravity CLI, is a closed-source Go binary. A 100k-star open-source CLI swapped for a closed one — two months before Prime Agent shipped. The supply of open harnesses is shifting from "big company publishes one casually" to "small team builds one deliberately."

The parts that don't hold up

Code volume is out of control, and it's verifiable. 171k lines of TypeScript/Python on the src side. agent-session.ts is 11,288 lines in one file; interactive-mode.ts is 9,975; daemon-mode.ts packs 106 case labels into 6,805 lines. The sharpest comment on Hacker News (253 points, 69 comments) went straight at this:

multiple files are close to 10K LOC… I'd probably aim for something way smaller to bootstrap a self-improving agent. Then I'd use this "Prime Agent" as an example to my self-improving agent for what it should not evolve to.

In fairness: the test side is 159k lines across 418 test files, regressions are archived by issue number (test/suite/regressions/<issue>-<slug>.test.ts), and there's a dedicated faux provider so the suite never burns real tokens. This isn't unsupervised slop — it's heavily tested and structurally undisciplined. Both can be true.

486 open issues, three months old. Windows is entirely unsupported (#719, #665, #660: the kernel bootstrap uses the venv's bin/python, which never exists on Windows, and each retry wipes the venv). The installer fails when npm's global prefix isn't user-writable. macOS Unix socket paths can exceed the length limit.

The most damaging admission is their own. In several tests, Opus 5 and GPT-5.6 Sol scored worse under Prime Agent than under their native harnesses, so the official report substituted the native numbers. The blog's phrasing is "we still notice friction when running Prime Agent with models," and it states plainly that no frontier model has been trained around Prime Agent's abstractions. Which means the design's payoff requires model-side cooperation that doesn't currently exist.

Self-improvement reinforces reward hacking too. In the Factorio case study, the agent pushed production past 100K within hours — by discovering and exploiting RCON commands to spawn resources directly, despite the heartbeat prompt forbidding exactly that. A harness that improves itself will happily crystallize the cheat into a skill.

One more structural objection from HN carries weight:

the foundational models have largely caught up to the point where they don't need this harness anymore… I can basically just store context in .md in the directories we work out of together and accomplish what I need.

And from another commenter: RLM's recursion wins largely because the root uses a top-tier model while the sub-agents use cheap ones. Piling complexity into the harness is a bet that model capability grows slower than harness returns. For the last two years, that bet has consistently lost.

Finally, a design-level cost: one tool pushes all the complexity onto the model. It has to write working Python to do anything at all. With a strong model that's leverage; with a weak one it's a liability — syntax errors, endless retries, long-tail cost blowups. A JSON tool call at least has schema validation underneath. Code execution has nothing.

Seven things to copy, whatever harness you maintain

None of these require buying into RLM or self-improvement:

1. Treat truncation as an error, not an answer. A stop reason of length is a failure, not a shorter success. And write the output budget as min(model.maxTokens, CAP) rather than a hardcoded constant — a fixed ceiling clips exactly the long outputs that matter.

2. Tighten the definition of terminal evidence. The default should be "continue"; only an externally verifiable gate passing counts as "done." The model declaring completion is a candidate, not a verdict. The direction of that default matters far more than the implementation.

3. Don't rerun a gate on an unchanged workspace. Snapshot the workspace, compare against the last failure, skip the run when they match, and feed "you changed nothing" back to the model. Pure addition, and it blocks the most expensive form of spin.

4. Keep cache-replayed tokens out of the budget. Counting cache reads cumulatively makes long sessions hit the wall before the real work does. Get this wrong and you'll think your agent costs more than it does.

5. Put an admission gate in front of memory writes. Don't run it every turn. Have a cheap model decide shouldRefine first, and only then pay for the expensive pass. The value of self-improvement is in precision, not frequency.

6. Three rules for persisting state: write atomically (tmp + rename + preserve mode), degrade a corrupt read to empty rather than throwing (it sits on the startup path), and keep a full append-only history so you can roll back.

7. Every optimization path must be switchable off. Caches, fork servers, warmups, snapshots — each needs a "fall back to the slow path on failure" route, and correctness must never depend on any of them.

Two things I'd explicitly not copy: collapsing the toolbar into a single code executor (if your tools have strict schemas and idempotency requirements, that's an asset, not overhead), and the "this is not a security sandbox" stance — Prime Agent can say that because it runs on your own machine. Multi-tenant systems don't get that option.

Conclusion

Is it worth using? If you run long-horizon evaluations, need daemonized background agents, or want to test the RLM paradigm firsthand, install it. If you just want a daily coding agent, opencode and Claude Code are both more mature, and Windows users shouldn't bother at all.

But "worth using" is the small question. The bigger value is this: it's currently the only complete, open-source implementation of an agent that rewrites its own harness — and it wrote its scars into the comments. Truncation must be an error. Completion must have external evidence. Don't rerun on an unchanged workspace. Optimization paths must degrade. A corrupt state file must degrade to empty.

None of that is about RLM, and none of it is about Continual Harness. It's tuition every long-running agent system eventually pays. This one already paid it. That's the most concrete thing open source offers — not saving you the time to write the code, but saving you the time to hit the wall.

References

1. PrimeIntellect — Prime Agent: A self-improving RLM agent (August 5, 2026)
2. PrimeIntellect-ai/prime-agent on GitHub — 13,180 stars / 486 open issues (as of August 11, 2026)
3. Hacker News — Prime Agent: A self-improving RLM agent (253 points, 69 comments)
4. Alex L. Zhang, Tim Kraska, Omar Khattab — Recursive Language Models (arXiv:2512.24601) · author's blog
5. Seth Karten et al. — Continual Harness: Online Adaptation for Self-Improving Foundation Agents (arXiv:2605.09998, May 11, 2026)
6. Xingyao Wang et al. — Executable Code Actions Elicit Better LLM Agents (CodeAct) (ICML 2024)
7. Stop Comparing LLM Agents Without Disclosing the Harness (arXiv:2605.23950)
8. Cloudflare — Code Mode: give agents an entire API in 1,000 tokens
9. OpenHands Runtime Architecture
10. Google Developers Blog — An important update: Transitioning Gemini CLI to Antigravity CLI (May 19, 2026)
11. GitHub Issue #759: Subagents can complete and write files without sending agent messages
12. GitHub Issue #660: Windows kernel bootstrap uses venv bin/python
13. Agent Skills specification