Anyone who has shipped real work through a browser agent knows the same failure modes: the page mutates between "read" and "click", the selector points at a node that no longer exists, or the site simply blocks automated traffic. The engineering community has a mitigation for each, and the leaderboards have settled their rankings. On October 13, 2026, Outcrawl open-sourced Scout, their production browser agent; it collected 642 GitHub stars in two days, with a headline claim of 92.5% versus 81.4%. We put this two-day-old project through four benchmark tasks and three real business missions — and hit, in the second mission, the situation no leaderboard covers: the brand's official domain had been hijacked.
This piece answers three questions. How far does Scout's engineering actually go? What is that 92.5% number measuring? And when the correct answer isn't at the URL you were given, what does an agent do? All measurements below were taken on October 15, 2026: local headless Chromium, no cloud, with gemini-3.8-flash on any chat-completions-compatible endpoint as the driving model.
What Scout Is
Scout is a library, not a product. You write a few lines of TypeScript — new Scout({ browser, models }).run(task) — and it opens a real browser, reads pages, clicks, types, scrolls, and brings back an answer. No CLI, and no built-in model: all four model roles take your own keys and speak the chat-completions protocol, including models running on your own machine.
const scout = new Scout({
browser: { launch: {} }, // local Chromium; or 'outcrawl' cloud / a CDP URL
models: {
hands: { model: 'gemini-3.8-flash', baseURL, apiKey }, // the only required role
brain: { model: '…' }, // optional: planning
eyes: { model: '…' }, // optional: screenshots
extract: { model: '…' }, // optional: schema extraction
},
});
const result = await scout.run('Open the first book in the Poetry category on books.toscrape.com and report its title and price');
The repository size is disproportionate for a two-day-old project: roughly 60,000 lines of TypeScript source against 33,000 lines of tests, Apache-2.0, with the input-planning sub-package agenthands under its own MIT license. It is a one-way mirror of Outcrawl's production repository — 16 sync commits, and a merged PR returns to the mirror only with the next sync. The largest package is runtime at 26,000 lines, and the main agent loop, loop.ts, is a single file of 14,000 lines. That file is both the most impressive and the most glaring thing about this codebase; both sides come up below.
The Core Mechanism: Freezing the TOCTOU Window Away
The classic browser-agent failure is TOCTOU — time-of-check to time-of-use. The agent reads a page, the model thinks for five to fifteen seconds, the page changes underneath, and the action lands on a node that is no longer what was inspected. The header comment of loop.ts surveys how the industry handles this today: Playwright's refs are invalidated when the page changes, browser-use regenerates its indices after every action, and Stagehand does not freeze the page at all, paying for staleness with self-healing — one measured burn cited in that comment cost 21,410 tokens to heal a single action. Scout takes the page-level lock across the entire think-and-act phase: the page is frozen while the model thinks, and the action executes against the frame-qualified node ID captured at lock time. The window is eliminated structurally, not mitigated. Exactly one thing may take the freeze down mid-step — an in-page captcha solve, because a frozen page cannot run challenge scripts. Where Scout concentrates complexity into a single deterministic loop, the decentralized canvas-agent school distributes orchestration across collaborating surfaces — opposite bets on the same problem.
The second design worth unpacking is the four-role model split. hands runs on every step, so it sets the run's speed and cost. brain is the optional planner, and its input is bounded by hard budgets: each step receives one compact state line plus the current page brief, while history lives as append-only compact state lines in a thread that rides the prefix cache. eyes appears only when the agent asks to look at a screenshot; absent, the agent works entirely from the accessibility tree. extract, when absent, is handled by hands. Each role can point at a different provider with a different key, and structured output degrades in three tiers: tool_choice by default, then json_object for endpoints without function calling, then a strict json_schema — and the failure message tells you which role needs which switch.
The contract is equally restrained. Every run returns a result with one of four statuses (completed, partial, capped, failed); caps (steps, duration, budget) end a run as a normal outcome; configuration errors are rejected before any browser opens. In our tests, pointing Scout at a non-existent domain returned status: failed with a one-sentence accurate error and exit code 0 — failure is data here, not an exception.
Benchmarks: Four for Four, Including the Failure Path
All four benchmark tasks passed on the first attempt (October 15, 2026, local headless Chromium, hands=gemini-3.8-flash):
| Task | Result | Time | Steps |
|---|---|---|---|
| First Poetry-category book on books.toscrape | matches the official demo answer | 35.1s | 4 |
| Full Poetry catalog via a JSON Schema | 19 books, every field intact | 31.3s | 3 |
| Log in on quotes.toscrape, then read across pages | login succeeded, data correct | 82.4s | 9 |
| Unreachable domain (failure path) | status: failed, accurate error |
27.6s | 3 |
Two details deserve note. The default tool_choice structured-output mode worked against a Gemini-family endpoint with zero configuration; all 19 books came back with prices and stock states, verbatim. The login mission took 9 steps for form-filling plus two pages of data gathering, with no wasted navigation. For a purely local run — your own key, no cloud — that is above the passing line.
Real Missions: Three Tasks, Three Harnesses
Benchmarks pass; then comes real work. The missions come from the material-research workflow of a generative video product (we surveyed why AI remake pipelines are unstable in a previous piece), and all three briefs are that pipeline's daily prep. Locate a viral short video's full metadata for a remake pipeline; pull a flagship product's name, price, hero image and tagline from a niche internet brand's official site; find the official full-length recording of a long keynote and honestly assess whether the page exposes any heat or chapter signals for clip positioning (turning "where is the highlight" into a computable signal is an old problem in short-video engineering). The same three briefs went to three executors: Scout, opencli v1.8.8 (a CLI bridged to a real Chrome), and a coding agent's built-in browser driven step by step by the coding agent itself.
| Mission | Scout | opencli | Coding agent's built-in browser |
|---|---|---|---|
| Remake: viral video metadata | ✅ 49.3s / 4 steps | ✅ 7 commands | ✅ 78.6s / 4 calls |
| Brand site (hijacked domain) | ⚠️ partial, 202.9s / 20 steps | ✅ 2 commands to surface the anomaly | curl verdict — tool-independent |
| Brand site (healthy domain) | ✅ 514.8s / 22 steps | ✅ 12 commands | ✅ 69.1s / 2 calls |
| Clipping: official full recording | ✅ 169.8s / 6 steps | ✅ 5 commands | ✅ 30.8s / 2 calls |
On every verifiable field the three harnesses agree: two Bilibili video IDs, uploader names, durations and view counts cross-confirm each other, with zero hallucination. On the clipping mission Scout explained its negative finding — "no episode list in the right rail, no timeline in the description, no heat bar in the player" — separating "not found" from "here is why it isn't there". Most agents only manage the first half.
Speed is a different story. The coding agent driving its built-in browser needed 2-4 calls and 31-79 seconds per mission; opencli took 5-12 commands; Scout, with one LLM call per step, ran 49 to 515 seconds across the four missions, and its slowest run mixed in exploratory navigation across a full order of magnitude more actions. One of four Scout runs also ended with a broken CDP connection — the browser process crashed and the mission closed as partial. Stability is the hardest gate between Scout and production.
The Hijack: One Agent's Unassisted Recovery
The second mission ran into what no leaderboard covers. Our "niche internet brand" was Saturnbird Coffee, and the brief said to open its official site, saturnbirdcoffee.com. On test day that domain served a server-side 301 to crownjunk.com — a gambling SEO spam site. A brand's own domain changing hands is unremarkable in 2026; for a browser agent it is a brutal exam, because the URL in the brief is wrong and the correct answer lives somewhere else entirely.
opencli surfaced the anomaly in two commands — the page title and body were casino spam, and judgment passed to whoever (or whatever agent) was driving it. The one that walked the entire path alone was Scout, and its behavior chain is worth recording in full:
It navigated to saturnbirdcoffee.com and landed on the crownjunk spam page. Then its extraction log produced this line: "saturnbirdcoffee.com was redirected to crownjunk.com" — it identified the hijack instead of extracting the spam site's content as an answer. Next came a search-engine descent: Google (unreachable from our network), then DuckDuckGo, then Bing, and it located the genuine overseas storefront at saturnbird.com, from which it extracted the product name and official tagline. The price field genuinely does not exist on the brand site, so it did not invent one: the run ended as status: partial with an honest error naming what was missing. Twenty steps, 202.9 seconds, plus one CDP disconnect at the very end.
Set next to the engineering above, this run was not luck. The accessibility tree of a casino spam page is semantically far from "coffee brand", so the agent had enough signal to know something was wrong; and the contract — always return a result, mark it partial, name what's missing — meant it never had to choose between inventing an answer and crashing. The flip side: it burned roughly 40% of its steps trial-and-erroring across search engines. A caller with domain allowlist validation would have stopped this hijack at step one. That is the takeaway for anyone wiring browser agents into a production pipeline: the agent will self-recover, but validating the domain is the caller's job, and neither substitutes for the other.
What 92.5% Actually Measures
Back to the README's headline. Outcrawl's methodology page (three full runs, October 7, 2026) is candid about scope: the metric is whether a page can be successfully opened, on Browser Use's 80-site benchmark, 71 scored sites. Outcrawl's cloud browser behind Scout scored 92.5%; the next best, Browser Use Cloud, scored 81.4%. The page also discloses the two details that matter most: the competitor figures come from Browser Use's own published runs of March 2026, driven by Browser Use's agent; Outcrawl's runs are its own agent on its own cloud, seven months later. So 92.5% measures cloud-browser anti-blocking — residential exits, captcha solving, humanized input. It is not a score for agent intelligence, and it says nothing about Scout's local, open-source operation; the headless-Chromium control in the same benchmark scored 2.2%.
With the number properly filed, the benchmark earns respect rather than suspicion: the dimension it measures is real (our own runs hit an unreachable Google and a Bilibili login wall, both reminders of that dimension), and Outcrawl never claims the cloud number as an agent score in its README. Most people quoting it, though, never read the methodology page.
Scores
Across both test rounds and the source read (October 15, 2026):
| Dimension | Score | Basis |
|---|---|---|
| Core mechanism design | 9/10 | action-lock removes TOCTOU structurally; four-role split with hard budgets; three-tier structured-output fallback |
| Engineering quality & auditability | 9/10 | 60K lines of source against 33K of tests; design constraints documented at the code site |
| Measured task success | 8/10 | 7 of 8 missions fully correct; the only partial came from an external hijack, with a complete recovery path |
| Speed & cost efficiency | 5/10 | one LLM call per step, 49-515s per mission; one browser crash in four runs |
| Ecosystem & maturity | 3/10 | 2 days old, v0.1.1, one-way mirror, unsettled API |
| Documentation honesty | 8/10 | methodology page discloses the date and driver mismatch; the README headline still invites misquotation |
The scores below 9 are not aimed at the code; they are aimed at time. A 14,000-line loop.ts whose entry point starts at line 8556 is painful to read in a mirror repo, and two days with three npm versions means any interface may change next week.
Conclusion
Three sentences. First, Scout is the highest-quality open-source agent codebase I have read in the past six months: action-lock, the four-role split, and the failure-is-data contract deserve wholesale study by anyone building an agent runtime, regardless of maturity. Second, do not put it into production today — wait for v0.2, community issue history, and a verdict on CDP stability; at its current speed and crash rate it fits unattended batch jobs that tolerate minute-scale latency and per-run retries. Third, the hijack lesson belongs to every caller, not to Scout: an agent can recover on the URL it was given, but whether that URL is still worth visiting must be validated by the pipeline — an allowlist plus fingerprint check is one line of code that would have saved 40% of the wasted steps.
On the buying question: if your goal is anti-blocking scraping, what you are buying is the Outcrawl cloud, the true source of the 92.5%. Scout's open-source value is the engineering exemplar and local control; keep the two accounts separate.
References
- Outcrawl — outcrawlai/scout GitHub repository (open-sourced 2026-10-13; 642 stars and v0.1.1 as of 2026-10-15)
- Outcrawl — Browser Use 80-site benchmark methodology (runs on 2026-10-07)
- npm — @outcrawl/scout package (v0.1.1, 2026-10-13)
- TOCTOU analysis cited in the loop.ts header — arXiv:2603.00476
- Browser Use — 80-site anti-blocking benchmark and control data (comparison runs published 2026-03)
- jackwener — @jackwener/opencli (v1.8.8, one of the comparison harnesses)
- All measurements (2026-10-15) reproduced by the author on local headless Chromium with gemini-3.8-flash; briefs and raw logs retained in the evaluation workspace