digital-human-koubo Deep Dive: The Pipeline Behind One-Photo Talking-Head Videos

The host presenting that course in the livestream clip does not exist. In October 2026, an open-source repo published this 27-second shopping ad: the face, the voice, the studio background, the animated captions, the price sticker — all AI-generated. The human input was one photo and one sentence: "Make a 30-second livestream ad with this digital human, selling a beginner course, 10 lessons for 99 yuan." The author reports it cost 482 platform credits.

The repo is digital-human-koubo by Berryxia, MIT-licensed, created on 2026-10-26. As of 2026-10-28 it counts 1,129 stars and 268 forks. A thousand stars in two days is common on GitHub these days; what is uncommon is the route it validates: a talking-head video pipeline that needs no SaaS subscription and no local GPU, run end to end by a coding agent orchestrating a cloud generation API and local editing scripts.

Not an App, a Playbook

The clone is 55MB with no package.json and no src/. The entire repo is two Agent Skill folders: digital-human-koubo (the main pipeline) and finance-explainer (the engine for editing style A). Each is a SKILL.md playbook plus Python/Shell scripts plus Remotion video templates.

Driving it takes an agent with three capabilities: it can install skills, connect to a remote MCP server, and run local commands. Claude Code and Codex both qualify; every demo video in the repo was produced with Claude Code.

This choice of granularity is the repo's most valuable decision. The production chain splits into three layers: process knowledge lives in the skill (script conventions, slicing rules, QC checklists, known failure modes), compute lives in the cloud API (Grok Video 1.5 for lip-synced segments, GPT Image 2.5 for scene plates), and verification plus assembly stay in local scripts (ffmpeg, whisper.cpp, Remotion). Swap the generation model and the workflow survives; swap the edit template and generation is untouched; upgrade QC and you edit one script.

We have covered OpenMontage, which approaches agentic video production with 12 pipelines and 500+ skills aiming at a general-purpose studio. koubo narrows to one genre, talking-head, and tunes every step until the outcome is reproducible. The scoring section weighs that trade-off.

Architecture: Generation in the Cloud, QC and Assembly on Your Machine

The cloud side has exactly one dependency: Renoise. It is an AI generation aggregator that pools models such as Grok Video, GPT Image, Seedance, and Kling behind a credit balance. Its MCP endpoint (https://renoise.ai/api/public/mcp) authenticates through browser OAuth; no API key gets copied anywhere. koubo uses two of its models: Grok Video 1.5 for lip-synced talking segments and GPT Image 2.5 for scene plates and four-view character sheets. The README also documents Wan 3.0 and OmniHuman (audio-driven lip sync) usage, but as of 2026-10-28 neither appears in the model list on Renoise's site — verify platform support before relying on them.

The local side needs four tools: ffmpeg, Python 3 (numpy/Pillow), Node 18+ (Remotion), and whisper.cpp. All CPU work, no graphics card required — the direct payoff of splitting generation from assembly.

The persona is fixed in persona.json: the reference image's material id on Renoise, a measured speech rate of 5.4 characters per second, two resolution tiers (720p full-screen, 480p picture-in-picture), the English prompt template for Grok (which ends with "no subtitles, no on-screen text", because Grok occasionally burns subtitles into the frame uninvited), and an act library per shot type (full shot, PiP explainer, vlog selfie, depth medium shot). Beyond the default digital host, other_hosts defines a personal avatar and two animal hosts (an orange-cat shopkeeper, a shiba), so multi-host relay and multilingual output are solved at this layer.

Nine Steps, Three Gates

The flow runs from an environment check (doctor.sh, which lists install options for anything missing) through content diagnosis, scripting, quoting, generation, download, QC, voice finishing, and Remotion rendering. The engineering details deserve a closer look.

Script slicing comes first. Each line of the script must fit 15 seconds (Grok's per-segment cap); duration is estimated as ceil(characters / 5.4 + 1.0) and clamped to 4–15 seconds, where 5.4 chars/second is the measured speech rate. Japanese and Korean use 7 characters/second, Latin scripts 2.6 words/second. A segment estimated over 15 seconds raises an error that forces you to split the line, rather than silently truncating it.

Then come three gates, all designed around one fact: generated content burns credits and cannot be refunded.

1. The quote gate. build_tasks.py computes segments, seconds, and estimated credits, cross-checks your balance with estimate_cost, and only calls create_tasks after you confirm. Every task carries an idempotencyKey (dh-<project>-<segment>-v1), so re-runs never double-charge.
2. The first-segment gate. Before batch generation, one segment is produced for review; the remaining segments only go out after you approve the voice and lip sync. Generation is submitted in batches of 10 tasks.
3. The QC gate. Every segment is transcribed by whisper and compared against the script; four frames per segment are extracted to catch Grok burning subtitles into the frame. The SKILL.md says it plainly: the prompt cannot fully prevent it, so frame sampling is the backstop, and a hit segment gets reshot.

The transcription step hides another whisper trap: passing --prompt with the original line makes whisper drop the first half of some segments, so alignment uses a neutral prompt. The repo documents at least six of these only-learned-by-running details.

The download step carries real engineering too: Renoise's US CDN often truncates parallel downloads, so rangedl.py splits each file into 6 concurrent HTTP ranges with resume support, then validates the result by decoding it with ffmpeg.

Voice finishing decides whether 30 separately-generated segments sound like one recording. voicefx/unify.py does four things: matches the energy of 22 one-third-octave bands (100 Hz–12.5 kHz) toward the reference median (±4 dB via ffmpeg's firequalizer); de-esses and tames harsh bands with three EQ cuts; adds a consistent room reverb scaled by how much each segment already has; normalizes to -18 LUFS. The author's measured number: inter-segment sibilance variance dropped from 13.2 dB to 5.9 dB. During editing the avatar video track stays muted and the voice plays on its own track, avoiding comb filtering from millisecond-level misalignment.

The Credit Bill: Priced per Second per Resolution

Unit prices (as documented in the repo; live quotes govern): Grok Video 720p at 17 credits/second, 480p at 10 credits/second (used for picture-in-picture, 40% cheaper), GPT Image 2.5 2k at about 23 credits/image.

Three real bills the author published:

Output	What was generated	Credits
Livestream ad · 9:16 · 27s	1 scene plate + 5 segments of 720p speech	482
Step-by-step tutorial · 16:9 · 4m36s	32 speech segments (720p full / 480p PiP)	~3,250
Multi-host multilingual · 16:9 · 5m55s	30 new segments + 7 scene plates, 17 segments reused	~3,409

The first bill reconciles exactly: 482 = 23 (scene plate) + 459 (27 seconds × 17 credits/second). At these rates, a 3-minute finished video costs roughly 2,000–3,500 credits to generate.

One caveat: credit pricing is a black box. Renoise's MCP page does not publish a credit-to-currency rate; it only states that prices match the website, generations cannot be cancelled or refunded, and accounts are limited to 300 requests/minute. You learn what a credit costs on the website's pricing page after signing up. The mandatory quote gate frames the risk to "the order you confirmed".

Our earlier comparison of eight open-source AI video generation tools covered tools whose billing unit is "a clip". koubo pushes granularity down to seconds × resolution, fine enough that the script stage can price the entire video — a real contribution of pipeline-izing generation.

48 Components Distilled from 25 Videos

The 48 packaging components were not invented from thin air. The author frame-by-frame deconstructed 25 videos from top creators (MrBeast, Mark Rober, MKBHD, MediaStorm-style Chinese tech channels, Johnny Harris, Vox, Kurzgesagt, and several top Chinese explainer channels), published the teardown reports under references/research/, and distilled them into playbook rules inside the skill: a new visual event every 3–5 seconds in the first 30; the thesis stated in the first 3 seconds, no logo intro; BGM runs the whole video while sound effects mark events only (1–3 per minute in top Chinese talking-head videos); the cheapest pacing device for a single-camera digital human is jump-cut push-ins at sentence boundaries (1.0× / 1.18× / 1.32×).

Components are declared in a pack array in the Remotion template, triggered by "segment:word" (for example "3:credits" fires when the word "credits" is spoken in segment 3), auto-adapted to 9:16, and statically checked by check_edit.py before rendering.

The depth-text component TextBehind (large type sandwiched between the person and the background, composited in three layers) is the repo's only functional platform lock-in: per-frame person segmentation uses macOS Vision (personmask.swift, compiled by swiftc on first run) with no ready substitute on Windows or Linux. One component out of 48 is unavailable off-macOS.

Running It on Windows: Yes, with Four Pitfalls

Our original question was whether this runs on Windows. After reading the code: yes. Generation happens in the cloud, leaving only CPU work locally; the official docs even give winget commands, and doctor.sh recognizes Git Bash (classifying MINGW/MSYS as win) to list Windows install paths. whisper.cpp ships official Windows builds, and the ggml-medium-q5_0.bin model is about 540MB from HuggingFace.

Four pitfalls, the first two requiring code edits:

1. Templates hardcode macOS fonts. The Chinese font stack in the caption template is "PingFang SC","Hiragino Sans GB", which degrades to generic sans-serif on Windows and visibly degrades the typography. Fix: add "Microsoft YaHei" to the font stacks in the template files.
2. finance-explainer's sheet.py hardcodes /System/Library/Fonts/Menlo.ttc, which throws FileNotFoundError on Windows; switching it to arial.ttf is a one-line fix.
3. node_modules cloning falls through. new_edit.sh first tries to clone dependencies from a Remotion project on the author's machine (an APFS clone), which never exists on Windows, so it falls back to npm install. Point npm at the npmmirror registry first; the dependency tree includes three.js and is not small.
4. The depth-text component is unavailable (see above): 47 of 48 components work.

Deployment is four commands: install Git for Windows (which bundles Git Bash); winget install Gyan.FFmpeg Python.Python.3.12 OpenJS.NodeJS.LTS; copy the two skill folders into %USERPROFILE%\.claude\skills\; install the Renoise plugin in Claude Code and sign in via the browser. Then run doctor.sh until it reports all clear.

Route	Representative	Billing	Packaging freedom	Onboarding cost
SaaS hosting	HeyGen / CapCut avatar	Monthly subscription + quota	Template-based, in-app controls	Lowest
Local open-source models	SadTalker / MuseTalk family	Free, electricity	DIY everything	High (GPU and tuning)
Agent orchestration (this repo)	digital-human-koubo	Pay-per-credit	Remotion code-level, 48 components	Medium (needs an agent workflow)

Our take: SaaS fits people who never want to touch a terminal; local open-source fits teams with GPUs and algorithm needs; the route koubo represents suits people already living in a coding agent who want a reproducible, customizable production line — better cost structure and controllability, at the price of binding generation to a single cloud platform.

Six-Dimension Score

Dimension	Score	Basis
Pipeline engineering	9/10	Three gates, idempotencyKey, 6-range resumable download, spectral voice matching — production-grade details throughout
Generation quality	8/10	Author reports 94–100% transcription accuracy across six languages; subtitle-burning has a QC backstop; not independently reproduced here
Documentation and failure-mode capture	9/10	Every encountered pitfall is written back into the skill; doctor.sh automates environment checks
Out-of-box experience	5/10	Not an install-and-run app; requires an agent, four local tools, and a cloud signup
Platform compatibility	6/10	Cross-platform but four Windows pitfalls; one component is macOS-only
Autonomy and portability	5/10	Generation bound to Renoise; edit templates MIT and modifiable; Remotion's license is free only for individuals and companies of ≤3

A disclosure: this is a source-code reading plus published-data review; we did not spend credits to reproduce generation quality. All cost and quality figures come from the repo's docs and scripts, as of 2026-10-28.

Conclusion

The most valuable thing in this repo is not the digital host itself; it is the granularity the author chose for AI content production: process knowledge distilled into a skill, compute outsourced to a metered cloud API, verification and assembly kept in local scripts. The three gates (quote confirmation, first-segment review, dual QC) and details like the idempotencyKey are worth copying wholesale by any team building agent-orchestrated content generation. It is the same standard we applied in our Scout review: with a two-day-old repo, take the design rather than enshrine the tool.

Advice by reader type: individuals and small teams already using coding agents can adopt it directly, watching Remotion's ≤3-person license terms and credit costs; Windows users can route around the four pitfalls above since generation spares local hardware; engineers building video products should read it as a pipeline design reference, starting with build_tasks.py's billing precision and voicefx/unify.py's finishing chain.

References

1. andyhuo520 (Berryxia) — digital-human-koubo repository (created 2026-10-26, MIT license; star/fork counts as of 2026-10-28)
2. Repository source files — SKILL.md, persona.json, scripts/build_tasks.py, scripts/voicefx/unify.py, scripts/rangedl.py
3. Renoise — MCP integration page (free to connect; generation billed in credits at live website prices)
4. whisper.cpp — Releases (Windows builds) and the ggml-medium-q5_0.bin model (~540MB)
5. Remotion — company license terms (free for individuals and companies of up to 3 people)