VibeVoice's Half-Open Source: The Weights Are Still There, the Inference Code Isn't

Pull the commit history of microsoft/VibeVoice and something odd shows up immediately. The oldest surviving commit is dated 5 September 2025, SHA 8107244b, and its message is a single word: statement.

Its parents field is an empty array. This is a root commit — the entire history from the original 25 August 2025 launch was force-pushed out of existence, and the repo restarted from nothing. What it restarted with was eight files: .gitignore, LICENSE, SECURITY.md, four images, and a 96-line README. Zero Python files.

The opening paragraph of that README read:

After release, we discovered instances where the tool was used in ways inconsistent with the stated intent. Since responsible use of AI is one of Microsoft's guiding principles, we have disabled this repo until we are confident that out-of-scope use is no longer possible.

Today (2026-08-17) is month eleven. The repo itself came back to life long ago — 52,793 stars, 140 commits, ASR and realtime models progressively open-sourced, integrations landed in Hugging Face Transformers and Azure AI Foundry. But the specific thing that sentence was about has not returned. Not one piece of it.

I've written about this project before, in Microsoft VibeVoice Deep Dive, which covered the technical stack: the 7.5 Hz continuous tokenizer, next-token diffusion, ASR-BitNet's heterogeneous quantization. This piece isn't about the stack. It's about a thornier question: when an MIT-licensed project ships its weights and its code separately, what exactly is left of "open source"?

---

1. The exact shape of the gap

"The TTS code is gone" isn't worth much as repeated news. So I pulled the three modeling files off main and grepped them one by one. The gap turns out to have a much more specific shape than the headline suggests.

$ grep -n "def generate\|GenerationMixin" *.py

modeling_vibevoice_asr.py:11:  from transformers.generation import GenerationMixin
modeling_vibevoice_asr.py:152: class VibeVoiceASRForConditionalGeneration(
                                   VibeVoiceASRPreTrainedModel, GenerationMixin):

modeling_vibevoice_streaming_inference.py:9:   from transformers.generation import GenerationMixin, ...
modeling_vibevoice_streaming_inference.py:163: class VibeVoiceStreamingForConditionalGenerationInference(
                                                   VibeVoiceStreamingPreTrainedModel, GenerationMixin):
modeling_vibevoice_streaming_inference.py:574: def generate(
modeling_vibevoice_streaming_inference.py:887: def sample_speech_tokens(self, condition, neg_condition, cfg_scale=3.0):

ASR has it. Streaming realtime has it — generate() at line 574, CFG sampling included. And modeling_vibevoice.py, the 495-line main model for long-form multi-speaker TTS, matches nothing at all.

Then look at the class itself:

# modeling_vibevoice.py:213
class VibeVoiceForConditionalGeneration(VibeVoicePreTrainedModel):
    ...
# end of file
AutoModel.register(VibeVoiceConfig, VibeVoiceModel)
AutoModelForCausalLM.register(VibeVoiceConfig, VibeVoiceForConditionalGeneration)

Those two lines are the ones worth staring at. The class is registered as an AutoModelForCausalLM, yet it does not inherit GenerationMixin. Since transformers 4.50, generate() lives on GenerationMixin rather than PreTrainedModel — so AutoModelForCausalLM.from_pretrained() will happily load the full 1.5B weights, and .generate() gives you a flat AttributeError.

The only public methods left are forward() and forward_speech_features(). And forward()'s signature says plainly what it's for:

def forward(self, input_ids=None, ...,
            speech_tensors=None, speech_masks=None,
            speeches_loss_input=None, speech_semantic_tensors=None,
            acoustic_input_mask=None, acoustic_loss_mask=None,
            ddpm_batch_mul: int = 1, **kwargs):

acoustic_loss_mask, ddpm_batch_mul — these are training-path arguments, not inference-path ones. What remains is the half that computes the diffusion loss. What was removed is the half that iteratively denoises back into acoustic features.

The demo/ directory tells the same story: of six scripts, four are ASR and two are Realtime. None are TTS. And in the README's model table, the "Quick Try" column for TTS-1.5B reads Disabled.

So this isn't stale documentation or code hiding on another branch. It's precise surgery: the entire inference path for long-form multi-speaker synthesis excised, everything else left intact.

---

2. Why the gap can't be closed — and won't be

Here's the question that usually gets skipped: why did Microsoft restore ASR and Realtime, but never TTS?

Put the three models' capabilities side by side and the answer is blunt:

Model	Core capability	What misuse looks like
ASR-7B	60-minute transcription + diarization	Surveillance/transcription — barely about the model
Realtime-0.5B	Single-speaker streaming synthesis	Generates speech, but can't become a specific person
TTS-1.5B / 7B	Zero-shot cloning from seconds of reference audio, 4 speakers, 90 minutes	Is the deepfake

For the first two, "out-of-scope use" is a side effect you can fence off. For TTS, the abusable capability is the product. Its entire pitch is: take a reference clip, reproduce anyone's voice, and have that reproduction converse naturally with three others for ninety minutes.

Which is why "until we are confident that out-of-scope use is no longer possible" remains unfulfilled eleven months later. It doesn't set a temporal condition. It sets a logically unsatisfiable one. Watermarks, AI disclosures, README disclaimers — none of them change what the weights can do.

My read is that this gap is permanent, not pending. Any technical decision premised on "Microsoft will eventually put it back" is betting on something that isn't going to happen. It's the mirror image of huggingface/speech-to-speech, which I've dissected previously: a lower capability ceiling, but every layer swappable and no single link hanging on one company's compliance decision.

---

3. The shadow supply chain it created

The interesting part is that the code was removed while the weights were not fully recalled. That asymmetry produced a supply chain that is still running fine today.

I hit all four model repos against the Hugging Face API (measured 2026-08-17):

Model	Status	Downloads	Likes
microsoft/VibeVoice-ASR	Public	695,866	1,270
microsoft/VibeVoice-Realtime-0.5B	Public	666,107	1,271
microsoft/VibeVoice-1.5B	Public, not gated	114,807	2,459
microsoft/VibeVoice-Large (7B)	HTTP 401	—	—

Two numbers deserve a pause.

First, 1.5B is the most-liked model in the family (2,459, nearly the sum of ASR and Realtime) — and it is the only one you cannot officially run. The weights page is public, ungated, one huggingface-cli download away from the full parameter set. You just don't get the ~200 lines that start it.

Second, the 7B really was pulled: the API returns 401. But it didn't disappear. The community re-upload aoi-ot/VibeVoice-Large sits at 2,433 downloads and 236 likes, with AWQ-INT4 and GGUF quantizations circulating alongside it.

The code side is filled in by the community fork vibevoice-community/VibeVoice (1.5k stars / 682 forks, MIT), which preserves inference paths for 1.5B, 7B and streaming, plus unofficial fine-tuning scripts. But its last update was 2025-12-04 — none of Microsoft's subsequent ASR, BitNet or vLLM-plugin work is in it.

The most counterintuitive link is furthest downstream: fal.ai commercially sells "VibeVoice 7B" inference at $0.04 per generated minute, up to four speakers. Replicate hosts an endpoint too. A model whose own Microsoft Hugging Face repo won't open is, right now, being metered and billed by the minute.

I don't think anyone here is in violation — MIT explicitly permits commercial use and redistribution, and all of this became lawful the moment the weights shipped. That's exactly the point: taking down an MIT model is not a recall in law and is merely an inconvenience in engineering. What actually got reduced is official support, not availability.

---

4. If you want to use it today, three paths

Setting governance aside, the engineering decision comes down to three options. My verdict assumes the current state holds, since it has held for eleven months:

Path	What runs	Cost	My verdict
Official repo, self-hosted	ASR-7B / Realtime-0.5B only	None	Recommended for ASR. It's the only part with ongoing first-party maintenance, Transformers integration and an Azure endpoint
Community fork, self-hosted	1.5B / 7B multi-speaker TTS	8 months stale; third-party weight provenance; no security updates; MIT-licensed but unauditable supply chain	Not recommended. An unmaintained dependency chain in exchange for a capability the vendor abandoned
Hosted API (fal / Replicate)	1.5B / 7B multi-speaker TTS	$0.04/min; provider may delist on compliance grounds	Recommended when you genuinely need multi-speaker long-form. Outsourcing the compliance risk along with the ops beats self-hosting

Concretely, split by what you actually need:

If you need long-form transcription with diarization — turning an hour of meeting or podcast audio into timestamped structured text — go straight to the official ASR-7B. It's the most solid part of this project today: 60 minutes in a single pass, 50+ languages, a BitNet quantization that runs on CPU, and none of it sits anywhere near the grey zone this article is about.

If you need multi-speaker conversational voiceover, answer one question honestly first: do you need "multi-speaker long-form," or do you need "zero-shot cloning of one specific person"? If it's the former, segment the script, synthesize each turn with Realtime-0.5B or any single-speaker TTS, and concatenate — entirely workable, and it sidesteps the whole problem. That's the same orchestration logic I worked through in the Seed-Audio deep dive. Only when you truly need four voices held consistent across ninety minutes does VibeVoice TTS become irreplaceable — and at that point, use the hosted API.

The one path to actively avoid is self-hosting the community fork. It combines three liabilities at once: a model the vendor abandoned, inference code frozen for eight months, and weights of unauditable provenance. Put those together in production and the upside is saving $0.04 a minute while nobody on the chain is accountable for any of it.

---

5. The general lesson

VibeVoice matters beyond itself. It drags into the open something usually left vague: "open source" in the model era names at least four things that can be shipped separately.

• Weights (VibeVoice: TTS 1.5B shipped, 7B recalled)
• Modeling code (shipped — but only the training path)
• Inference code (not shipped)
• License (MIT, the most permissive tier)

The license is the loudest item and the least consequential. MIT promises you may do anything; it has never promised the thing runs. When a company wants a model back but doesn't want to be seen breaking an open-source promise, deleting the inference code is the cheapest move available: not a word of the license changes, not a star is lost, the README stays up — only the glue that turns weights into audio is gone.

So when evaluating any "open-source model," add one check: don't read the license, check whether demo/ has a matching script, and whether the model class inherits GenerationMixin. Both take under a minute and tell you more than ten readings of the LICENSE file.

---

Conclusion

microsoft/VibeVoice is now two projects sharing one repository: a speech-recognition project that is actively evolving, first-party maintained and safe to adopt; and a speech-synthesis project with public weights, missing code, kept alive by community mirrors and third-party APIs.

For ASR needs, it's among the best open options available, with no reservations.

For multi-speaker TTS, the right posture isn't waiting for Microsoft to restore it. It's accepting that the gap is deliberate and most likely permanent — then either going through a hosted API or choosing a path that doesn't depend on zero-shot cloning at all. "Until we are confident that out-of-scope use is no longer possible" is not a progress bar. It's a polite no.

As for the shadow supply chain already running — weights circulating through community mirrors, commercial APIs billing by the minute — it reads like a conclusion that arrived early: once weights are copyable, delisting a model is only ever a rerouting of distribution, never a recall. That fact matters equally to companies deciding what to open-source and to engineers deciding what to build on. It just means opposite things to each.

---

References

1. Microsoft — microsoft/VibeVoice GitHub repository (as of 2026-08-17: 52,793 stars / 5,957 forks / 140 commits)
2. Microsoft — Root commit 8107244b "statement" (2025-09-05, contains the original takedown notice)
3. Microsoft — vibevoice/modular/modeling_vibevoice.py (495 lines, no generate())
4. Microsoft — modeling_vibevoice_streaming_inference.py (906 lines, generate() at line 574, used as the control case)
5. Microsoft — VibeVoice Technical Report (arXiv:2508.19205, August 2025)
6. Hugging Face — microsoft/VibeVoice-1.5B (download and like counts verified via HF API on 2026-08-17)
7. Hugging Face — aoi-ot/VibeVoice-Large (community re-upload of the 7B)
8. Community — vibevoice-community/VibeVoice (fork retaining inference code, last updated 2025-12-04)
9. fal.ai — VibeVoice 7B Text to Speech API ($0.04 per generated minute)
10. Replicate — microsoft/vibevoice
11. Hugging Face — VibeVoice ASR in Transformers