Qwen3-ASR: The Open Whisper Alternative You Can Run Locally

This is another entry in our Local AI Stack series. In the Whisper guide we set up local speech-to-text with the model everyone reaches for by default. Whisper is still excellent. It is also no longer the only serious open option, and for a lot of real audio it is no longer the best one.

Qwen3-ASR is Apache 2.0, comes in 0.6B and 1.7B sizes, and the two Transformers-native builds have pulled over 206,000 downloads. The headline you will see is "it beats Whisper." That is too simple, and the real picture is more useful.

TL;DR

• Two models, Apache 2.0, 0.6B and 1.7B, covering 52 languages and dialects (30 languages plus 22 Chinese dialects).
• It is not a Whisper killer, it is a Whisper complement. Whisper still edges it on pristine English audio. Qwen wins on noisy audio, accents, Mandarin and Cantonese, sometimes by enormous margins.
• Not new, despite the recent buzz: released 2026-01-29. What changed is native Transformers support on 2026-06-26, which is what made it a two-line install and drove the download numbers.

What Qwen3-ASR actually is

Two all-in-one speech recognition models from the Qwen team, built on the Qwen3-Omni foundation model. They do language identification and transcription in a single pass, and they handle more than clean dictation: speech, singing voice, and songs with background music are all in scope. Offline and streaming inference run through one unified architecture rather than separate models.

Sizes and adoption, from the Hugging Face API:

On the Hugging Face Open ASR Leaderboard the 0.6B posts a mean WER of 6.31 percent and the 1.7B posts 5.59 percent. Those are aggregate English numbers, and on their own they undersell the model, because the interesting behaviour is in the spread rather than the mean.

Where it beats Whisper, and where it does not

Here is the project's own benchmark table against Whisper-large-v3. Word error rate, so lower is better, and the winner in each row is bold.

The two Librispeech rows are the ones worth sitting with, because they are the same corpus split by difficulty. "Clean" is studio-quality read speech, and Whisper wins it. "Other" is the harder, noisier split, and Qwen wins that. That single pair tells you the shape of the whole comparison: Whisper is marginally better on pristine audio, and Qwen is better as soon as the audio gets difficult.

Then look at Cantonese. Whisper posts 31.04 percent WER, which is roughly one word in three wrong, meaning it is not usable. Qwen posts 4.12 percent. That is not an incremental gain, it is the difference between a feature you can ship and one you cannot. Mandarin shows the same pattern less dramatically, and accented English sits in between.

So the honest summary is not "Qwen3-ASR is better." It is that Whisper was trained in a way that made clean English its strongest case, and Qwen3-ASR is much more robust everywhere else. If your audio is podcast-quality American English, your existing Whisper pipeline is fine. If it is phone calls, accented speakers, background noise, or anything Chinese, this is a meaningful upgrade.

Running it

Native Transformers support is the whole reason this is now easy. One install:

pip install "transformers>=5.13.0"

Then transcribe. The processor has a dedicated transcription-request helper, so you do not hand-build the prompt:

from transformers import AutoProcessor, AutoModelForMultimodalLM

model_id = "Qwen/Qwen3-ASR-0.6B-hf"   # or Qwen/Qwen3-ASR-1.7B-hf
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")

inputs = processor.apply_transcription_request(
    audio="https://example.com/audio.wav",
).to(model.device, model.dtype)

output_ids = model.generate(**inputs, max_new_tokens=256)
generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]

transcription = processor.decode(
    generated_ids, return_format="transcription_only",
)[0]
print(transcription)

audio= takes a local path as readily as a URL. Language identification is automatic, so there is no language flag to set unless you want to pin one.

If you are pushing volume, the project documents torch.compile giving roughly a 2.4x speedup on an A100 at batch size 4, and supports vLLM for serving.

Word-level timestamps

The ASR models transcribe but do not align. If you need per-word timings, for subtitles or for seeking into a recording, that is the separate Qwen3-ForcedAligner-0.6B model. This is a design decision worth knowing before you plan a subtitle pipeline: Whisper gives you segment timestamps in the same pass, whereas Qwen splits transcription and alignment into two models. Two steps, but the alignment model is small and you only pay for it when you need it.

Hardware reality

Be careful here, because the parameter counts invite a wrong assumption. A 0.6B model sounds like something that runs comfortably on a laptop CPU, and it may well. But the project does not document CPU inference at all. Every piece of guidance is GPU-shaped: cuda:0 in the examples, vLLM for serving, FlashAttention 2 for speed, A100 numbers for the benchmarks.

That does not mean CPU is impossible. It means nobody has published what to expect, so if your plan is CPU-only transcription, budget time to find out for yourself rather than treating it as a supported path. The 0.6B is the obvious candidate if you try.

For comparison, this is a real advantage Whisper retains: years of community work has produced whisper.cpp and a mature quantized CPU story. Qwen3-ASR has the better model on hard audio; Whisper has the better ecosystem on constrained hardware. That trade is the actual decision.

Which should you use

Stay on Whisper if your audio is clean English, if you are running CPU-only or on a Raspberry Pi class device, or if you depend on the surrounding tooling (whisper.cpp, faster-whisper, the many wrappers). None of that is displaced.

Switch to Qwen3-ASR if you transcribe Mandarin or Cantonese, where the gap is not close. Also switch if your English audio is accented, noisy, or recorded over a phone, where the Librispeech "other" and accented-English rows say you will measurably do better.

Run both if you are building a product. They are both Apache 2.0, both small, and routing by detected language costs you almost nothing.

The ten-minute move: take the worst piece of audio you have, the one your current pipeline mangles, and run it through the 0.6B with the snippet above. The aggregate benchmarks will not tell you whether this helps your data. That one file will.

Sources and further reading

• Our guide to running Whisper locally
• QwenLM/Qwen3-ASR on GitHub (Apache 2.0, benchmark tables)
• Qwen3-ASR-0.6B on Hugging Face
• Qwen3-ASR-1.7B on Hugging Face
• Qwen3-ForcedAligner-0.6B on Hugging Face
• Qwen3-ASR technical report (arXiv 2601.21337)

Tested on: not independently tested. Our bench has no NVIDIA GPU, and the project documents no CPU path, so we did not run inference. Every WER figure above is quoted from the project's own published benchmark table against Whisper-large-v3, not from a third-party summary, and the download counts and Apache 2.0 licensing were read directly from the Hugging Face API on the date below. Release dates come from the repository's own changelog.
Date checked: 2026-07-25