PersonaPlex-7B

License NVIDIA Open Model License

License NVIDIA Open Model License

TL;DR

• A 7B full-duplex speech-to-speech model that listens and speaks at once, replacing the recognition, language and synthesis chain with one network. Fine-tuned from Kyutai Moshi.
• 0.170 s smooth turn-taking latency and 0.240 s under interruption on Full-Duplex-Bench, with 0.65 speaker similarity on zero-shot voice cloning. Gemini Live still wins role adherence, 4.73 to 4.48.
• Trained in six hours on 8x A100 from Moshi weights, on 1,840 hours of synthetic dialog. MIT code, NVIDIA Open Model License weights, gated download, English only.

☍ Announcement ⬇ Download Model

System Requirements

RAM	16 GB system RAM
GPU	Any 16 GB NVIDIA GPU; CPU offload available
VRAM	About 16 GB at bf16, about 6 GB on community 4-bit builds

✓ Apple Silicon

Table of Contents

1. Why we are writing about a January model
2. What full duplex actually means
3. The numbers
4. Six hours on eight A100s
5. The licence stack
6. Running it
7. Nine months of ecosystem
8. Limitations
9. Who should use it
10. Sources and further reading

Your voice stack probably has three models in it. Speech recognition turns audio into text, a language model answers, text-to-speech turns the answer back into audio. Three hops, three places to add latency, and a system that physically cannot hear you while it is talking.

PersonaPlex-7B replaces all three with one model that listens and speaks at the same time. You give it a role in text and a voice in audio, and it holds a conversation you can interrupt. On Full-Duplex-Bench it answers an interruption in 0.24 seconds.

The weights are downloadable, the code is MIT, and the number that should interest anyone building on open models is that NVIDIA trained it in six hours on eight A100s.

Why we are writing about a January model

Straight up: this is not a new release. NVIDIA's ADLR group published the paper on 14 January 2026 and the repository has not been touched since 2 March.

It is here because of what happened in the nine months since. 162,834 downloads in the last thirty days. 2,831 likes on the model card, 11,175 stars and 1,531 forks on the code. Community ports to MLX, GGUF, ONNX and WebGPU. Fine-tunes into Hindi, Hinglish, Korean and Indonesian. And in June, Kyutai, whose Moshi model PersonaPlex is built on, published an RL-tuned version of NVIDIA's fine-tune of their own model.

Somewhere in there it stopped being a paper and became the default open answer to real-time voice. If you are putting a voice interface on anything in 2026, this is the model you have to argue your way out of using.

What full duplex actually means

Half duplex is a walkie-talkie. One side transmits, the other waits. Every assistant built on an ASR plus LLM plus TTS chain is half duplex by construction, because the recognizer has to decide you finished speaking before anything else can start. That decision is why assistants talk over you, and why they sit silently through a pause you did not mean as a handover.

Full duplex means both audio streams exist the whole time. PersonaPlex follows the Moshi architecture: it consumes your audio tokens and emits its own continuously, so backchannels ("mhm", "right") and interruptions are just things that happen in a stream rather than exceptions the system has to handle.

What NVIDIA added is control over who is speaking. They call it a Hybrid System Prompt, and it has two segments:

• A text segment that sets the role. Plain prose, no schema: "You work for Jerusalem Shakshuka which is a restaurant and your name is Owen Foster", followed by the facts the agent is allowed to know.
• An audio segment that sets the voice, cloned zero-shot from a sample. There is a neat detail here: during the voice prompt the user's audio channel is replaced with a 440 Hz sine wave, which keeps the model's two-stream structure intact while nothing real is being said to it.

You can skip the audio segment and pick one of 18 pre-packaged voice embeddings instead: eight labelled natural (four female, four male) and ten labelled variety. Skipping it also cuts latency, which is a sensible default for a product with one fixed voice.

The numbers

All of these are NVIDIA's own, from the paper. Nobody here has run Full-Duplex-Bench.

One term first. TOR is takeover rate: how often the model starts speaking. For pause handling and backchannels a low rate is correct, because it should stay quiet. For smooth turn taking and interruptions a high rate is correct, because it should respond. Latency is in seconds.

The checkpoint you can actually download is not the one in the paper's main tables, which matters, so these are the released figures:

Against the field, on the paper's experimental checkpoint, the gap that stands out is response quality under interruption:

Read the Moshi row carefully, because it is the entire point of this model. Moshi was already fast, at 0.257 seconds. What it could not do was say anything useful: 0.765 out of 5 on answer quality. PersonaPlex keeps duplex speed and brings the answer quality up by a factor of five. Qwen-2.5-Omni scores higher on the answer and takes nearly seven times as long to produce it, which in a live conversation is not a trade you can hide.

Speaker similarity is where the architectural choice shows. PersonaPlex scores 0.57 on cloning the requested voice (0.65 on the released checkpoint). Gemini scores 0.00, because it does not clone voices at all. That column is not a close contest, it is a feature nobody else in the comparison has.

Where it loses, honestly. On Service-Duplex-Bench, NVIDIA's own extension for customer-service roles, Gemini Live wins: 4.73 mean against PersonaPlex's 4.48, with Freeze-Omni at 4.02, Qwen-2.5-Omni at 2.76 and Moshi at 1.75. Gemini is also better at staying quiet, with a 0.255 pause takeover rate against 0.584 and 0.091 on backchannels against 0.327. A 7B open model losing to Google's production system on instruction adherence is the expected result, and the paper reports it plainly rather than burying it.

One curiosity from the ablation: training on half the data scored higher on Full-Duplex-Bench answer quality (4.52 against 4.21) while scoring lower on the service benchmark (4.24 against 4.48). More role-specific data bought role adherence and cost a little general conversational ability.

Six hours on eight A100s

This is the part worth copying.

PersonaPlex is not a model trained from scratch. It initialises from Moshi's weights and fine-tunes with the hybrid prompt. The full recipe from the paper: 24,576 steps, batch size 32, maximum sequence length 2,048 tokens (163.84 seconds of audio), Adam with cosine annealing, learning rate 4e-6 on the depth transformer and 2e-6 on the temporal one. Six hours on 8x A100.

Call it 48 GPU-hours to turn a fast-but-vacant duplex model into a persona-controllable one. At spot prices that is lunch money.

The training data is almost entirely synthetic, generated with open models:

The speech was produced by Chatterbox TTS and Dia, the voices in the released checkpoint come from Tortoise TTS with pitch and formant augmentation in Praat for privacy, the cloning samples are drawn from VoxCeleb, Libriheavy, LibriTTS, CommonAccent and Fisher, and the Fisher prompts were labelled at three levels of detail by GPT-OSS-120B.

Read that list again: an open duplex base model, open TTS models to manufacture the data, an open LLM to label it, 48 GPU-hours. That is a pipeline a small team can run. If you want a duplex agent with your brand's voice and your product's facts, the paper is effectively the instructions.

The licence stack

Three licences, and the permissive one is not the one on the weights.

The download is also gated: you accept the licence on Hugging Face and agree to share your contact information before you get the files, and you need an HF_TOKEN at runtime. That is a real consideration for an air-gapped build or a CI pipeline, and it is worth knowing before you design around it.

Running it

Install the Opus development library first, then the package:

sudo apt install libopus-dev
git clone https://github.com/NVIDIA/personaplex.git
cd personaplex
pip install moshi/.

Accept the licence on the model page, export your token, and start the server. It needs TLS for microphone access, so the repo generates throwaway certificates:

export HF_TOKEN=<your_token>
SSL_DIR=$(mktemp -d); python -m moshi.server --ssl "$SSL_DIR"

The web UI comes up on localhost:8998. If the GPU is short on memory, add --cpu-offload (it needs accelerate). For reproducible testing there is an offline path that takes a WAV in and gives a WAV out:

python -m moshi.offline \
  --voice-prompt "NATM1.pt" \
  --text-prompt "$(cat assets/test/prompt_service.txt)" \
  --input-wav "assets/test/input_service.wav" \
  --seed 42424242 \
  --output-wav "output.wav" \
  --output-text "output.json"

Two things that will cost you an evening. On Blackwell GPUs you need PyTorch from the CUDA 13.0 index, which is the repository's own first issue. And at bf16 the 7B weights want roughly 16 GB of VRAM, so a 12 GB card means either CPU offload or one of the community quantizations below.

Nine months of ecosystem

None of this is NVIDIA's, which is the interesting bit. The code is MIT, so people ran with it.

• Apple Silicon. Several independent MLX ports, including 4-bit and 8-bit conversions with real download counts, plus repositories offering local and web realtime modes and offline WAV inference.
• Consumer GPUs. GGUF q4_k and q8 builds, bitsandbytes 4-bit conversions, and a fork specifically aimed at making it fit in 16 GB.
• Browser and edge. ONNX exports and WebGPU q4_k builds.
• Languages. LoRAs and full fine-tunes for Hindi, Hinglish, Korean and Indonesian, which matter because the base model is English only.
• Serving. A Rust OpenAI-compatible server claiming 128 concurrent PersonaPlex sessions on a single GPU, which is the gap between the demo script and a product.
• Capability add-ons. Separate efforts on tool calling, voice diarization and speaker-emotion classification heads.
• And back upstream. In June, Kyutai published an RL-tuned variant of PersonaPlex. The lab that built Moshi took NVIDIA's fine-tune of their model and improved it further. That is the loop working exactly as it should.

For the pieces of a conventional voice stack this would replace, we have covered Qwen3-ASR as an open Whisper alternative on the input side and MisoTTS and Orpheus TTS on the output side. The closest relative in spirit is Sesame's CSM-1B, which also chased natural conversational speech but stayed half duplex.

Limitations

• English only. The model card lists one language. Everything else is a community fine-tune with no published evaluation.
• Nothing here is independently tested. Every figure is NVIDIA's, on NVIDIA's harness, including the benchmark extension they wrote themselves.
• The absolute naturalness scores are low. The released checkpoint's dialog MOS is 2.95 out of 5. It beats Gemini's 2.80 and Moshi's 2.44 in the same study, but the paper states outright that these scores are not comparable to its other table. Treat them as a ranking, not a grade.
• The repository is seven months stale. Last push 2 March 2026, with 70 open issues. The community is maintaining the edges, not NVIDIA.
• One voice per conversation. The voice prompt is set at the start. This is not a multi-speaker model.
• It is trained on service and casual conversation. Role adherence outside that distribution is explicitly described as emergent, which is the paper being honest about a feature it advertises.
• Gated weights. Fine for a laptop, awkward for a reproducible build.

Who should use it

Use it if latency is the product. Drive-through ordering, phone support, live tutoring, anything where a 1.2-second pause reads as a broken system. Nothing open touches 0.24 seconds under interruption.

Use it if the voice has to be yours. Zero-shot cloning from a sample, with a speaker similarity score no other model in the comparison even competes on.

Use it if you want to fine-tune. The 48 GPU-hour recipe and the fully synthetic data pipeline are both documented, and the code is MIT.

Do not use it if you need strict instruction following. Gemini Live still wins that, and the paper says so. If the agent must never offer a discount it cannot honour, a cascaded stack with a real LLM and guardrails is the safer architecture.

Do not use it for anything but English yet unless you are prepared to evaluate a community fine-tune yourself.

Sources and further reading

• NVIDIA/personaplex on GitHub, MIT licensed code
• nvidia/personaplex-7b-v1 on Hugging Face, gated, NVIDIA Open Model License
• PersonaPlex: Voice and Role Control for Full Duplex Conversational Speech Models, the paper and all figures above
• NVIDIA ADLR demo page
• Full-Duplex-Bench, the benchmark, and Moshi, the base model
• Sesame CSM-1B, the closest attempt at natural conversational speech before this
• MisoTTS and Orpheus TTS, the output half of the stack this replaces
• Qwen3-ASR, the input half

Ten minutes: accept the licence, install it, and run the offline script on assets/test/input_service.wav with the restaurant prompt from the repository's prompting guide. You get a WAV you can listen to without wiring up a microphone or TLS, and it tells you in one pass whether the role conditioning is good enough for your use case. Then swap in your own role text and your own voice sample. The whole point of the hybrid prompt is that neither of those requires training anything.

Not independently tested. Every benchmark figure above is published by NVIDIA in the PersonaPlex paper; we have not reproduced Full-Duplex-Bench. Parameter count, base model, licences, voice list, install steps, download and engagement counts, and the community ports were verified directly against the GitHub repository, the Hugging Face model card metadata and the Hugging Face API on the date below.
Date checked: 2026-10-14