Nemotron 3.5 Bolt E4B

License OpenMDW 1.1

License OpenMDW 1.1

TL;DR

• Post-trained from Google Gemma-4-E4B-it; the two config.json files differ in exactly one field
• Context halved from 128K to 64K, and the licence narrows from Apache 2.0 to OpenMDW 1.1
• Beats Nano 4B everywhere, but loses 3 of 4 multi-turn tool-use benchmarks to Qwen 3.5 4B

☍ Announcement ⬇ Download Model

System Requirements

RAM	About 5 GB VRAM at 32K cached; the Q4_K_M download is 6.18 GB
GPU	RTX 5070 or better for sub-second responses; runs on Jetson Orin Nano 8GB at roughly 6 s to first token
VRAM	3.5 GB to 5.0 GB depending on cache depth, NVIDIA-measured at Q4_K_M batch size 1

✓ Ollama

Table of Contents

1. First, which Bolt?
2. What shipped
3. It is Gemma 4, and the configs say so
4. What NVIDIA actually did
5. The licence went the wrong way
6. Against its predecessor, a landslide
7. It loses at the thing it is sold for
8. The encoders you did not ask for
9. About that 4GB
10. The six-second robot
11. Who should care
12. Running it
13. Sources and further reading

NVIDIA's newest open model is Google's open model. Nemotron 3.5 Bolt E4B is post-trained from Gemma-4-E4B-it, and if you diff the two config files you find exactly one architectural change: the context window has been cut in half. Everything else NVIDIA did here happened in post-training, which is worth saying plainly because the results are genuinely good and the announcement never quite tells you whose architecture you are running.

First, which Bolt?

There are two. Nemotron 3.5 Bolt 74B-A9B shipped on 15 October: a Mamba-Transformer mixture of experts with a 256K context, compressed from Super VL. Nemotron 3.5 Bolt E4B 4GB shipped today and is an 8B dense Gemma derivative, sharing a product name with the 74B and nothing else. The community GGUFs you will find under "Nemotron 3.5 Bolt" are mostly for the big one.

What shipped

Bolt E4B went up on 23 October, marked GA, aimed at local assistants, interactive characters and robots. It is 8B total parameters with 4B effective, a 64K context in and out, 35 supported languages, multi-turn tool calling with guidance for asynchronous calls, and a reasoning mode you can switch off or cap. NVIDIA claims roughly 40% higher average quality than its predecessor Nemotron 3 Nano 4B and up to 80% fewer output tokens when reasoning is on. It is the smallest thing the Nemotron family has shipped.

The "E" means effective. Gemma 4 uses Per-Layer Embeddings: a 262,144-entry table at 256 dimensions for each of 42 layers, which is billions of parameters on disk that only ever get looked up, never multiplied. That is how an 8B file behaves like a 4B model at inference.

It is Gemma 4, and the configs say so

The BF16 checkpoint still declares Gemma4ForConditionalGeneration and model_type: "gemma4". Here is the full diff against the base model.

One field. Gemma 4 E4B gives you 128K of context and Bolt gives you 64K, and that is the entire architectural contribution. If long context is what you need from a small model, the base model is strictly better and it is a free download from Google.

What NVIDIA actually did

Post-training, and quite a lot of it. Bolt was trained with RLVR and RLHF through NeMo-RL and NeMo-Gym, on tool-calling and tool-schema datasets, instruction-following sets including SysBench and InverseIFEval, instruction hierarchy, structured outputs and an updated GenRM set. Every sample was used in both reasoning and non-reasoning modes at a 2:1 ratio per batch, with length penalties raised to curb verbosity.

There is also a DFlash drafter that proposes draft tokens in a single forward pass, which NVIDIA says is up to 21% faster than the MTP drafter Gemma 4 shipped with. The Q4_K_M build was quantized with imatrix calibration on about 1M tokens of the model's own RL rollouts, which is a nicer touch than calibrating on generic text.

The licence went the wrong way

Gemma 4 was the first Gemma released under Apache 2.0, in April, after three generations under Google's own restrictive terms. That was a real moment for open weights.

Bolt E4B ships under OpenMDW 1.1, the same licence we looked at on Super VL four days ago. The model card is explicit that the Gemma 4 base remains Apache 2.0 while use of this model is governed by OpenMDW. It is still an open licence and still fine for commercial use, so nobody is being trapped. But a derivative under a narrower licence than its base is the opposite of the direction we spent all year asking for, and no coverage of this release has mentioned it.

Against its predecessor, a landslide

All figures NVIDIA-reported, BF16, from the research page.

Against Nano 4B this is not a contest. RoleBench goes from 12.0 to 67.0 and LongMemEval from 25.2 to 72.1. Against the Gemma base it wins 14 of 17 published rows.

The exception is the one you would least expect. EQ-Bench v3 went down, 41.1 on the base to 39.4 after post-training, in a model NVIDIA is pitching for interactive characters. Small, but it is the emotional-intelligence benchmark and it moved the wrong way.

It loses at the thing it is sold for

The announcement leads with tool calling. Here is the multi-turn tool-use table in full, with the size peer NVIDIA included.

Qwen 3.5 4B takes three of the four, and Tau2-Bench is not close: 75.1 against 55.1, a twenty-point gap on the benchmark that most resembles a real multi-turn agent session. Bolt's win on When2Call-MT is real and matters, because that benchmark measures knowing when not to call a tool, which is exactly the failure mode that makes small agent models unusable. But "best small tool-calling model" is not what this table shows.

Qwen also wins RULER at 64K (95.1 to 87.0) and LongBench v2 (59.9 to 50.5). Bolt leads everything shown on AgentIF, IFBench, IHEval, InverseIFEval, GaRAGe and RelyToolBench, so the instruction-control story is the strong one. That is the pitch NVIDIA could have led with.

The encoders you did not ask for

Both vision_config and audio_config survive untouched in the Bolt checkpoint: a 16-layer vision tower and a 12-layer audio encoder, inherited from Gemma 4, which is natively multimodal across text, image and audio. NVIDIA's materials describe Bolt purely as a conversational and tool-calling model and say nothing about image or audio input.

So either the multimodal capability is intact and unadvertised, or it is untested dead weight you download anyway. The second reading is the stronger one, because in NVIDIA's own flagship demo the Reachy Mini robot gets its vision from Cosmos 3 Edge running alongside Bolt, not from the vision tower Bolt is already carrying. On a model whose entire pitch is a 4GB footprint, that is a strange thing to ship and not mention.

About that 4GB

"4GB" is in the repository name. The only model file in that repository is a 6.18GB Q4_K_M GGUF, and the BF16 build lives in a separate repo at roughly 16GB. The 4GB refers to runtime VRAM, not a download.

Even then it is a target rather than a promise. From NVIDIA's own measurements, Q4_K_M, batch size 1, llama.cpp with CUDA:

Only one row of the eight NVIDIA published comes in under 4GB. Both Jetson boards sit at 4.9 and 5.0GB once the cache fills.

The six-second robot

The Reachy Mini demo runs on a Jetson Orin Nano 8GB. On that board, NVIDIA's own table reports 5,785 ms to first token cold, and 6,259 ms with 32K cached, at 13 to 15 tokens per second.

That is close to six seconds of silence before your robot starts answering, measured by the vendor on the hardware in the video. The same model on an RTX 5070 answers in 548 ms, which is a tenth of the wait. Credit to NVIDIA for publishing the Jetson numbers at all, because plenty of edge-model launches quietly benchmark on a desktop GPU and let you find out later. But if you are building the robot in the demo, 548 ms is not your number.

Who should care

Use it if you need instruction-following and output control in 5GB on a desktop GPU. IHEval at 84.8 against the base model's 67.0, plus InverseIFEval and StructEval-T, is a real and useful gap, and it is the honest headline for this release.

Use Qwen 3.5 4B if your agent does long multi-turn tool sessions. Tau2-Bench 75.1 against 55.1 is too large to argue with, and Qwen also holds 64K context better.

Use Gemma 4 E4B if you want the 128K context or the multimodal inputs. You are running that model either way; the base just has not had half its context window removed.

Do not plan a conversational robot around the Orin Nano without testing the latency yourself first. Six seconds to first token changes the product.

Running it

It is a GGUF, so llama.cpp and anything built on it will serve it.

hf download nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB --local-dir bolt-e4b
llama-server -m bolt-e4b/Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf -c 8192 -ngl 99

Keep the context modest unless you need it. The VRAM table above is measured at 32K cached, and that is where the footprint crosses 4GB.

Sources and further reading

• NVIDIA Research page for Bolt E4B, with every benchmark and latency table
• The model card, licence terms and training-data tables
• Bolt E4B config.json, source of the architecture diff
• Gemma-4-E4B-it config.json, the base model for comparison
• Gemma-4-E4B-it, Apache 2.0, 128K context

Your under-ten-minutes move: open both config.json files side by side and scroll to max_position_embeddings. One field is the whole architectural story of this release, and seeing it yourself is a better habit than trusting anyone's summary, including ours.

Tested on: not independently tested. All benchmark figures, latency, VRAM and throughput numbers are NVIDIA-reported and were read from the NVIDIA Research page and the Hugging Face model card on the date below. The architecture comparison was computed by diffing the two published config.json files. File sizes were read from the Hugging Face repository listings on the same date.
Date checked: 2026-10-23