What actually shipped

MotionBricks lives in NVIDIA's GR00T-WholeBodyControl repository, in a motionbricks/ subdirectory. The repo sits at roughly 3.2k stars. The paper was accepted to SIGGRAPH 2026 and published in ACM Transactions on Graphics, with the preprint posted to arXiv on 2026-04-27.

The release is dual-licensed. Source code is Apache 2.0. Pretrained weights fall under the NVIDIA Open Model License, which permits commercial use with attribution, subject to NVIDIA's trustworthy-AI terms. The checkpoints ship through Git LFS rather than Hugging Face, and total about 2.2 GB:

• motionbricks_pose, 1.6 GB
• motionbricks_root, 391 MB
• motionbricks_vqvae, 273 MB
• G1-clip.ckpt, 7.5 MB

One thing to set expectations on early: this is a preview. The repo says full integration into NVIDIA's GEAR-SONIC pipeline was targeted for roughly one month after the April release. It is August now, and MotionBricks is still labelled preview. You get the interactive demo and the checkpoints; you do not yet get the fully embedded production pipeline.

The problem: animation graphs do not scale

If you have never shipped a character controller, here is the shape of the problem. Traditional real-time animation runs on a state machine, often called an animation graph. Each node is a clip or a blend space. Each edge is a transition with conditions, blend durations, and priority rules. Walking to running is one edge. Walking to running while injured while carrying a crate while on a slope is a combinatorial mess.

The cost is not the clips. Mocap is cheap now. The cost is the wiring, and the wiring grows faster than the content. Adding a zombie gait to a finished game is not one clip, it is every transition into and out of that gait, plus the QA pass to catch the ones you missed. This is why animation teams grow superlinearly with game scope.

Robotics hit the same wall from the opposite direction. A humanoid like the Unitree G1 does not have 15,000 curated clips; it has a controller that is either hand-tuned per behaviour or trained per task with reinforcement learning. Both approaches are per-skill. Neither gives you one policy that walks, crouches, picks something up, and transitions between all three without a human specifying the transitions.

MotionBricks proposes the same fix for both: stop authoring the graph, and learn the whole space of motions in one generative model that you query at runtime.

How it works: tokenizer, root, pose

The system is three modules, and the total is small enough to be surprising. About 224M parameters covers all 350,000 skills. For a site that mostly writes about 100B-class releases, that number is worth sitting with.

1. The motion tokenizer (23.5M params)

Motion is continuous, but the generative backbone works on discrete tokens, so the first job is converting one into the other. MotionBricks uses a VQ-VAE, a vector-quantized autoencoder: it compresses input into a latent space, snaps each latent vector to the nearest entry in a learned codebook of discrete symbols, then reconstructs from those symbols. The codebook entries become your vocabulary, and motion becomes a sequence you can model like text.

The encoder is a 1D convolutional U-Net with 1024 channels and three residual conv layers per level, downsampling at 2x and 4x so that T input frames become T/4 tokens. It encodes joint positions and rotations but deliberately excludes root information. The decoder mirrors it and reinjects the root trajectory through skip connections at every level.

The quantization is multi-head: instead of one codebook per latent, each embedding is split along the feature dimension and quantized separately against K parallel codebooks. That gives you a combinatorial vocabulary from small codebooks, which is how you get 350,000 skills out of something this size.

2. The root module (50M params)

The root is where the character actually goes in world space. It is a transformer, 512 dimensions, 12 heads, 12 layers, and it runs in two steps. First it predicts timing: given the target, how many frames should this motion take? It outputs a distribution over frame counts at 4-frame resolution. Then it predicts the actual root trajectory over 16 learnable frame-slot embeddings, covering up to 64 frames.

Splitting timing from trajectory matters. It is why a reach can take longer when the target is farther away without anyone authoring a speed curve.

3. The pose module (150M params)

The largest module, and the one that does the animation. A transformer at 1024 dimensions, 16 heads, 16 layers, modelling the distribution over pose tokens conditioned on the root trajectory and any keyframe constraints. Training uses masked token modelling with a cosine-scheduled curriculum: you hide a fraction of the tokens and train the model to fill them back in, starting easy and masking progressively harder. It is BERT-style pretraining applied to motion, and it is what makes the model good at in-betweening, since filling gaps is literally the training objective.

Training cost, for reference: 32 H100s across 4 nodes, 2 million updates, batch size 256 per GPU, Adam at 5e-5 decaying to 2e-6. Roughly 7 days for the tokenizer, 3 for the root module, 7 for the pose module.

Smart primitives: the part you actually program against

The architecture is the interesting half. The interface is the useful half. NVIDIA calls the control layer "smart primitives," and it comes in two flavours.

Smart Locomotion is the navigation interface. You give it velocity, heading, and a style, and it produces the gait. Styles include stylized variants like zombie, injured, skipping, and strafing, and you can switch between them continuously at runtime instead of triggering a transition clip. Under the hood the root trajectory gets refined in stages: a critically damped spring model produces a first pass, a neural refinement pass improves it, and the decoder refines it once more. That layering is why the output does not foot-skate when you yank the stick.

Smart Objects is the interaction interface, and it is the one that eats the most authoring time in a traditional pipeline. You place a proxy keyframe describing intent, for example the hand pose at the moment of grabbing a handle, and the backbone fills in the approach, the contact, and the follow-through. You do not author the reach.

Keyframe enforcement has a dial. Setting the tolerance to zero makes the keyframe a hard constraint, so the hand lands exactly there, which is what you want for a door handle or a ladder rung. Setting it above zero makes it soft guidance, so the model gets close but prioritizes natural motion, which is what you want for a gesture. Objects bind through collision tracing for detection, a socket system for placement, and keyframe anchoring, and each behaviour can use a different number of keyframes.

Benchmarks

The paper evaluates against six baselines on its 350k dataset. The headline table:

Read the first two rows together, because they are the honest story. Conditional in-betweening is faster: 27,000 FPS against 15,000. It also has a 0.8% human win rate, five times the joint jitter, and six times the foot skate. Speed was never the hard part of this problem. Quality at speed was, and that is where the gap sits.

The metrics, briefly: FID measures distribution distance from real motion, lower is better. MMD is another distributional distance. Joint jitter and foot skate are the two artifacts that make generated animation read as fake, high-frequency limb noise and feet sliding on the ground while planted. Win rate is a human preference study.

The same ordering holds on HumanML3D and LaFAN1-G1, which are public benchmarks, so this is not purely a home-field result. Note the FPS figures are measured on an RTX 5090.

Does it scale? The 350k versus 70k question

The paper trains on four datasets, and the pair that matters is the internal 350k corpus and its 70k subset. The full set is 700 hours, 315k training clips, 36 categories, roughly 9,300 unique skills. The 70k subset is 140 hours and 62k training clips, and exists specifically as a scaling benchmark.

The scaling result is the argument for the whole approach: quality improves with corpus size, which means this is a data problem rather than an architecture problem. Given that the architecture is 224M parameters, the implied roadmap is obvious. Do not scale the model, scale the mocap.

Which raises the question of where you get the mocap.

The dataset catch: what "open" means here

This is the part worth reading carefully before you plan around MotionBricks.

The 350k corpus the shipped checkpoints were trained on is proprietary. It is production-grade mocap from real actors, and it was not released. What was released, on 2026-03-16, is BONES-SEED: 142,220 annotated human motions, roughly 288 hours, in SOMA and Unitree G1 formats, with natural-language descriptions, temporal segmentation, and skeletal metadata. It is a genuinely large and well-annotated dataset.

It is also gated. Access on Hugging Face is restricted, and the terms cover academic use and qualifying startups; anything else goes through licensing@bones.studio. So the practical position for a commercial team is this asymmetry:

• You can ship the pretrained weights in a commercial product, under the NVIDIA Open Model License, with attribution.
• You cannot reproduce those weights, because the corpus behind them is not public.
• You cannot straightforwardly retrain on the open substitute either, because BONES-SEED is gated for commercial use, and it is 288 hours against the 700 the model actually saw.

The training scripts underline the point. They ship and they run, but they default to synthetic data. Real training means bringing your own corpus or clearing a license.

That is a meaningfully different posture from NVIDIA's own Nemotron 3 Ultra, which shipped weights, data, and recipes under a permissive license and can be reproduced end to end. MotionBricks is open-weights and open-code, not open-pipeline. Both are legitimate releases. They are not the same thing, and the difference decides whether you can fine-tune on your own studio's motion library or only consume what NVIDIA trained.

Two audiences, one repo

MotionBricks is aimed at game and animation developers on one side and robotics teams on the other, and the two get quite different value today.

Game and animation developers get the strongest immediate case. The state machine problem is real, the smart primitives map cleanly onto how character controllers already work, and 2 ms latency fits inside a frame budget with room to spare. What you do not get is an engine plugin. There is no Unreal or Unity integration in the box, so shipping this means writing the bridge yourself and validating it against your rig. The retargeting caveat below bites here.

Robotics teams get a G1-specific path: the interactive G1 demo, G1 skeleton meshes, and BONES-SEED's G1 MuJoCo trajectories. The gap is that MotionBricks is kinematic, not physics-simulated. It generates poses, not torques, and it has no awareness of whether a motion is dynamically feasible or within the robot's joint limits. It also assumes ground-truth object poses and terrain geometry, which a real robot does not have. You need a perception stack and a physical feasibility layer between MotionBricks and hardware. That work is not small, and it is exactly what the SONIC side of the repo is for.

If you follow the site's physical-AI coverage, the split is familiar. Cosmos 3 predicts what the world does next; MotionBricks decides how a body moves through it. They sit at different layers of the same stack, and neither one closes the perception gap on its own.

Limitations and gotchas

• Preview status. Full GEAR-SONIC integration was targeted for about a month after the April release and has not landed. Plan around the demo and checkpoints, not the production pipeline.
• Kinematic, not physical. The authors state the model can produce physically implausible motion, including self-collisions and motions exceeding hardware constraints. Nothing in the model prevents this.
• No visual planning. It assumes ground-truth object poses and terrain geometry. Real robots need a vision-driven kinematic planner in front of it.
• Retargeting is unsolved. Adapting motion across different body proportions still trades runtime speed against quality. If your character is not human-proportioned, expect work.
• Dataset coverage. The authors call 350,000 motions "small" with limited coverage of rare motion types. Unusual or highly stylized motion is where it will thin out.
• Linux and X11 only, in practice. The demo uses a keyboard key-grab workaround that conflicts on Wayland, macOS, and Windows.
• Crawling modes lack side-only directions. A small but documented hole in the locomotion coverage.
• NVIDIA GPU required. CUDA only. There is no Apple Silicon or AMD path.

Who should use it, and who should wait

Use it now if you are a technical animator or gameplay engineer who wants to prototype what a graph-free character controller feels like, or a robotics researcher already working with a Unitree G1 who wants a motion prior instead of per-skill policies. Also use it if you are writing a paper: the architecture is small, the code is Apache 2.0, and public benchmarks are supported.

Wait if you need a production engine integration, if you need to retrain on your own motion library and cannot clear a BONES-SEED license, or if you need physically guaranteed output for hardware. And skip it entirely if you are not doing character motion. This is a narrow, deep tool, not a general model.

Run the demo in about 10 minutes

You need a CUDA GPU, Python 3.10 or newer, and Git LFS. Budget 2.2 GB for the checkpoints. These commands are transcribed from the NVLabs README and are not independently tested here.

# Git LFS first, or the checkpoints arrive as pointer files
git lfs install

git clone https://github.com/NVlabs/GR00T-WholeBodyControl.git
cd GR00T-WholeBodyControl

# Pull only what the demo needs (~2.2 GB), not the whole repo's LFS objects
git lfs pull --include="motionbricks/out/**" --exclude=""
git lfs pull --include="motionbricks/assets/skeletons/g1/meshes/**" --exclude=""

cd motionbricks
conda create -n motionbricks python=3.10 -y
conda activate motionbricks
pip install -e .
pip install pynput python-xlib   # Linux only, for the keyboard grab

Then launch the interactive G1 demo. WASD moves, the mouse controls the camera, and V/Z/X/B/R/T/C/E/F/G/Q switch motion styles at runtime. Switching styles mid-stride is the thing to try first, because that transition is exactly what you would otherwise hand-author.

DISPLAY=:1 python scripts/interactive_demo_g1.py

If you want to go past the demo, the training pipeline is three scripts run in order. The tokenizer comes first because both transformers consume its codebook:

# 1. Motion tokenizer (VQ-VAE). Everything downstream needs this.
python scripts/train_vqvae.py

# 2. Pose model, conditioned on the tokenizer's codebook
python scripts/train_pose.py

# 3. Root model (timing plus trajectory)
python scripts/train_root.py

# Note: these default to SYNTHETIC data. Real training needs your own
# corpus or a cleared BONES-SEED license.

Short on GPU? The ten-minute version is the project page, which has video of the style transitions and the smart-object interactions. Watch the stylized gait switching and the object approach fill-in, then read section 4 of the paper on the multi-head quantization. That trick, splitting the latent across parallel codebooks, is the reason a 224M-parameter model covers 350,000 skills, and it generalizes well beyond motion.

Sources and further reading

• MotionBricks project page (NVLabs)
• MotionBricks paper on arXiv (2604.24833)
• ACM Transactions on Graphics, SIGGRAPH 2026
• NVlabs/GR00T-WholeBodyControl on GitHub
• BONES-SEED dataset on Hugging Face (gated)

Tested on: not independently tested. MotionBricks needs a CUDA GPU, which is beyond our bench, and our standing policy is not to install AI tooling locally. All figures are as reported by the paper, the repository, and the project page; the 15,000 FPS and 2 ms latency numbers are NVIDIA-measured on an RTX 5090 with no independent reproduction. MotionBricks was still labelled a preview release at the date below.
Date checked: 2026-08-11

Muse Glimmer 30B: a local agent model with a real open license

Muse Glimmer is a 30B dense model. Every parameter activates for every token, so there is no Mixture-of-Experts routing (the technique where only a slice of a much larger network fires per token). Dense costs more compute per token than MoE, but it also means predictable memory use and no expert-routing overhead, which matters when the whole model has to live on one consumer GPU.

The model was not trained from scratch at this size. Meta distilled it from Muse Spark, its larger flagship, using logit distillation: the small model learns to match the big model's full output distribution instead of just its final answers. It also carries a dedicated perception encoder for image input, a context window of 120K+ tokens, and coverage of 100+ languages.

The headline for builders is the license. Apache 2.0 means commercial use, modification, and redistribution with no acceptable-use policy, no monthly-active-user cutoff, and no lawyer required. After years of "open-ish" Llama community licenses, Meta shipping a straight Apache 2.0 model is the actual news here.

How a 30B model fits in under 20 GB

Full-precision Muse Glimmer needs 55+ GB of memory, which rules out every consumer card. Meta's answer is two official quantized variants, K-Quant-Dynamic and K-Quant-17GB, that compress the weights to roughly 4-bit precision and land under 20 GB with what Meta describes as minimal accuracy loss. That fits a 24 GB card like an RTX 3090 or 4090, or unified memory on an Apple Silicon Mac.

Speed comes from DFlash, a small companion drafter model for speculative decoding. The drafter guesses several tokens ahead and the main model verifies the batch in one pass, so you get identical output faster. Meta reports 3.1x faster generation on an RTX 5090, 1.8x on an M5-Max MacBook, and 1.5x on an M4-Max. NVIDIA separately reports 20+ tokens per second per GPU at BF16/NVF4 precision on its Blackwell Ultra hardware.

Benchmarks: wins the agent tests, not everything

Meta compared Muse Glimmer against the two open-weight models in its size class, Gemma 4-31B and Qwen3.6-27B. These are Meta-reported numbers from the launch post; we have not independently tested them, and you should read them the way you read any vendor deck.

The pattern is clear and Meta, to its credit, publishes it. Muse Glimmer wins the agentic benchmarks, tool calling by 13 points over its nearest rival, and agentic search by 3.5. On raw coding it does not: Qwen3.6-27B edges it on SWE-Bench Verified and also leads on OSWorld-Verified and TerminalBench 2.1 in Meta's own table. Meta additionally reports strong results on tau-Bench, its published agent-workflow suite.

So the honest summary: if your workload is an agent loop, browsing, searching, calling MCP tools, recovering from failures, this is now the model to beat at 30B. If your workload is pure code generation, GLM-5.2 class coders and Qwen3.6 still have the edge.

Get it running in under 10 minutes

Launch-day ecosystem support is unusually complete. Official weights, an official GGUF conversion, and an ExecuTorch build are on Hugging Face, Unsloth has its usual quant spread, and there is a first-party Ollama library entry. If you have Ollama v0.32 or newer, this is the whole install:

ollama run muse-glimmer

For llama.cpp or LM Studio, pull a GGUF directly. The Unsloth repo carries the full quant ladder if you want something smaller than Meta's official 4-bit:

# official Meta GGUF
huggingface-cli download meta-models/Muse-Glimmer-30B-GGUF

# or Unsloth's quant spread
huggingface-cli download unsloth/Muse-Glimmer-30B-GGUF

Mac users get MLX support at launch, so LM Studio on an M-series machine works day one; Meta's own test hardware was an M4-Max and M5-Max. If you are serving the model rather than chatting with it, vLLM and SGLang recipes shipped alongside the release, and NVIDIA has a prebuilt NIM container plus NeMo AutoModel support for fine-tuning with SFT, LoRA, and RL. We covered the same local-first release pattern when Kimi K3 GGUFs landed, but at 594 GB that one was a server toy. This one actually fits.

Limitations and gotchas

A few things to know before you rearrange your stack. First, the 20 GB figure is the quantized model; full BF16 needs 55+ GB, so serious fine-tuning still wants workstation or cloud hardware. Second, the DFlash speedups assume you run the drafter alongside the main model, which costs extra memory; budget for that on a 24 GB card. Third, cards below 16 GB are out of luck locally, though Together AI, Fireworks, and OpenRouter all have hosted endpoints live.

And the benchmark caveat bears repeating: every number above is vendor-run. Independent replications will land within days for a release this big. We would treat the agentic wins as directionally real, since the margins are wide, and the coding parity claims as wait-and-see.

Who should use it, and what comes next

Use it now if you are building local agent workflows: MCP tool servers, research agents, browser automation, anything with a loop and a failure mode. A 30B Apache 2.0 model that wins tool-calling benchmarks and runs on a 4090 is exactly the gap the open ecosystem had, sitting between small models like DeepSeek-V4 Flash and the 100B+ giants nobody runs at home.

Skip it if you mainly need a code-completion model, where Qwen3.6 is cheaper to run at 27B and slightly better, or if you need vision output rather than vision input.

What comes next is the bigger story. Meta says an open release of Muse Spark 1.2, the teacher model Glimmer was distilled from, is on the roadmap. Zuckerberg framed the release around distributing capable models widely rather than centralizing them, and whatever you think of the framing, Apache 2.0 weights are the receipts. If Spark ships under the same license, the open-weight landscape shifts again.

Your under-10-minute move: run ollama run muse-glimmer, point it at an MCP server you already use, and see whether that 75.5 tool-calling score survives contact with your own stack.

Sources and further reading

• Meta: Introducing Muse Glimmer
• Muse Glimmer 30B on Hugging Face
• Unsloth GGUF quantizations
• Muse Glimmer on Ollama
• NVIDIA: Muse Glimmer local agent workflows
• Constellation Research: Muse Spark 1.2 on tap

Tested on: not independently tested. All benchmark and performance figures are Meta- or NVIDIA-reported from the launch materials linked above.
Date checked: 2026-08-10

TL;DR

• What it is: the official release of DeepSeek's small V4 variant, a 284B Mixture-of-Experts model with 13B active parameters and a native 1M-token context, re-post-trained for agents and released under MIT on Hugging Face.
• Why it matters: on DeepSeek's own benchmark table, the 0731 build beats V4-Pro-Preview everywhere, and third-party Artificial Analysis puts it one point behind OpenAI's budget GPT-5.6 Luna at roughly 60% lower cost per task.
• The catch: 284B total parameters still means server-class hardware for self-hosting, the preview's high hallucination rate has no published 0731 re-measurement yet, and the API is labeled public beta.

What Actually Landed in DeepSeek-V4-Flash-0731

The official changelog is unusually direct about what changed: nothing in the architecture. "DeepSeek-V4-Flash-0731 keeps the same model architecture and size as DeepSeek-V4-Flash-Preview, and was only re-post-trained." Same hybrid CSA plus HCA attention, same FP4 expert weights, same 1M context. If you want the architecture tour, read the April article; all of it still applies.

Three things are genuinely new:

• A full agentic retraining pass. The gains below came entirely from post-training, which is a quiet argument that open-weight labs still have large headroom without touching pretraining budgets.
• Native Responses API support. The changelog notes the model "natively supports the Responses API format and is specifically adapted for Codex," so it drops into OpenAI-shaped agent harnesses without adapter glue.
• The DSpark checkpoint. The 0731 release ships with a speculative-decoding module attached, which is also why the Hugging Face parameter counter reads 304B instead of 284B: the extra ~20B is the draft module, not the model growing.

Benchmarks: The Small Model Ate the Big One

All numbers below are DeepSeek-reported, from the 0731 model card. We have not run these ourselves.

Read the DeepSWE row twice. The preview scored 7.3. The same architecture, after retraining, scores 54.4. The changelog adds four more agent evals in the same pattern: DSBench-FullStack 68.7, DSBench-Hard 59.6, Toolathlon verified 70.3, and the still-hard tail of Agent Last Exam at 25.2 and Automation Bench (Public) at 25.1. Across all nine published benchmarks the 0731 build lands above V4-Pro-Preview, which makes the current DeepSeek lineup mildly absurd: the cheap model is the good one until V4-Pro gets the same retraining pass.

Third-party numbers point the same direction. The Decoder reports an Artificial Analysis Intelligence Index of 50 versus 51 for GPT-5.6 Luna, "about 60 percent less per task, even after OpenAI's 80 percent price cut," a GDPval jump from 1,189 to 1,559 Elo, and 12% fewer tokens used than the preview. For context, when we covered the preview in April, V4-Pro led the open-weights GDPval board at 1,554. The Flash just walked past that too.

Pricing: Still the Cheapest Seat at the Table

API pricing is unchanged from the preview, per the DeepSeek platform docs:

The cache-hit price is a 98% discount, which matters more than it looks for agents: long-horizon loops re-read the same context constantly, so a well-structured agent pays the cache-hit rate for most of its input tokens. Combine that with the 1M-token window and the KV-cache compression from the V4 architecture, and this stays the cheapest way to run serious long-context agent workloads on someone else's GPUs.

What It Takes to Run It Yourself

Honesty section: 13B active parameters does not mean 13B-sized hardware. The full FP4 plus FP8 checkpoint still wants a single big-memory accelerator or a small tensor-parallel group.

The vLLM single-GPU config from the official recipe is a useful reference for the moving parts (FP8 KV cache, the V4 tokenizer mode, and MTP speculative decoding at 3 draft tokens):

vllm serve deepseek-ai/DeepSeek-V4-Flash \
  --tensor-parallel-size 1 --kv-cache-dtype fp8 \
  --trust-remote-code --gpu-memory-utilization 0.92 \
  --tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 \
  --enable-auto-tool-choice --reasoning-parser deepseek_v4 \
  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Hands-On: Point Your Agent at It in Under 10 Minutes

The API is OpenAI-compatible, so switching an existing agent stack is a one-line change:

from openai import OpenAI

client = OpenAI(
    api_key="YOUR_DEEPSEEK_API_KEY",
    base_url="https://api.deepseek.com/v1",
)

response = client.chat.completions.create(
    model="deepseek-v4-flash",
    messages=[
        {"role": "user", "content": "Read this repo and propose a fix plan."},
    ],
)
print(response.choices[0].message.content)

For self-hosting, grab the weights and follow the vLLM recipe above:

pip install -U "huggingface_hub[cli]"
huggingface-cli download deepseek-ai/DeepSeek-V4-Flash-0731

If you are on the Codex or Responses API side of the fence, the 0731 build is explicitly tuned for that harness, which is the first time DeepSeek has shipped a model adapted for a specific third-party agent frontend.

Limitations and Gotchas

• Hallucination is an open question. The preview measured a 96% hallucination rate on Artificial Analysis's AA-Omniscience eval, among the worst of its peers. Nobody has published a 0731 re-measurement yet. Until someone does, keep retrieval grounding in fact-heavy pipelines.
• No Jinja chat template. Prompt encoding still goes through DeepSeek's Python encoder scripts in the repo. Day-one preview users tripped on this; nothing changed.
• The agent ceiling is real. Beating V4-Pro-Preview is impressive, but Agent Last Exam at 25.2 and Automation Bench at 25.1 are absolute scores. Long-horizon autonomy is still mostly unsolved, for everyone.
• Param-count confusion. Hugging Face shows 304B for the 0731 repo because the DSpark speculative-decoding module ships attached. The model itself is unchanged at 284B.
• Public beta API. DeepSeek labels the 0731 API a public beta, so expect endpoint behavior to shift.

Who Should Use It

If you run coding or tool-use agents against a hosted API and your bill matters, this is now the default open-weight choice: near-Luna scores at 40% of the price, MIT weights as your exit hatch if the API terms ever change. That exit hatch is not theoretical; we made the case for open-weight fallbacks when a US export order switched off a closed frontier model overnight. If you self-host, V4-Flash on a single H200-class card with quantization remains the practical play, same as the preview. And if you were waiting on the V3 line to get a proper successor for agent work, this is it.

What to watch next: a V4-Pro build with the same post-training recipe. If retraining alone took Flash from 7.3 to 54.4 on DeepSWE, the same pass over the 1.6T model is the obvious next shoe to drop.

Sources and Further Reading

• deepseek-ai/DeepSeek-V4-Flash-0731 on Hugging Face (weights, benchmark table)
• DeepSeek-V4-Flash-DSpark (speculative-decoding checkpoint)
• DeepSeek official release announcement (V4-Flash-0731)
• DeepSeek API changelog (official release notes and benchmark list)
• DeepSeek-V4 technical report (arXiv)
• vLLM serving recipe for DeepSeek-V4-Flash
• Artificial Analysis: DeepSeek V4 Flash 0731 analysis
• The Decoder: Flash matches GPT-5.6 Luna at 60% lower cost
• SingularityByte: DeepSeek-V4 Preview breakdown (April launch, full architecture tour)

Tested on: not independently benchmarked. Every score in this article is DeepSeek-reported or third-party (Artificial Analysis, The Decoder) as labeled; we have not run the 0731 build locally.
Date checked: 2026-08-01

Nine days later, two other labs made the same promise. Black Forest Labs announced FLUX 3 on July 23 with an open-weight backbone to follow. MiniMax announced H3 on July 31 and said the weights ship "in the coming days." Neither has published a file.

That sounds like more of the same, and in one sense it is. But the arithmetic underneath changed, and it is worth looking at directly, because the distance between open and closed video is now small enough to be measured in a single number.

TL;DR

• The open-weights video gap is 262 Elo. The best downloadable text-to-video model with audio is LTX-2.3 Fast at 980 on the Artificial Analysis arena. MiniMax H3, announced last week, sits at 1242.
• Three labs are holding the trigger. FLUX 3 Dev, MiniMax H3, and Wan 3.0 are all announced as open-weight. As of today, none of the three exists on Hugging Face.
• One weight drop closes almost the entire gap. If MiniMax ships what it announced, open-weight video goes from "good enough to iterate with" to second place in the world overnight.
• The catch is the license, not the capability. Nobody at the frontier is shipping Apache 2.0 anymore. Revenue-capped community licenses are the new default.

The 262-Elo gap in open-weight video models, measured

Artificial Analysis runs a blind arena for video: two clips from the same prompt, users vote, Elo comes out the other end. It has a filter for open weights, which makes it the cleanest available read on how far behind downloadable models are.

Comparing like for like on the with-audio board, because H3 and FLUX 3 both generate native audio:

Two things stand out. The top ten is entirely closed or unshipped. And the entire open-weights leaderboard, every position of it, is Lightricks. There is no second open lab in the running right now.

1242 minus 980 is 262. That is the number this article is about. It is the amount of quality that would transfer from the closed column to the open column if exactly one company followed through on exactly one announcement.

For scale: 262 Elo in a preference arena is not a rounding error and not an unbridgeable chasm. It is roughly the distance between "usable for previz and social clips" and "usable for the shot you actually deliver."

Three promises, zero files

Part four published a one-command check for whether a model is genuinely released. Here it is pointed at all three claims at once, run on the date below:

for q in Wan3 Wan2.7 MiniMax-H3 FLUX.3; do
  echo -n "$q -> "
  curl -s "https://huggingface.co/api/models?search=$q&limit=5" \
    | python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d), [m['modelId'] for m in d])"
done

Every one of them comes back empty, or with unrelated community repos that happen to match the string. Checking what the three labs have actually published, newest first:

That last row is the contrast worth sitting with. While three labs were announcing open weights, Lightricks pushed 17 repository updates in July alone, including eight new IC-LoRAs in the final four days of the month: dubbing, relighting, in/outpainting, spatial upscaling, deblur, decompression, water simulation, and one that shaves beards. Announcements are cheap. Adapters are not.

What each drop would actually change

MiniMax H3: the one that moves the number

H3 is a multimodal video model that takes text, images, video, and audio in one context and returns up to 15 seconds at 2K with native stereo sound. Artificial Analysis ranks it first in the world for video editing and second for text-to-video with audio.

MiniMax has committed to the MiniMax Community License: free for non-commercial use, free for commercial use by organizations under 20 million dollars in annual revenue, with attribution required. As of today there is no H3 repository on the MiniMaxAI org.

If it ships, this is the whole story. Everything else in this article is a footnote to it.

FLUX 3: the one that changes the shape

FLUX 3 is Black Forest Labs' first video model, and it is not only a video model. It is jointly trained on image, video, audio, and action prediction in one architecture, which is why it also powers FLUX-mimic for robotics. Video output runs up to 20 seconds with native audio.

The open-weight piece is called FLUX 3 Dev, described in BFL's own announcement as "open-weight access to a multimodal backbone, for content creation (video, audio and image) and action prediction." What the announcement does not contain is a date or a license name.

Worth being precise about what this would mean if it lands. It would be the first downloadable model that generates video, audio, images, and robot actions from one set of weights. That is a different kind of release from H3, and for anyone building agents that need to both see and act, it is arguably the more interesting one. It is also the vaguest of the three promises.

Wan 3.0: the one to discount

Wan 3.0 is pre-announced as an Apache 2.0 open-weight release for mid-2026. Treat that with more caution than the other two, for reasons that are entirely evidence-based.

We could not find a primary Alibaba source for Wan 3.0 at all. The specifications circulating contradict each other, with the same "confirmed" model described as 60B dense in one place and a 27B mixture-of-experts with 14B active in another. Those pages trace back to single-purpose domains built around the model name, the same pattern part four documented for Wan 2.7.

The track record does not help either. Wan 2.5 was pre-announced as open and never appeared. Wan 2.6 shipped closed. Wan 2.7 is on the arena at 1163 and remains API only. Official Wan open weights still stop at 2.2, where they have been since August 2025.

None of that makes Wan 3.0 fake. It makes it unverifiable, which for planning purposes is the same thing.

The licenses are the real story

The reflex reading of an open-weights wave is that the field is getting more permissive. It is getting more available, which is not the same thing.

The only genuinely permissive entry is the oldest one. Every model that has landed or been announced since is a revenue-capped community license, and the caps come with details worth reading before you assume you are under them. The LTX threshold counts subsidiaries, affiliates, and companies under common control collectively, so a small studio inside a large group may already be over it.

For most people reading this, a 10 or 20 million dollar cap is functionally no cap. That is exactly why it works as a strategy, and exactly why it is worth naming: the industry has found a license that feels open to almost everyone who downloads it while staying commercially closed to everyone who matters to revenue.

The lawsuit nobody is pricing in

One complication sits underneath the MiniMax promise specifically.

Disney, Marvel, Lucasfilm, Twentieth Century Fox, Universal, DreamWorks Animation, and Warner Bros. Discovery sued MiniMax on 2025-09-16 in the Central District of California, alleging Hailuo was trained on unauthorized copies of their work and reproduces recognizable characters on simple prompts. On 2026-05-26, Judge Stanley Blumenfeld denied MiniMax's motion to dismiss, finding the studios had plausibly alleged widespread intentional infringement. The case is live.

So the scenario on the table is a lab open-sourcing the weights of a video model whose predecessor is under active infringement litigation from most of Hollywood. We are not aware of a precedent for that, and the honest answer about downstream exposure for people who fine-tune and ship products on those weights is that nobody knows yet.

Factor it into planning, not into panic. It is a reason to read the license text when it lands rather than a reason to skip the download.

What you can actually run today

One model in this entire article is downloadable, and it is the same one as last time. LTX-2.3: 22 billion parameters, synchronized audio and video from a single model, 2.13 million downloads on the base repository.

The shortest path from nothing to a clip:

pip install -U diffusers transformers accelerate

python3 - <<'PY'
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video

pipe = DiffusionPipeline.from_pretrained(
    "Lightricks/LTX-2.3", torch_dtype=torch.bfloat16, device_map="cuda"
)
image = load_image("your-still.png")
out = pipe(image=image, prompt="A man with short gray hair plays a red electric guitar.").frames[0]
export_to_video(out, "output.mp4")
PY

Budget 24GB of VRAM with the int8 build for 1080p work. Community reports put a 10-second 1080p clip at roughly 4 to 6 minutes on an RTX 4090. Two constraints will bite you on the first run: width and height must be divisible by 32, and frame count must be divisible by 8 plus 1. Ask for 100 frames and you get an error rather than a video.

If you are on Apple Silicon, our LTX-2 on Mac walkthrough still applies, and the hardware stack guide covers where a 22B model sits in memory terms.

What to watch in the next 90 days

Three checks, in descending order of how much they matter:

1. Does MiniMaxAI/MiniMax-H3 appear on Hugging Face? "Coming days" was July 31. If it is still empty at the end of August, the promise has entered Wan territory.
2. Does FLUX 3 Dev get a date and a license name? The capability claim is the most interesting of the three. The commitment behind it is currently the thinnest.
3. Does Lightricks stay alone at the top? If both promises slip, the open-weights leaderboard remains a single-vendor board for another quarter, and the 262 stays where it is.

The optimistic case here is real, and it is narrower than the coverage suggests. Open-weight video does not need three labs to follow through. It needs one. The gap has never been this close to closing on a single upload.

Set a reminder for two weeks out and run the check yourself. It takes ten seconds:

curl -s "https://huggingface.co/api/models?author=MiniMaxAI&sort=lastModified&direction=-1&limit=5" \
  | python3 -c "import sys,json; [print(m['modelId'], str(m['lastModified'])[:10]) for m in json.load(sys.stdin)]"

Sources and further reading

• Part four: what actually shipped in mid-2026, and what did not
• Part one: our tested open-source video model comparison
• Artificial Analysis open-weights text-to-video leaderboard
• Artificial Analysis text-to-video leaderboard (all models)
• Black Forest Labs: FLUX 3 announcement
• SCMP: MiniMax challenges ByteDance with open weights for H3
• Disney Enterprises, Inc. v. MiniMax, 2:25-cv-08768 (C.D. Cal.)
• Lightricks/LTX-2.3 on Hugging Face
• The LTX-2 Community License Agreement

Tested on: not independently tested. We did not run LTX-2.3, H3, or FLUX 3. Every release claim above was verified directly against the public Hugging Face API on the date below, using the commands published in this article: model existence, organization publishing history, download counts, and last-modified dates. Elo figures are read from the Artificial Analysis video arena on the same date and are user-vote preference scores, not capability benchmarks. Capability descriptions, license terms, and release intentions are the projects' own. VRAM and generation-time figures for LTX-2.3 are community-reported.
Date checked: 2026-08-01

If you build or run MCP servers, both stories are about you. The protocol you target changed shape this week, and the failure mode the new auth rules exist to prevent just got a name, a nickname, and a patch you should already be running.

The MCP 2026-07-28 spec: what shipped

• The stateless core removes the initialize handshake and the Mcp-Session-Id header. Every request describes itself, so any server instance can answer it (SEP-2575, SEP-2567).
• Authorization gets hardened: RFC 9207 issuer validation becomes mandatory, client credentials are bound to the server that issued them, and Dynamic Client Registration is deprecated in favor of Client ID Metadata Documents.
• Deprecations now run on a formal clock: a feature keeps working at least 12 months after deprecation, shortened to 90 days only for security emergencies.

The spec went final on July 28 under the Agentic AI Foundation, the Linux Foundation directed fund that now stewards the protocol Anthropic open-sourced in late 2024. The release candidate was locked on May 21 and then sat through ten weeks of validation by SDK maintainers and client implementers before publication. Lead maintainer David Soria Parra is not underselling it: "The new release is MCP's most important since remote MCP first launched over a year ago."

The scale explains the caution. The Tier 1 SDKs (TypeScript, Python, Go and C#) pull close to half a billion downloads a month, with TypeScript and Python each past one billion total. All four speak 2026-07-28 as of publication day with migration notes included, Rust support is in beta, and FastMCP 4.0 ships stateless out of the box.

Stateless core: the handshake is gone

Until now, every MCP connection opened with an initialize/initialized exchange that negotiated versions and capabilities, and remote transports pinned the result to an Mcp-Session-Id header. That made every server stateful by default. Load balancers needed sticky sessions, a restart dropped your clients, and scaling horizontally meant shared session storage.

2026-07-28 deletes the ceremony. The handshake and the session header are gone, and each request carries its protocol version, client identity and capabilities in the _meta field, so the server learns everything it needs from the request itself:

{
  "jsonrpc": "2.0",
  "id": 1,
  "method": "tools/call",
  "params": {
    "_meta": {
      "io.modelcontextprotocol/protocolVersion": "2026-07-28",
      "io.modelcontextprotocol/clientInfo": { "name": "ExampleClient", "version": "1.0.0" },
      "io.modelcontextprotocol/clientCapabilities": { "elicitation": {} }
    },
    "name": "confirmThenGreet",
    "arguments": {}
  }
}

That wire format is lifted straight from the Go SDK's 2026-07-28 conformance suite, not paraphrased. A version mismatch returns UnsupportedProtocolVersionError, and clients that want to probe first can call the new server/discover RPC, which advertises supported versions and capabilities. The payoff: any instance can answer any request, so a plain round-robin load balancer is enough. The servers you already wire into Claude Code get simpler to deploy, not harder to use.

Multi round-trip requests replace server-initiated calls

MCP servers could previously call the client mid-request: elicitation/create to ask the user something, sampling/createMessage to request a completion, roots/list to inspect the workspace. Server-initiated calls are the natural enemy of statelessness, so SEP-2322 inverts the flow. A server that needs input finishes its response instead, returning resultType "input_required" plus a machine-readable list of what it needs:

{
  "resultType": "input_required",
  "requestState": "step=1",
  "inputRequests": {
    "who": {
      "method": "elicitation/create",
      "params": {
        "mode": "form",
        "message": "What is your name?",
        "requestedSchema": {
          "type": "object",
          "properties": { "name": { "type": "string" } }
        }
      }
    }
  }
}

The client gathers answers and retries the original call, attaching "inputResponses": {"who": {"action": "accept", "content": {"name": "MCP Go"}}} and echoing the opaque requestState token back. The server then completes with resultType "complete". No channel held open to the client, no state parked on the server between the two calls, and the SDKs run the retry loop for you by default.

Routing headers and cacheable lists

Two smaller changes matter to anyone running MCP behind real infrastructure. SEP-2243 requires clients to send Mcp-Method and Mcp-Name HTTP headers with each call, so gateways, WAFs and metering proxies can route on the operation without parsing JSON bodies. Your rate limiter can treat a shell-adjacent tools/call differently from a harmless resources/read without deep packet inspection.

SEP-2549 makes list results cacheable: tools/list, prompts/list, resources/list and resources/read responses now carry ttlMs and cacheScope fields. Caching a tool catalog used to be pointless because lists could vary per connection. Stateless servers return the same lists to everyone, so a CDN can finally hold them for the advertised lifetime.

Auth hardening: issuer checks, bound credentials, CIMD

The authorization changes read like a direct response to a year of agent-security incidents. Clients must validate the iss parameter of an authorization response before redeeming the code (RFC 9207, SEP-2468). That blocks mix-up attacks, where a malicious server maneuvers a client into spending its credentials at the wrong authorization server. Client credentials are also bound to the server that issued them (SEP-2352): if a resource migrates, the client re-registers instead of reusing credentials across servers.

Token audience is enforced at protocol level through RFC 8707 resource indicators: a token minted for server A is dead on arrival at server B, closing the confused-deputy replay that security write-ups have warned about since 2025. Operators must publish RFC 9728 metadata at /.well-known/oauth-protected-resource. Dynamic Client Registration is formally deprecated in favor of Client ID Metadata Documents (CIMD), with backward compatibility for authorization servers that have not caught up, and the new application_type parameter finally makes localhost redirects behave for desktop and CLI apps (SEP-837).

All of this assumes one thing: that your bridge has authentication to harden. Last week showed how low the ecosystem's floor still sits.

RufRoot: the CVSS 10.0 argument for all of this

Ruflo is the multi-agent orchestration platform that started life as Claude Flow, with more than 66,500 GitHub stars. Its MCP bridge, the Express.js server that fronts all tool invocations, shipped a default docker-compose.yml that bound port 3001 to 0.0.0.0 and answered POST /mcp with no authentication of any kind. Noma Labs researcher Eli Ainhorn found that one unauthenticated HTTP POST reached 233 tools, including ruflo__terminal_execute.

From that single request an attacker could run shell commands inside the container, read every LLM provider key from the environment, dump stored conversations from the bundled MongoDB, drop a payload into /app for persistence, and poison the AgentDB pattern store so the platform's agents learn attacker-chosen behavior. Spawning agent swarms billed to the victim's API keys was the closing insult. The public write-ups landed this week.

Credit where due: maintainer Reuven Cohen shipped 3.16.3 within 24 hours of the June 30 report. The lesson is not that one project was uniquely careless. The lesson is that "an MCP bridge on 0.0.0.0 with no auth" was a shippable default in 2026 at all. A spec cannot patch a door that ships open, but audience-bound tokens and mandatory issuer checks are the same class of failure finally taken seriously at protocol level.

Extensions, deprecations, and the 12-month clock

Tasks leave the experimental core and become the io.modelcontextprotocol/tasks extension. The blocking tasks/result call is removed, long-running work is polled through tasks/get, and notification streams collapse into a single subscriptions/listen per type (SEP-2663). Roots, sampling and logging are deprecated (SEP-2577), and the legacy HTTP+SSE transport starts a one-year goodbye in favor of stateless Streamable HTTP.

Deprecated does not mean dead. The new feature lifecycle guarantees at least twelve months between deprecation and the earliest possible removal, with a 90-day exception reserved for security emergencies. Nothing switches off on you today: servers speaking 2025-11-25 keep working, and nobody is forced to upgrade this quarter. The clock is real, though, and it started on Monday.

What this means for your server

We run MCP servers in production ourselves, so here is the migration map we are actually working from, not a rewording of the changelog.

The session removal is the one that changes your code. If your server stashes per-connection state behind Mcp-Session-Id, that pattern is dead: mint explicit handles and pass them as ordinary tool arguments, the way REST APIs have passed resource IDs forever. It is more honest anyway, since hidden session state is exactly how list results ended up varying per connection. Read _meta on every request, implement server/discover, and delete your handshake path with prejudice.

Your 10-minute audit

RufRoot's root cause was not exotic. It was a listener on the wrong interface with nothing in front of it. Check your own machines tonight:

# Anything bound to 0.0.0.0 on these ports answers the internet, not just you
ss -tlnp | grep -E ':(3001|27017)'

# Probe your own bridge: anything but 401, 403 or connection refused
# means it talks to strangers
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:3001/mcp \
  -H 'Content-Type: application/json' -d '{}'

If you run Ruflo, upgrade to 3.16.3, close ports 3001 and 27017 at the firewall, rotate every LLM key the container could read, and audit the AgentDB store for patterns you did not put there. If you exposed an n8n workflow as an MCP tool, ask the same question: who can reach that endpoint, and what tells them no? Run the two commands above before you close this tab. That is the whole ten minutes.

Sources and further reading

• The 2026-07-28 Specification, official announcement
• MCP specification 2026-07-28, full text
• modelcontextprotocol releases on GitHub
• Noma Security: RufRoot advisory (CVE-2026-59726)
• The Hacker News: Ruflo MCP flaw report
• WorkOS: what the 2026 spec changes for agent authentication
• SingularityByte: turn an n8n workflow into an MCP tool

Tested on: not independently tested. No 2026-07-28 SDK ran on our bench long enough for a compliance pass; spec details are read from the official announcement and spec text, the JSON wire formats are copied verbatim from the Go SDK's conformance suite, and RufRoot details come from the Noma Labs advisory and The Hacker News reporting, not reproduced by us.
Date checked: 2026-07-31

Run Kimi K3 locally: what shipped

• unsloth/Kimi-K3-GGUF holds six Dynamic quants of the 2.8T-parameter MoE, from a 594 GB 1-bit (UD-IQ1_S) to a near-lossless 1,560 GB Q8, shipped as split GGUF files that llama.cpp loads directly.
• Vision works. You build Unsloth's llama.cpp branch kimi-k3-fullsize-vision and pass the bundled mmproj-BF16.gguf projector file alongside the model.
• Moonshot's recommended settings carry over: temperature 1.0 with top_p 0.95, or top_p 1.0 for agent work. Thinking is always on, with low, high and max effort levels.

The hardware bill

Unsloth's rule of thumb: combined RAM plus VRAM roughly equal to the quant size. Less still works through disk offloading, just much slower. Here is the honest table.

So "locally" means an NVIDIA DGX Station, a fat EPYC box, or, per Unsloth, "a Mac Studio connected to a 128GB RAM device". That last one adds up: a 512 GB M3 Ultra plus one 128 GB machine totals 640 GB, which clears the 1-bit line. On B200-class hardware that fits the whole quant, Unsloth reports about 20 tokens per second generation and over 120 tokens per second throughput. Your 4090 is not in this story, and we respect you too much to pretend otherwise.

Run it today

Build the branch, then pull the 1-bit quant and the vision projector. The 1-bit alone is 14 split files, so budget the disk and the weekend.

git clone https://github.com/unslothai/llama.cpp
cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision
cd ..
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first

pip install -U "huggingface_hub[cli]"
hf download unsloth/Kimi-K3-GGUF --local-dir unsloth/Kimi-K3-GGUF \
    --include "*mmproj-BF16*" --include "*UD-IQ1_S*"

Then point llama-cli at the first shard and the projector:

./llama.cpp/build/bin/llama-cli \
    --model unsloth/Kimi-K3-GGUF/UD-IQ1_S/Kimi-K3-UD-IQ1_S-00001-of-00014.gguf \
    --mmproj unsloth/Kimi-K3-GGUF/mmproj-BF16.gguf \
    --temp 1.0 --top-p 0.95

One gotcha for agent builders, straight from the model card: K3 always thinks and returns reasoning_content, and multi-turn calls must feed the complete assistant message back into history, reasoning and tool calls included, not just the final text.

Why this matters, and the caveat

Three days ago, self-hosting K3 meant roughly 19 H100s just to load the raw MXFP4 weights. A 610 GB RAM+VRAM target moves that to one serious workstation, and K3's linear-attention design keeps the KV cache tame if you push the 1M context. The caveat: a Dynamic 1-bit is not uniformly 1-bit. Unsloth, the two-person outfit we profiled, mixes bit-widths per tensor, and nobody has published quality numbers for these exact files yet. The Hugging Face counter already shows 12,178 downloads, which is a lot of terabytes pulled on faith.

Your under-10-minutes move: check whether any machine you own clears the line before you burn 594 GB of disk finding out.

# Your RAM + VRAM in GB, against the 610 GB the 1-bit wants.
echo "$(( $(free -g | awk '/Mem:/ {print $2}') + $(nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits 2>/dev/null | awk '{s+=$1} END {print int(s/1024)}') )) GB available"

If the number is a few hundred GB short, the hosted API remains the sane path, and Kimi K2.7 Code still covers the single-node crowd.

Sources and further reading

• Unsloth: Kimi K3 local deployment guide (sizes, settings, commands)
• unsloth/Kimi-K3-GGUF on Hugging Face (all six quants + mmproj)
• unslothai/llama.cpp, branch kimi-k3-fullsize-vision
• moonshotai/Kimi-K3 (original weights) and the Kimi K3 License, which applies to these quants too
• SingularityByte: what actually shipped in the K3 weights drop
• SingularityByte: Kimi K3 model breakdown

Tested on: not independently tested. The smallest Kimi K3 quant needs about 610 GB of combined RAM and VRAM, which is beyond our bench. Quant sizes, hardware guidance and throughput figures are Unsloth-reported; sampling settings and the multi-turn rule are from the model card; file sizes and download counts were read from the Hugging Face repo.
Date checked: 2026-07-30

TL;DR

• What it is: a 2.8-trillion-parameter Mixture-of-Experts model with 104B active parameters, 93 layers, a 1,048,576-token context window, and native text, image and video input. Quantization-aware trained in MXFP4.
• Why it matters: the first open 3T-class model, and on Moonshot's own numbers it trades the lead with Claude Fable 5 and GPT-5.6 Sol across coding and agentic suites. You can audit it, fine-tune it, and serve it yourself.
• The catch: 1,561 GB of weights. That is roughly 19 H100s just to hold the model, before you allocate a single token of KV cache. And the license is not Apache 2.0.

What actually landed

One repository, not the family of base, instruct and quantized variants that usually accompanies a release. Everything is in moonshotai/Kimi-K3, and the weights are already quantized: there is no separate FP16 checkpoint to download.

The license is custom, and clause 2 has teeth

It is not Apache 2.0, despite that being reported widely in the days before the drop. It is a bespoke "Kimi K3 License", MIT-shaped at the core with two commercial conditions bolted on. Read it yourself, it is short, but here is the shape of it.

Clause 2 targets Model-as-a-Service specifically. If you give third parties access to inference or fine-tuning in a way that lets them control inputs, parameters or training data, and you or your affiliates clear 20 million US dollars in aggregate revenue over any consecutive 12 months, you must sign a separate agreement with Moonshot before any commercial use. Note the trigger is total company revenue, not revenue from the model.

Clause 3 is an attribution rule. Ship a product with more than 100 million monthly active users or more than 20 million US dollars in monthly revenue, and "Kimi K3" has to appear prominently in your user interface.

Clause 4 is the escape hatch most readers will land in: neither condition applies to internal use, defined as use that does not expose the model, its outputs, or its capabilities to third parties. Run it inside your own company and you are clear. Resell inference at scale and you are negotiating.

The numbers everyone published before the file listing existed

This is worth your attention as a builder, because it is now a pattern. In the week before the drop, a wave of posts appeared with hardware tables, vLLM commands, and download instructions for a file that did not exist yet. We checked the Hugging Face API repeatedly across that window: the moonshotai org held 18 models, newest Kimi K2.7 Code from June 15, and the K3 page was a countdown placeholder with a notify button.

Now that the real repo is up, we can score the claims.

None of these were malicious. They were written ahead of a traffic event and nobody checked, because there was nothing to check against. The practical lesson for anyone about to pull a terabyte and a half: ask the API, not a blog. It takes one command. We went through the same exercise when Wan 2.7 was widely reported as open-weights and was not, and the method there works for any release claim.

# What has a lab actually published? No repo, no download,
# regardless of what any article says.
curl -s "https://huggingface.co/api/models?author=moonshotai" \
  | python3 -c "import sys,json;[print(m['modelId'], m.get('lastModified')) for m in json.load(sys.stdin)]"

# Real file list, real byte total, real license field:
curl -s "https://huggingface.co/api/models/moonshotai/Kimi-K3?blobs=true" \
  | python3 -c "import sys,json;d=json.load(sys.stdin);s=d.get('siblings',[]);\
print('license:', d.get('cardData',{}).get('license_name'));\
print('files:', len(s));\
print('total GB:', round(sum(f.get('size') or 0 for f in s)/1e9, 1))"

The architecture, and why 1M context is affordable here

K3 runs 93 layers, and the attention is split: 69 layers use Kimi Delta Attention (KDA), Moonshot's linear attention variant, and only 24 use Gated MLA, the full-attention path. That ratio is the interesting part. Linear attention carries a fixed-size recurrent state instead of a KV cache that grows with sequence length, so at a 1M-token context only those 24 layers accumulate a conventional cache. That is roughly a quarter of the model paying full freight for long context, which is how a 1M window becomes tractable at all.

The MoE side is unusually sparse: 896 routed experts with 16 selected per token, plus 2 shared experts that run every time, routed by a sigmoid gate. Moonshot claims about 2.5x better overall scaling efficiency than K2 from this combination. Activated parameters land at 104B, so you store 2.8T and compute roughly 104B per token.

The tech report published alongside the weights adds a detail worth flagging: K3 uses no explicit positional embedding at all. There is no RoPE, and therefore no RoPE rescaling or interpolation to reach long context. Position is encoded implicitly through KDA's recurrent gating and decay, which is why the model extrapolates to 1M tokens without the usual positional surgery. Training grew the window in four stages, 8K to 64K during pre-training and 256K to 1M during cooldown, which keeps the expensive long-sequence compute inside a small slice of the budget.

Here is the part almost nobody covering the drop mentioned: both headline architecture pieces have been open on GitHub for months. FlashKDA, the KDA kernels, went up April 20, 2026 under MIT and sits at 473 stars. Attention-Residuals has been public since March 15, 2026, now past 3,390 stars. You could read and benchmark K3's attention design roughly four months before you could download K3. If you build your own long-context stack, those repos are worth more to you than the checkpoint.

What MXFP4 actually covers

Moonshot applied quantization-aware training from the supervised fine-tuning stage onward, using MXFP4 weights with MXFP8 activations. This is not a post-training squeeze, so the usual "the quant broke it" caveat does not apply the same way.

But the 4-bit format does not cover the whole model. The config's ignore list keeps self-attention, the shared experts, the dense MLP projections, the LM head, the vision tower and the multimodal projector out of MXFP4. Only the routed expert Linear layers are packed to 4 bits, at group size 32. That is exactly why 2.8T parameters still weigh 1,561 GB rather than the ~1.4 TB a naive 4-bit calculation gives you.

What it takes to run

Blunt version: this is a multi-node deployment or nothing. The weights alone need 1,453.8 GiB resident before you allocate KV cache, activations, or any headroom for fragmentation.

So a single 8x H100 node does not load this model, and the widely repeated "8x H100 minimum" figure is short by more than a factor of two. On B200s you can technically fit the weights across eight cards, but you are leaving almost nothing for a 1M-token KV cache, so plan for two nodes.

Moonshot lists three supported inference engines in the model card: vLLM, SGLang, and TokenSpeed, each with a published K3 recipe. Day one there are no community GGUF, AWQ or Unsloth conversions, which is unsurprising: requantizing a QAT MXFP4 MoE of this size is not a weekend job, and anything that appears in the next few days deserves scrutiny before you trust it.

For scale, here is how much the previous open Kimi models actually get pulled, straight from the Hugging Face API. These are 30-day counts, and most of them are quantization pipelines, mirrors and eval harnesses rather than production deployments, which is worth remembering whenever a release gets called widely deployed. K3 is too fresh to compare: its counter had not meaningfully moved at the time of writing, and Hugging Face updates these on a lag.

Benchmarks, with the usual asterisk

Every number below comes from Moonshot's own model card, run on their harness. We have not reproduced any of it, and K3 is far beyond what we can bench locally. Treat this as a vendor claim until third parties replicate it. The pattern across all 46 reported benchmarks is that K3 and Fable 5 trade wins, with GPT-5.6 Sol close behind.

Credit where it is due: Moonshot says so itself. The tech report's abstract states that K3's overall performance "still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol", while beating every other model in their suite. A lab publishing that sentence in its own launch paper is worth more than any leaderboard screenshot, and it is a useful corrective to the week's louder claims that K3 simply beats everything.

K3 leads on SWE-Marathon by 7 points, on MCPMark-Verified by 7.1, and on BrowseComp by 3.2. It trails on FrontierSWE, GDPval and AA-Briefcase. Long-horizon agentic work is where it looks strongest, which lines up with what it was built for. For the launch-week pricing and arena context, see our Kimi K3 model breakdown.

What it actually built

The most concrete evidence in the report is not a benchmark, it is two artifacts you can go read. Both are Apache 2.0 on GitHub, published July 23.

nano-kpu is an inference-chip prototype that K3 designed, optimized and verified in a single 48-hour autonomous run driving open-source EDA tools. Inside a 4 mm2 area budget it closes timing at 100 MHz and simulates decode throughput above 8,700 tokens per second, with 1.46M standard cells and an INT4 MAC array. MiniTriton is a compact Triton-like GPU compiler K3 wrote end to end, from a tile-level Python frontend through MLIR passes to PTX code generation, with autograd and distributed primitives on top.

On kernel optimization, Moonshot reports K3 cutting AttnRes latency from 283.6 ms to 114.4 ms and reducing KDA runtime by 73.6 percent, and says an early K3 checkpoint was already doing most of their kernel work during late development. These are still first-party claims, but unlike a benchmark score the output is sitting in a public repo where you can judge it yourself.

Limitations and gotchas

• Read the LICENSE before you build a business on it. The 20 million dollar Model-as-a-Service threshold is aggregate company revenue, not model revenue, and it is easy to misread as the latter.
• Benchmarks are entirely first-party at this point, produced on Moonshot's own agent harness. Nothing has been independently replicated yet.
• Size is the wall, not a temporary inconvenience. There is no path where this runs on one GPU, and MXFP4 is already applied, so there is no easy 2x saving left on the table.
• No community quants on day one. Early conversions of a QAT MXFP4 MoE this large are likely to be broken. Check the repo discussions before trusting one.
• The vision tower and multimodal projector are unquantized, so multimodal serving carries more memory than the headline 4-bit number suggests.
• Using the hosted API instead means sending your code to a China-hosted endpoint. Self-hosting is the entire point of this release for anyone with a data residency constraint.

Who should use it

Pull the weights if you already run multi-node inference, you need frontier-class agentic coding you fully control, and license terms or data residency matter more than convenience. Pull them too if you research architectures, because a 69/24 linear-to-full attention split at 2.8T scale is the most interesting thing in this release regardless of the benchmark table.

Skip it if you have one GPU, or one node. That is arithmetic, not a judgment. For open coding models you can actually run this week, Kimi K2.7 Code and GLM-5.2 remain the sane picks, and our June 2026 open-weight roundup covers the smaller end of the field.

Get started in under 10 minutes

If you have the hardware, start the download and read the model card while it runs. Check the license and the layer split first, both are small files.

pip install -U "huggingface_hub[cli]"

# Read the license and architecture before committing 1.5 TB of disk.
hf download moonshotai/Kimi-K3 LICENSE config.json --local-dir ./kimi-k3
python3 -c "import json;c=json.load(open('./kimi-k3/config.json'))['text_config'];\
la=c['linear_attn_config'];\
print('layers:', c['num_hidden_layers']);\
print('KDA:', len(la['kda_layers']), 'full attention:', len(la['full_attn_layers']));\
print('experts:', c['num_experts'], 'active:', c['num_experts_per_token'])"

# Then the weights. Budget the disk and the time.
hf download moonshotai/Kimi-K3 --local-dir ./kimi-k3

No cluster? The API is OpenAI-compatible, so you can point an agent you already run at K3 by changing two lines.

# Any OpenAI-style SDK works; change base_url and model, keep the rest.
from openai import OpenAI

client = OpenAI(
    api_key="sk-...",                      # key from platform.kimi.ai
    base_url="https://api.moonshot.ai/v1",
)

resp = client.chat.completions.create(
    model="kimi-k3",
    messages=[{"role": "user",
               "content": "Refactor this repo's auth module and add tests."}],
)
print(resp.choices[0].message.content)

Long single-shot builds are where a 1M-token context earns its keep. Hand it an entire spec, let it run, and compare the result against whatever you pay for now.

Sources and further reading

• moonshotai/Kimi-K3 on Hugging Face (weights, model card, config)
• The Kimi K3 License, full text
• Kimi K3 technical report (PDF): architecture, training recipe, infrastructure, case studies
• Moonshot: official Kimi K3 tech blog
• MoonshotAI/nano-kpu: the inference chip K3 designed autonomously and MoonshotAI/minitriton: the GPU compiler it wrote
• MoonshotAI/FlashKDA: Kimi Delta Attention kernels (MIT)
• MoonshotAI/Attention-Residuals
• vLLM and SGLang, the two supported open inference engines
• SingularityByte: Kimi K3 model breakdown (launch week)

Tested on: not independently benchmarked. Kimi K3 is a 2.8T-parameter MoE requiring multi-node inference, which is beyond our local bench, so every benchmark figure here is Moonshot-reported. Repository contents, file count, byte total, license text, architecture parameters, quantization config and download counts were read directly from the Hugging Face and GitHub APIs, not from secondary reporting. Hardware figures are computed from the measured 1,453.8 GiB weight total.
Date checked: 2026-07-27

Search for that and you get the same list every time. Ollama, llama.cpp, LM Studio, vLLM, SGLang, ranked by tokens per second as though they were five competitors doing the same job. They are not. Understanding why makes the choice obvious in about a minute.

TL;DR

• There are two families, not five competitors. Ollama and LM Studio both embed llama.cpp, so they are close cousins. vLLM and SGLang are a different architecture entirely.
• Picking between Ollama and LM Studio is a UX decision. Picking between Ollama and vLLM is an architecture decision. Only one of those is worth agonising over.
• Everything here is MIT or Apache 2.0 except LM Studio, which is closed source but free, including at work.

The split that explains everything

Family one is built on llama.cpp. llama.cpp is the inference engine: a C++ implementation that runs GGUF-quantised models on almost anything, including CPUs and Apple Silicon. Ollama wraps it in a daemon with a model registry. LM Studio wraps it in a desktop GUI.

This is why throughput comparisons between Ollama and LM Studio are close to meaningless. Community benchmarks put them within roughly five percent of each other, and that is exactly what you would expect from two programs running the same engine underneath. The differences that matter are ergonomics and how quickly each one ships upstream llama.cpp improvements, not speed.

Family two is built for concurrency. vLLM and SGLang use continuous batching and PagedAttention. Instead of processing requests one at a time, they keep the GPU saturated by continuously slotting new requests into the batch as older ones finish, and they manage the KV cache in pages rather than contiguous blocks, which cuts the memory waste that otherwise limits how many requests fit at once.

Ollama and LM Studio do not do this. For a single user that costs you nothing, because there is only one request in flight. Under concurrent load it is the whole ballgame. Community benchmarks report vLLM at more than an order of magnitude above Ollama's concurrent throughput, and while the exact multiple depends entirely on your workload, the mechanism is not in dispute.

So the honest framing is: if you are one person on one machine, benchmark differences between these tools are noise. If you are serving multiple users, Ollama is the wrong shape, not the slower option.

Family one: the llama.cpp lineage

llama.cpp

The engine everything else in this family is built on, at 121,567 stars and MIT licensed. Note it now lives under the ggml-org organisation rather than the original ggerganov account.

Use it directly when you need control or when your hardware is unusual. It runs on CPUs, on Apple Silicon, on modest GPUs, and on things that are not really meant to run language models at all. Its quantisation format, GGUF, is the one you keep seeing on Hugging Face, and if you want to understand what those suffixes mean, we broke that down in quantization formats explained.

The cost is ergonomics. You manage model files, flags and server processes yourself.

Ollama

The most popular project in this entire comparison at 176,859 stars, MIT licensed, and the reason most people's first local model ever ran. It hides llama.cpp behind a daemon and a registry:

ollama run qwen3

That is the whole onboarding. It also exposes an OpenAI-compatible endpoint, which is why nearly every local AI tutorial, including our own n8n generator build, assumes it. It has been moving fast lately, picking up an interactive agent mode in v0.32.

Its limitation is the one above: no continuous batching. Excellent for one user, wrong tool for fifty.

LM Studio, and Jan

LM Studio is the polished desktop app: browse models, click, chat, with a local server when you want one. It bundles both llama.cpp and MLX, so it handles Apple Silicon natively.

It is also the one closed-source tool here, and that deserves a precise statement rather than a warning. LM Studio is proprietary, developed by Element Labs, but it is free for personal and commercial use. The separate work licence was removed in July 2025, so using it at your job requires no form and no fee. Its lms CLI is MIT.

So the reason to prefer an alternative is auditability, not cost. If you want a GUI and open source, Jan is the Apache 2.0 option covering the same ground. Closed source does not mean paid here, and it is worth being clear about which objection actually applies to you.

Family two: built for concurrency

vLLM

The default choice for serving, at 87,140 stars and Apache 2.0. It originated PagedAttention, supports a very wide range of model architectures, and exposes an OpenAI-compatible API so most clients work unchanged.

It is what you reach for when requests arrive in parallel. It is also what the wider ecosystem assumes: both the NemoClaw agent stack and Qwen3-ASR document vLLM as their serving path.

The tradeoff is that it wants a real GPU and more setup than ollama run. There is no CPU-only story worth having.

SGLang

The younger sibling at 30,736 stars, also Apache 2.0, also continuously batched. Its specialisation is workloads where many requests share a long common prefix, which describes RAG almost exactly: the same retrieved context and system prompt in front of many different questions. On that shape it can edge out vLLM.

For general serving with wide model support, vLLM remains the shorter path. Reach for SGLang when your traffic actually has that prefix-heavy shape rather than on principle.

The one that quietly fell behind

Hugging Face's Text Generation Inference used to appear in every list like this. It is worth checking whether it still should. Repository activity, read from the GitHub API on the date below:

Four of the five pushed commits the same day we checked. TGI last saw a push four months earlier and sits at roughly an eighth of vLLM's stars. We are not calling it dead, and Apache 2.0 code does not stop working. But if you are choosing a serving stack to build on in 2026, that row is the one to think hardest about, and the dates say more than any opinion would.

Apple Silicon

MLX is Apple's array framework built for unified memory, and mlx-lm runs language models on it. On an M-series Mac it is generally the fastest path, because it is designed for the architecture rather than ported to it.

You do not necessarily need to touch it directly. LM Studio bundles MLX alongside llama.cpp and picks appropriately, which is a large part of why it is the common recommendation for Mac users who do not want to think about any of this. We used a Mac-native path in running LTX-2 locally on Mac for the same reason.

So which one

Just starting, or one developer: Ollama. The onboarding is genuinely one command and the OpenAI-compatible endpoint means nothing you build against it is wasted if you migrate later.

You want a GUI: LM Studio if you want the most polished experience and do not mind closed source, Jan if you would rather stay open. Both are free.

Building a product with real users: vLLM. Not because Ollama is slow, but because concurrent serving is a different problem and Ollama does not attempt to solve it. Migrating later is not painful, since both speak the same API shape, so starting on Ollama and moving when you have traffic is a reasonable plan rather than a mistake.

RAG at volume: benchmark SGLang against vLLM on your own traffic before committing. The prefix-sharing advantage is real but workload-specific.

Weird hardware, or no GPU: llama.cpp, and see AirLLM for the extreme end of low-VRAM tricks.

The ten-minute move: if you have never run a local model, install Ollama and pull one. If you already have and are wondering whether to switch, the useful question is not which is faster but whether you have concurrent users. If you do not, you are already on the right tool.

Sources and further reading

• Quantization formats explained: what GGUF and the rest actually mean
• Run a full local AI stack on your own hardware
• ggml-org/llama.cpp on GitHub (MIT)
• ollama/ollama on GitHub (MIT)
• vllm-project/vllm on GitHub (Apache 2.0)
• sgl-project/sglang on GitHub (Apache 2.0)
• LM Studio: free for use at work
• Apple MLX on GitHub

Tested on: not independently tested. Our bench has no NVIDIA GPU, so we ran no throughput benchmarks and quote none as our own. The star counts, licences and last-push dates in the table above were read directly from the GitHub API on the date below and are the load-bearing facts in this article. Throughput comparisons are community-reported and workload-dependent; we describe the architectural mechanism, continuous batching and PagedAttention, rather than repeating a specific multiplier as though it were universal. LM Studio's licensing is quoted from its own announcement.
Date checked: 2026-07-25

Qwen3-ASR is Apache 2.0, comes in 0.6B and 1.7B sizes, and the two Transformers-native builds have pulled over 206,000 downloads. The headline you will see is "it beats Whisper." That is too simple, and the real picture is more useful.

TL;DR

• Two models, Apache 2.0, 0.6B and 1.7B, covering 52 languages and dialects (30 languages plus 22 Chinese dialects).
• It is not a Whisper killer, it is a Whisper complement. Whisper still edges it on pristine English audio. Qwen wins on noisy audio, accents, Mandarin and Cantonese, sometimes by enormous margins.
• Not new, despite the recent buzz: released 2026-01-29. What changed is native Transformers support on 2026-06-26, which is what made it a two-line install and drove the download numbers.

What Qwen3-ASR actually is

Two all-in-one speech recognition models from the Qwen team, built on the Qwen3-Omni foundation model. They do language identification and transcription in a single pass, and they handle more than clean dictation: speech, singing voice, and songs with background music are all in scope. Offline and streaming inference run through one unified architecture rather than separate models.

Sizes and adoption, from the Hugging Face API:

On the Hugging Face Open ASR Leaderboard the 0.6B posts a mean WER of 6.31 percent and the 1.7B posts 5.59 percent. Those are aggregate English numbers, and on their own they undersell the model, because the interesting behaviour is in the spread rather than the mean.

Where it beats Whisper, and where it does not

Here is the project's own benchmark table against Whisper-large-v3. Word error rate, so lower is better, and the winner in each row is bold.

The two Librispeech rows are the ones worth sitting with, because they are the same corpus split by difficulty. "Clean" is studio-quality read speech, and Whisper wins it. "Other" is the harder, noisier split, and Qwen wins that. That single pair tells you the shape of the whole comparison: Whisper is marginally better on pristine audio, and Qwen is better as soon as the audio gets difficult.

Then look at Cantonese. Whisper posts 31.04 percent WER, which is roughly one word in three wrong, meaning it is not usable. Qwen posts 4.12 percent. That is not an incremental gain, it is the difference between a feature you can ship and one you cannot. Mandarin shows the same pattern less dramatically, and accented English sits in between.

So the honest summary is not "Qwen3-ASR is better." It is that Whisper was trained in a way that made clean English its strongest case, and Qwen3-ASR is much more robust everywhere else. If your audio is podcast-quality American English, your existing Whisper pipeline is fine. If it is phone calls, accented speakers, background noise, or anything Chinese, this is a meaningful upgrade.

Running it

Native Transformers support is the whole reason this is now easy. One install:

pip install "transformers>=5.13.0"

Then transcribe. The processor has a dedicated transcription-request helper, so you do not hand-build the prompt:

from transformers import AutoProcessor, AutoModelForMultimodalLM

model_id = "Qwen/Qwen3-ASR-0.6B-hf"   # or Qwen/Qwen3-ASR-1.7B-hf
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")

inputs = processor.apply_transcription_request(
    audio="https://example.com/audio.wav",
).to(model.device, model.dtype)

output_ids = model.generate(**inputs, max_new_tokens=256)
generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]

transcription = processor.decode(
    generated_ids, return_format="transcription_only",
)[0]
print(transcription)

audio= takes a local path as readily as a URL. Language identification is automatic, so there is no language flag to set unless you want to pin one.

If you are pushing volume, the project documents torch.compile giving roughly a 2.4x speedup on an A100 at batch size 4, and supports vLLM for serving.

Word-level timestamps

The ASR models transcribe but do not align. If you need per-word timings, for subtitles or for seeking into a recording, that is the separate Qwen3-ForcedAligner-0.6B model. This is a design decision worth knowing before you plan a subtitle pipeline: Whisper gives you segment timestamps in the same pass, whereas Qwen splits transcription and alignment into two models. Two steps, but the alignment model is small and you only pay for it when you need it.

Hardware reality

Be careful here, because the parameter counts invite a wrong assumption. A 0.6B model sounds like something that runs comfortably on a laptop CPU, and it may well. But the project does not document CPU inference at all. Every piece of guidance is GPU-shaped: cuda:0 in the examples, vLLM for serving, FlashAttention 2 for speed, A100 numbers for the benchmarks.

That does not mean CPU is impossible. It means nobody has published what to expect, so if your plan is CPU-only transcription, budget time to find out for yourself rather than treating it as a supported path. The 0.6B is the obvious candidate if you try.

For comparison, this is a real advantage Whisper retains: years of community work has produced whisper.cpp and a mature quantized CPU story. Qwen3-ASR has the better model on hard audio; Whisper has the better ecosystem on constrained hardware. That trade is the actual decision.

Which should you use

Stay on Whisper if your audio is clean English, if you are running CPU-only or on a Raspberry Pi class device, or if you depend on the surrounding tooling (whisper.cpp, faster-whisper, the many wrappers). None of that is displaced.

Switch to Qwen3-ASR if you transcribe Mandarin or Cantonese, where the gap is not close. Also switch if your English audio is accented, noisy, or recorded over a phone, where the Librispeech "other" and accented-English rows say you will measurably do better.

Run both if you are building a product. They are both Apache 2.0, both small, and routing by detected language costs you almost nothing.

The ten-minute move: take the worst piece of audio you have, the one your current pipeline mangles, and run it through the 0.6B with the snippet above. The aggregate benchmarks will not tell you whether this helps your data. That one file will.

Sources and further reading

• Our guide to running Whisper locally
• QwenLM/Qwen3-ASR on GitHub (Apache 2.0, benchmark tables)
• Qwen3-ASR-0.6B on Hugging Face
• Qwen3-ASR-1.7B on Hugging Face
• Qwen3-ForcedAligner-0.6B on Hugging Face
• Qwen3-ASR technical report (arXiv 2601.21337)

Tested on: not independently tested. Our bench has no NVIDIA GPU, and the project documents no CPU path, so we did not run inference. Every WER figure above is quoted from the project's own published benchmark table against Whisper-large-v3, not from a third-party summary, and the download counts and Apache 2.0 licensing were read directly from the Hugging Face API on the date below. Release dates come from the repository's own changelog.
Date checked: 2026-07-25

The headline change since April is not a new open model. It is that the web filled up with confident, well-ranked claims about a model that was never opened at all.

TL;DR

• Wan 2.7 is not open source. Official Wan open weights still stop at 2.2. There is no Wan2.7 repository and no weights on Hugging Face, despite a cluster of sites saying otherwise.
• LTX-2.3 is the real story. 22B parameters, synchronized audio and video from one model, and over 2.1 million downloads. It is the most-downloaded open video model we can find by a wide margin.
• None of the leaders is Apache 2.0 at the frontier. The "they all went permissive" claim is wrong in three different ways, laid out below.

The Wan 2.7 problem

Search for open-source video models today and you will be told, repeatedly and confidently, that Wan 2.7 shipped under Apache 2.0 in April 2026 with first/last-frame control and 9-grid image input. Several of those pages give install instructions and download links.

We went looking for the weights. They are not there:

Wan 2.7 does exist. It is a real model with real capabilities, available through Alibaba's cloud and API. What it is not is downloadable. Alibaba's pattern with Wan has been cloud first, open weights later, and for 2.7 the second half has not happened.

The sites saying otherwise share a tell: many are single-purpose domains built around the model name, publishing "open source guide" and "how to download" pages for something with nothing behind them. They rank well because nobody else was competing for those terms.

The practical consequence for readers of our April piece is reassuring. We tested Wan 2.1 and 2.2 because that was the open Wan, and 2.2 is still the open Wan. That article did not age out. The newer content did.

LTX-2.3 is what actually shipped

The model that genuinely changed the picture came from Lightricks. LTX-2 was open-sourced in January 2026; the current release is LTX-2.3, a 22-billion-parameter diffusion transformer that generates video and synchronized audio together from a single model rather than bolting a soundtrack on afterwards. It handles text-to-video, image-to-video, and video-to-video.

The adoption numbers are what stand out. Set against the other open contenders:

Downloads measure interest rather than quality, and the Wan totals are split across many repository variants, so do not read this as a capability ranking. But a 500x gap over HunyuanVideo-1.5 is not noise, and it lines up with the other signal: an active adapter ecosystem. Lightricks has shipped LoRAs through June and July for video-to-audio foley, cinemagraphs, in/outpainting, clean-plate removal, and spatial upscaling. That is what a model people actually build on looks like.

Requirements

LTX-2.3 wants Python 3.12 or newer, CUDA above 12.7, and PyTorch 2.7 or later. Resolution has to be divisible by 32, and frame count divisible by 8 plus 1, which will bite you the first time you request 100 frames and get an error.

The licensing claim, corrected

The tidiest version of the mid-2026 story is that the open video leaders converged on Apache 2.0. We were ready to write that. It is wrong three times over:

The LTX threshold has a detail worth reading before you assume you are under it: subsidiaries, affiliates, and companies under common control are counted collectively. A small studio owned by a large group may be over the line. Lightricks states there are no retroactive charges, so crossing the threshold is a conversation rather than a penalty.

For most readers of this site, "free under 10 million ARR" is functionally free. It is still not Apache 2.0, and if you are choosing a model on license terms the difference matters.

Check any model claim yourself in about a minute

The useful takeaway is not the specific facts above, which will move again. It is that both Hugging Face and GitHub expose public APIs that settle "is this actually released" in one command, with no key and no account.

Does the model exist on Hugging Face?

curl -s "https://huggingface.co/api/models?search=LTX-2.3&limit=5" \
  | python3 -c "import sys,json; [print(m['modelId'], m.get('downloads')) for m in json.load(sys.stdin)]"

An empty result is your answer. That single check is what showed us Wan 2.7 had no weights.

What has an organization actually published, most recent first?

curl -s "https://huggingface.co/api/models?author=Wan-AI&sort=lastModified&direction=-1&limit=10" \
  | python3 -c "import sys,json; [print(m['modelId'], str(m['lastModified'])[:10]) for m in json.load(sys.stdin)]"

And the repository's real license, rather than what a blog says it is:

curl -s https://api.github.com/repos/Wan-Video/Wan2.2 \
  | python3 -c "import sys,json; j=json.load(sys.stdin); print(j['license']['spdx_id'], j['pushed_at'])"

A NOASSERTION here means GitHub could not match the LICENSE file to a standard license, which is a strong hint you are looking at a custom community license rather than Apache or MIT. That is exactly what HunyuanVideo-1.5 returns.

What this changes about the April comparison

Less than you would expect, which is the point.

Our tested numbers for Wan 2.2 and HunyuanVideo still describe the current open field, because the open field did not move as much as the coverage suggests. The genuine gap is LTX-2.3, which we have not benchmarked and which did not appear in that article at all. If you want the hardware side, the self-hosted stack guide still applies; a 22B model sits above the Wan 2.2 5B variant and below the heavier A14B configurations in memory terms.

What we would test next

Three things for the next entry. LTX-2.3 against Wan 2.2 on the same prompts and hardware as the April run, so the comparison is like for like. Whether the single-model synchronized audio actually beats generating video and audio separately, which is LTX's core architectural bet. And whether the foley and upscaler LoRAs hold up outside the demos, since an adapter ecosystem is only worth something if the adapters work.

If Alibaba does open Wan 2.7, that becomes the story and we will cover it. It has not happened yet.

Sources and further reading

• Part one: our tested open-source video model comparison
• The self-hosted video stack hardware guide
• Lightricks/LTX-2.3 on Hugging Face
• The LTX-2 Community License Agreement
• Wan-Video/Wan2.2 on GitHub (Apache-2.0)
• Tencent HunyuanVideo-1.5 on GitHub

Tested on: not independently tested. We did not run LTX-2.3, which needs an NVIDIA GPU our bench does not have. Every release claim above was verified directly against the Hugging Face and GitHub public APIs on the date below, using the exact commands published in this article: model existence and download counts, organization publishing history, repository licenses, and last-push dates. Capability descriptions are the projects' own. Where our findings contradict widely-published claims, we have shown the check so you can repeat it.
Date checked: 2026-07-25

There is a catch, and it is the reason this article exists. The one-click install path still works, and that is the problem. It silently gives you the old build.

TL;DR

• StableDAW is archived and superseded by theDAW. Same author, same MIT license, roughly twice the application.
• The Pinokio launcher has not been updated. It still clones the archived repo, so the install succeeds and hands you the June build. There is no Pinokio path to theDAW right now.
• Latest release is v0.1.3 (2026-07-06). Use the Windows script, a release installer, the container image, or a source checkout instead.

What happened to StableDAW

The numbers tell the story plainly. The StableDAW repo took its final commit on 2026-06-13 and was archived later that month, sitting at 44 stars. theDAW is a separate, active repo: last push 2026-07-10, 116 stars, and four tagged releases in July. Both are MIT.

This was not a GitHub rename, which matters more than it sounds. A rename leaves a redirect, so old clone URLs quietly follow you to the new project. Here the old repo still exists as a read-only archive at its original address. Anything pointing at it keeps working and keeps serving June code.

Not sure which one you are running? The fastest check is the tab bar. Seven tabs ending in TRAIN means the archived build. Ten tabs including Foundry and Audimate means theDAW. Or ask git directly:

cd /path/to/your/install
git remote -v
# .../gantasmo/stabledaw  -> archived June build
# .../gantasmo/theDAW     -> current

The Pinokio problem

Part one told you to install through Pinokio, because at the time that was the best path available. That advice is now stale, and it fails in the most annoying way possible: quietly.

The launcher at cocktailpeanut/stabledaw.pinokio took its last commit on 2026-06-12, the day before StableDAW's final commit and before the archive. It has not been touched since. Its install step is a single hardcoded clone:

git clone https://github.com/gantasmo/stabledaw app

Because the archived repo is still readable, that clone returns 0. Pinokio reports a clean install. You get a working music studio. It is simply the June build, missing Foundry, Audimate, Perform, VST3 hosting, Demucs stems and MIDI transcription, and it will never update.

We checked for a replacement and there is not one yet: cocktailpeanut/thedaw.pinokio and the obvious variants all 404 as of 2026-07-25. So if you want theDAW today, you install it another way. Below are the paths that actually land on the current code.

Installing theDAW

Four options, roughly easiest first. Requirements across all of them: Python 3.10 or newer, Node.js 20.19+ or 22.12+, FFmpeg on your PATH, and for the Medium model or Magenta RealTime 2, an NVIDIA driver at 550 or later. The Small model still runs on CPU, Apple Silicon included.

Windows: the batch script

Double-click theDAW.bat. It inspects the machine, installs whatever is missing after one confirmation, and opens the app in your browser. There is also theDAW-desktop.bat for the Electron shell rather than a browser tab, and install/setup.ps1 if you would rather drive PowerShell yourself.

Release installers

Every GitHub release ships a Windows installer and a macOS disk image, which is new since part one. Grab theDAW-Setup-<version>.exe or theDAW-<version>.dmg from the releases page. Current version is v0.1.3.

Container

There is now an official image, which is the cleanest option if you already run containers and do not want Python and Node on your host:

docker pull ghcr.io/gantasmo/thedaw
# the repo also ships a docker-compose.yml

From source

The one to use if you want to track development. Note the submodules flag, the checkout is incomplete without it:

git clone --recurse-submodules https://github.com/gantasmo/theDAW
cd theDAW

# backend
uv run uvicorn backend.server:app --host 0.0.0.0 --port 8600 --reload

# frontend, in a second terminal
cd frontend && npm run dev

Port 8600 is not negotiable: the frontend proxies /api/* to it and that address is baked into the Vite config. The frontend takes 5173, and the launcher also clears 5187. Same port story as part one, so if you had those free before, you still do.

The four new workspaces

Underfit is TRAIN with a new name and more depth. It fits LoRA adapters to the Stable Audio 3 RF base models across eight adapter types: lora, dora-rows, dora-cols, bora, and an -xs variant of each. You can filter which layers get adapted with --include and --exclude, adjust adapter strength at runtime rather than retraining, and stack multiple adapters additively. Still needs an NVIDIA GPU.

Foundry is the one with no obvious equivalent anywhere else. It is an infinite-canvas designer for plugin interfaces: you lay out the controls, and the finished design exports as a .gan web-plugin that hosts inside the MIX chain right next to VST3 plugins and the built-in effects. Designing your own plugin UI is not something local AI audio tools normally let you do.

Audimate is a node-graph editor for building generation pipelines. Nodes cover library sources, generation (Stable Audio or Magenta), effects, merges and feedback stages, wired together with bezier edges on a pannable canvas. If you have used ComfyUI for images, the shape will be familiar.

Perform is a live clip launcher fed from imported projects, aimed at playing a set rather than arranging one. Together with the DJ tab's new stems, key-lock and automix, the live half of the app got most of this release's attention.

What else came along

Three additions are worth knowing about even though they do not have their own tabs. theDAW ships a CUDA port of Google's Magenta RealTime 2, which the project describes as the first non-Mac port. Demucs handles stem separation, so you can pull a track apart rather than only generating new material. And audio can now be transcribed to MIDI and rendered as notation, which closes the loop between generated audio and something you can actually edit as music.

Moving your existing setup

There is no migration tool, so treat this as a fresh install alongside the old one rather than an upgrade in place. Keep the archived install until you have confirmed the new one works.

Your generated audio is just files, so copy them out of the old install's data directory. LoRA adapters you trained in TRAIN target the same Stable Audio 3 RF base models that Underfit uses, so they should carry over, though verify strength settings after loading rather than assuming they match. The one thing you cannot bring is the LEARN genealogy graph, which is tied to the old install's library database.

Limitations and gotchas

• No Pinokio path. The headline issue. Until someone updates the launcher, the one-click route lands on archived code.
• Early version numbers. v0.1.3, with four releases in two days, and the repo carries a file named ERRORS-AND-FIXES-2026-07-05.md. This moves fast and breaks.
• The GPU line moved. More of the interesting surface now needs NVIDIA: Medium model, Magenta RT2, Underfit training. CPU users still get the Small model and the whole edit, mix and perform side.
• Licensing is unchanged. App code MIT, model weights under the Stability AI Community License. Commercial use is fine under 1 million dollars in annual revenue. Still no vocals: Stable Audio 3 makes instrumental music and sound effects.
• Submodules. A plain git clone gives you a broken checkout. Use --recurse-submodules.

Who should move, and when

If you installed through Pinokio after part one and have been happy, you are on the June build and you are missing a lot. Moving is worth it, but do it as a parallel install.

If you never installed it, skip StableDAW entirely and start at theDAW with a release installer or the container.

If you only ever wanted a local text-to-audio sketchpad on a laptop with no GPU, the archived build genuinely still does that. There is no urgency.

The ten-minute move: run git remote -v in your install directory. If it says stabledaw, you know where you stand, and the container pull above gets you current without touching your existing setup.

Sources and further reading

• Part one: StableDAW, the one-click AI music studio
• gantasmo/theDAW on GitHub (MIT)
• theDAW releases
• gantasmo/StableDAW, archived
• The unmaintained Pinokio launcher
• Stable Audio 3 Medium on Hugging Face

Tested on: not independently tested. Our bench has no NVIDIA GPU, so we did not run theDAW end to end. Everything above comes from the project's own repositories and release artifacts, verified against the GitHub API on the date below: archive status and commit dates on both repos, the release tags and timestamps, the launcher's clone target read from its install.js, and the absence of a theDAW Pinokio launcher. Feature descriptions are the project's own.
Date checked: 2026-07-25

TL;DR

• NemoClaw is a blueprint, not a product: Nemotron 3 Ultra for the model, LangChain Deep Agents Code for the harness, NVIDIA OpenShell for the sandboxed runtime.
• All three layers are open, and OpenShell is Apache 2.0. Despite the NVIDIA branding, none of it requires an NVIDIA GPU, because GPU support in OpenShell is optional and the model can sit behind any OpenAI-compatible endpoint.
• The headline "10x cheaper" number comes from LangChain's own eval suite, run by the two companies announcing the product, against an unnamed competitor. Treat it as vendor-reported.

What changed since part one

Part one covered a model. This covers the two pieces that were missing around it.

An agent is not a model. It is a model plus a loop that plans, calls tools, remembers things, and keeps going for hours. That loop is the harness. And because the loop runs shell commands and touches files, it needs somewhere safe to do that, which is the runtime. Until now you could get an open model easily and an open harness fairly easily, but the runtime layer was where you quietly ended up on somebody's hosted platform.

NemoClaw fills that gap. The harness is LangChain's Deep Agents Code, shipped as a terminal agent called dcode, in the same category as Claude Code or Codex. The runtime is NVIDIA OpenShell. Both are open, both are self-hostable, and that is the whole story.

OpenShell is the part worth paying attention to

The model gets the headlines, but OpenShell is the piece that did not exist before.

It is a sandbox for autonomous agents, written in Rust, that runs a K3s Kubernetes cluster inside a single Docker container. You do not install Kubernetes separately. Agents run unmodified inside it while OpenShell enforces filesystem, network, and process rules from a declarative YAML policy, and it keeps an audit trail of every allow and deny decision it makes.

That last part matters more than it sounds. Most people running coding agents today either give them full access to a real machine and hope, or run them in a container and lose track of what they actually did. An audit log of denied actions tells you what your agent tried to do, which is exactly the thing you want to know before you let it run unattended for eight hours.

It is also honest about its maturity. NVIDIA describes it as alpha and calls it "single-player mode": one developer, one environment, one gateway. That is not a stack you put in front of a team this quarter.

Getting it running

OpenShell supports Linux, Apple Silicon macOS, and WSL2 (experimental). There are two install paths. The PyPI one:

uv tool install -U openshell

And the official installer, which fetches a release package from GitHub:

curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh

Then you create a sandbox and run an agent inside it:

openshell sandbox create -- claude
openshell sandbox connect
openshell policy set default --policy policy.yaml

The gotcha nobody documents

The two install paths do not have the same requirements, and we hit this the hard way. The PyPI wheels are built for manylinux_2_39, which means glibc 2.39 or newer. On Debian 11 (glibc 2.31) uv refuses to resolve:

× No solution found when resolving dependencies:
╰─▶ Because all versions of openshell have no wheels with a matching
    platform tag (e.g., `manylinux_2_31_x86_64`) ...
    hint: Wheels are available for `openshell` (v0.0.91) on the following
    platforms: `manylinux_2_39_aarch64`, `manylinux_2_39_x86_64`,
    `macosx_13_0_arm64`

The shell installer is more forgiving. Read the script and it sets its own floor at glibc 2.28, because it installs a Debian or RPM package rather than a Python wheel. So on an older distro the uv path fails and the install.sh path works. The tradeoff is that the package install needs root and registers a systemd user service, which is a bigger commitment than a user-local tool.

If you are on Ubuntu 24.04 or later, or a current Fedora, both paths work and this never comes up.

Pointing the harness at Nemotron

Deep Agents Code installs with its own script:

curl -LsSf https://langch.in/dcode | bash

It is model-agnostic, which is the useful bit. Any provider that speaks tool calling works, and providers are declared in config.toml. Since Nemotron 3 Ultra is served over an OpenAI-compatible API, you register it like any other provider. Adapted from LangChain's documented schema:

[models]
default = "openrouter:nvidia/nemotron-3-ultra-550b-a55b:free"

[models.providers.openrouter]
display_name = "OpenRouter"
api_key_env = "OPENROUTER_API_KEY"
base_url = "https://openrouter.ai/api/v1"
models = ["nvidia/nemotron-3-ultra-550b-a55b:free"]
enabled = true

[models.providers.openrouter.params]
temperature = 0

One useful detail: dcode checks DEEPAGENTS_CODE_{NAME} before falling back to {NAME} for credentials. So you can set DEEPAGENTS_CODE_OPENROUTER_API_KEY and give the agent its own key without touching what every other tool on your machine uses.

Putting the harness inside the sandbox

Here is the part the announcement glosses over. OpenShell's four headline agents (Claude Code, OpenCode, Codex, Copilot CLI) are not a whitelist, they are the ones that come pre-installed in the base image with credentials auto-discovered from your shell. Everything after -- on sandbox create is passed verbatim to the container, so any binary runs. dcode simply is not one of the pre-baked ones, which means you put it in the image yourself.

That is what --from is for. It takes a community image name, a local directory with a Dockerfile, or any container image reference:

# build a sandbox image that has dcode installed
openshell sandbox create --from ./my-dcode-sandbox --name nemo -- dcode

# register the model endpoint explicitly, since auto-discovery
# only covers the four pre-installed agents
openshell provider create

Two flags worth knowing while you are here. --policy attaches your YAML rules at creation time, and --gpu requests GPU passthrough, which is opt-in. That last one is the clearest evidence that the GPU is genuinely optional rather than quietly assumed.

You do not need to self-host 550B to try this. OpenRouter carries a free Nemotron 3 Ultra endpoint at 200 requests per day with the full 1M context, which is enough to see whether the harness fits how you work. Paid pricing is $0.50 per million input tokens and $2.20 per million output. The free tier logs requests, so keep anything confidential off it.

About that 10x number

The announcement's headline claim is that Nemotron 3 Ultra under Deep Agents scored an aggregate 0.86 at $4.48 of inference cost, against $43.48 for "the next closest model."

Read that carefully. It is LangChain's own eval suite. It was run by the two companies announcing the product. The competitor is not named, and no third party has replicated it. None of that makes it false, and the underlying economics are plausible, since a 55B-active MoE really is cheaper to serve than a dense frontier model. But an unnamed baseline in a vendor benchmark is a number you quote with attribution, not one you plan a budget around.

The claim we would actually stand behind is the boring one: you can run this stack yourself, inspect every layer, and swap any of them out. That does not need a benchmark.

Limitations and gotchas

• OpenShell is alpha. NVIDIA says so directly. Single developer, single environment, single gateway.
• The glibc split above. Old distro, use the shell installer, expect to need root.
• dcode is not a pre-installed OpenShell agent, though it still runs. Only Claude Code, OpenCode, Codex and Copilot CLI ship in the base image with credentials auto-discovered, so getting the harness into the sandbox is on you. See the section above.
• Self-hosting the model is a datacenter problem. Nemotron 3 Ultra needs a multi-GPU node even at NVFP4, as covered in part one. The realistic setup for most people is open harness and open runtime locally, model over an API.
• Free-tier limits. 200 requests per day disappears fast when an agent runs a long task with many tool calls.

Who should use it

If you are building agents you intend to run unattended, on data you care about, OpenShell is worth an afternoon on its own, independent of anything NVIDIA. It is Apache 2.0 and it works with agents you already use.

If you are shopping for an agent stack you can audit end to end, with no layer you cannot inspect or replace, NemoClaw is currently the only complete answer. Just size the alpha warning honestly.

If you want a coding agent that works today, this is not that. Use what you already have and revisit when OpenShell leaves alpha.

The ten-minute move: install dcode, point it at the free Nemotron endpoint with the config block above, and give it one real task. That tells you whether the harness suits you before you invest in the runtime.

What we are watching next

Three things for the next entry in this series. Whether OpenShell picks up multi-user support, which is what turns it from a developer tool into infrastructure. Whether anyone independently replicates the cost claim. And whether the next Nemotron ships with the same open data and recipes, because that, not the parameter count, is what made part one worth writing.

Sources and further reading

• Part one: Nemotron 3 Ultra, a fully open 550B agent model
• LangChain: NemoClaw Deep Agents blueprint announcement
• NVIDIA: Nemotron with the LangChain Deep Agents harness
• NVIDIA OpenShell on GitHub (Apache 2.0)
• OpenShell documentation
• OpenShell: supported agents and base image contents
• OpenShell: managing sandboxes and the --from flag
• LangChain deepagents on GitHub
• Deep Agents Code documentation
• Nemotron 3 Ultra free endpoint on OpenRouter

Tested on: not independently tested end to end. We verified the install paths on Debian 11 (Intel i7-10510U, 6 cores, no GPU) and confirmed the glibc 2.39 wheel requirement and the installer's 2.28 floor from the release script. We did not run an agent under OpenShell: the PyPI path will not resolve on glibc 2.31, and the package path requires a root install of alpha software. Benchmark figures are vendor-reported by LangChain and NVIDIA and are labeled as such above.
Date tested: 2026-07-25

This is not a closed-model review. SingularityByte's editorial line is that we cover closed models only when they directly impact how the open-source ecosystem ships. Claude, GPT, and Gemini pass that test three ways: they subsidize prototype work that later runs on open weights, they generate the synthetic data that trains the next Qwen or Llama fine-tune, and they provide the evaluation baseline that every open-weights release measures itself against. If any of those pipelines breaks, the open-source flywheel slows down.

The Framing Question

A working definition for this piece: a "useful" free AI tier for open-source work is one that lets you run a weekend-scale prototype (roughly ten to twenty thousand tokens a day of real agent traffic) without funding a card, without losing the right to ship the code under an open-source license, and without getting your prompts ingested into the provider's next training run. Measured on those three axes, none of the big three is perfect, and the rankings change depending on what you are actually building.

We are not asking "which model is smartest." We are asking "which one lets you ship open-source code in 24 hours for zero dollars without a compliance headache."

Anthropic Claude Free Tier: Starter Credits, Not a Tier

Claude's free offering is the thinnest of the three, and Anthropic is upfront about it. A new account at platform.claude.com gets a small starter credit (community reports put it around five dollars at current Haiku 4.5 pricing) and no recurring monthly free allocation. Once the credit burns, the API returns HTTP 402 until you prepay. There is no "always free" Claude endpoint.

Where this stops mattering is the Claude for Open Source program, launched in late February 2026 and expanded on 2026-07-08. Qualifying maintainers get six months of Claude Max 20x free, roughly 1,200 dollars of value, across up to 10,000 spots. That is effectively unlimited Opus 5, Sonnet 5, and Haiku 4.5 traffic for anyone maintaining a reasonably active public project.

The July expansion widened eligibility past library authors to core contributors and critical packages. There are now four qualification tracks, and you only need one:

• Maintainers and library authors: 500+ dependent repos, 100+ dependent packages, or 200,000+ combined monthly downloads across npm, PyPI, crates.io, RubyGems or similar.
• Core contributors: a listed committer or maintainer on a recognized foundation or language project (CPython, the Rust team, Node.js TSC, Apache PMC, CNCF, Kubernetes, the Linux kernel, Django, Rails).
• Active contributors: 100+ pull requests merged into repos you do not own in the last 12 months.
• Community builders: one of your repos has had 20+ unique external contributors with merged PRs in the last 12 months.

Applications are reviewed on a rolling basis against that 10,000 cap, so check the official page for current status before you plan around it. None of these flows are self-serve: you fill in a form and wait.

For an open-source builder, the practical read is this. Use the starter credit to prove the agent loop works. If you clear any of the four tracks above, apply the same week and budget zero dollars after that. If you are a solo dev on a fresh GitHub account, your Claude free tier will last two days.

For sizing the starter credit, current API pricing per million tokens is Opus 5 at 5 dollars in and 25 out, Sonnet 5 at 3 and 15 (with an introductory 2 and 10 running through 2026-08-31), and Haiku 4.5 at 1 and 5. Haiku is what makes a small credit stretch.

What You Can Actually Build In 24 Hours (Claude)

• Full agent loop with Sonnet 5 and tool calling, enough tokens to run roughly 200 to 400 real agent turns.
• A synthetic instruction-tuning dataset of a few thousand examples for a Qwen or Llama fine-tune, if you use Haiku 4.5 as the generator.
• Evaluation harness comparing your open-weights model to Sonnet 5 on a benchmark of a couple of hundred prompts.

OpenAI GPT Free Tier: The Smallest Gate

OpenAI's free API story is the weakest of the three for open-source developers. There is no standing free tier on the API. New accounts get a small trial credit (historically three to eighteen dollars, now usually nothing unless you qualify for a promotional code) and the rest is prepaid. ChatGPT Free is a product, not an API: it does not give you an OpenAI API key, and you cannot call it from code.

The one thing that has moved the needle for open-source builders in 2026 is GPT-OSS, OpenAI's first open-weights release in years. The 120B and 20B variants are now hosted free on Groq, Cerebras, and OpenRouter. That is genuinely useful: you can evaluate against a GPT-adjacent model without touching OpenAI's API at all. GPT-OSS 120B is MIT licensed and runs on two H100s or the Cerebras free tier. If your angle is "I want to benchmark open-weights against the closest thing to a GPT we are allowed to see," go straight to Cerebras and skip OpenAI entirely.

For closed-source GPT-5 and o3-mini access, the answer for free is: use it through Microsoft's Azure AI Foundry free credits (one hundred dollars for students, one fifty for new Azure accounts), or wait for a marketing promo. The OpenAI free API tier that builders actually want does not exist.

What You Can Actually Build In 24 Hours (GPT)

• On OpenAI direct: nothing meaningful for free. Assume you need credits.
• On Groq or Cerebras via GPT-OSS 120B: a full agent loop, synthetic data generation, and benchmark harness with a million tokens a day headroom on Cerebras alone.
• Via Azure AI Foundry credits: GPT-5 access for the duration of the promo, with a clock attached.

Google Gemini Free Tier: The Volume Leader

Gemini is the answer if you want volume without a card. As of 2026-07-16 Google's pricing page lists free-tier rows for Gemini 3.5 Flash and Gemini 3.1 Flash-Lite on the standard tier. The practical quota for Flash lands around 15 requests per minute, 1 million tokens per minute, and 1,500 requests per day. Google Search grounding is included at 5,000 prompts per month.

One caveat that trips people up: Google no longer publishes a universal rate-limit table. Quotas are assigned per project and vary by region, account age, and whether billing is attached, so the numbers above are the commonly reported shape rather than a contractual floor. Your actual quota is visible in the AI Studio console, and that is the only figure worth planning against.

There is a price tag that is not measured in dollars. Google's pricing page is explicit: free-tier content is used to improve their products, paid-tier content is not. For open-source developers this is a two-way trade. If your prompts contain customer data, proprietary code, or anything under NDA, Gemini free tier is disqualified. If your prompts are public test harnesses, synthetic data generation on open datasets, or evaluation runs against open-weights models, the trade is fine, and you get the most generous free tier on the market.

Gemini Pro is not free on the standard tier. If you want Pro, you pay. Flash is the free workhorse, and it is strong enough to prototype against Qwen3 235B, GPT-OSS 120B, or Llama 3.3 70B with results you can learn from.

What You Can Actually Build In 24 Hours (Gemini)

• A high-throughput synthetic dataset generator running at the RPM cap for several hours, producing tens of thousands of training examples for an open-weights fine-tune.
• A multimodal RAG prototype using Gemini 3.5 Flash for text and image inputs, no credit card.
• An evaluation harness that benchmarks your Qwen or Llama output against Flash on thousands of prompts per day.

Head-to-Head: Open-Source-Friendly Ranking

Ranking for the specific question "which free AI lets me ship open-source in 24 hours for zero dollars?":

How These Free AI Tiers Subsidize Open-Source Builders

There is a real, measurable economic story hidden in the free-tier data. Every open-weights fine-tune shipped in the last eighteen months (Qwen, DeepSeek, Hermes, OpenHermes, Dolphin, Nous) used synthetic data generated from a closed model at some step of the pipeline. The free AI tiers from Anthropic, OpenAI, and Google are, indirectly, the R&D budget of the open-source fine-tuning community. When Google gives away Gemini 3.5 Flash for free, a dozen hobbyist fine-tunes that would never get built at three dollars per million tokens get built at zero. When Anthropic gives Claude Max away to open-source maintainers, the agentic framework ecosystem (CrewAI, Letta, LangGraph) gets its prototyping subsidy.

This is not charity. It is a distribution play. Every free-tier token is an impression: "our model is the one you reach for first." The builders who stay in the ecosystem eventually pay. The open-source community benefits from the overflow.

What this means operationally: do not feel guilty about using a free tier to prototype open-source code. Do feel responsible about citing the tier you used, and about publishing your evaluation harness so other builders can replicate it on the same free quotas.

The Rate-Limit Reduction Warning

One prediction for the next ninety days. Free AI tiers always shrink. Groq cut its Llama 70B daily token budget twice in 2025. Google silently reduced Flash free-tier RPM once already in 2026. Anthropic may narrow the open-source program eligibility when the first cohort reviews. OpenAI has trended toward less free access, not more. Build your agent with a fallback chain from day one: if Gemini free throws a 429, fall back to Groq, then Cerebras, then a local Ollama on Qwen3. Tools like LiteLLM, OpenRouter, and custom routers make this a three-line change in your client code.

Actionable Framework: The Free-Tier OSS Workflow

A checklist for using closed-model free tiers to ship open-source code without regret:

• Pick one closed tier per pipeline stage. Prototype on Gemini, eval on Claude Haiku 4.5, benchmark against GPT-OSS on Cerebras. Do not blend them.
• Log every prompt and response. Use Langfuse or Phoenix. When the free tier shrinks, your logs become a replay harness on open weights.
• Never ship a closed dependency into production. Closed-model calls belong in the notebook, not in the Docker image.
• Publish your evals on open weights. Your closed-model baseline is private infrastructure. Your Qwen or Llama numbers are the public artifact.
• Apply to open-source programs the same week you start a project. Claude for Open Source, GitHub Models, Azure AI Foundry credits. A ten-minute form can save six months of compute spend.

Risks and Open Questions

Three things to watch in the next ninety days. First, Gemini's data-for-compute trade gets tightened or loosened (it has moved twice in 2025). Second, Anthropic expands or contracts the open-source maintainer program based on cost. Third, OpenAI ships (or does not ship) a recurring free API tier to compete with Gemini. Any one of those three moves reshuffles this ranking.

The deeper question: as open-weights models close the gap with Claude and Gemini on instruction following and coding, does the free AI tier lose its value for open-source builders? Probably not for another 18 months. Gemini 3.5 Flash still catches failure modes that Qwen3 235B Instruct misses, and Claude Opus 5 still writes cleaner reasoning chains than any open model we have tested. Free closed-model tiers will stay useful as long as the frontier sits inside those APIs.

Sources and Further Reading

• Claude API pricing and free credits
• Claude for Open Source program
• Google Gemini API pricing
• OpenAI API pricing
• GPT-OSS open-weights release on Hugging Face

Open Google AI Studio, grab a free Gemini 3.5 Flash key, set it as GEMINI_API_KEY, run the starter example at ai.google.dev, and you have a multimodal free AI endpoint in the terminal in under ten minutes.

Free tiers and model lineups move fast. Every figure above was verified against the providers' own pricing and program pages on the date below; check the linked sources before planning around any single number.
Date checked: 2026-07-25

What changed since part one

Part one drew a hard line between two things people both call "edge." A microcontroller like the ESP32 has kilobytes to low megabytes of usable memory and runs TinyML: tiny quantized classifiers that spot a keyword, a sound, or an anomaly. A single-board computer like a Raspberry Pi has gigabytes and can run a small language model on its CPU. The gap between them is three orders of magnitude, and no amount of quantization closes it.

The esp32-ai project does not break that rule. It bends the memory question instead. The previous ESP32 language-model demo we cited held a 260K-parameter network. This one holds 28.9M parameters, coherent enough to finish a sentence and keep a short story on track, on the same class of $8 chip. The output is the same category as before, short synthetic stories, but the model behind it is two orders of magnitude larger. The interesting part is not the story. It is how 28.9M parameters fit somewhere that cannot hold them in RAM.

The trick: Per-Layer Embeddings from flash

Start with the constraint. Transformer inference wants its weights in fast memory, because it touches them on every token. The ESP32-S3 has 512KB of on-chip SRAM. A 28.9M-parameter model at 4-bit is roughly 14.5MB. It cannot live in RAM. Bolting on the 8MB of PSRAM does not fix it either, and PSRAM is slow anyway, which matters later.

The move is to notice that not every parameter is a compute parameter. Per-Layer Embeddings (PLE) is a design from Google's Gemma models. The idea: give each token, at each layer, its own learned embedding vector pulled from a big lookup table, instead of folding all that capacity into the dense weights the math runs over. The table is huge in parameter count but you only ever read the handful of rows a given token needs. It is a dictionary, not a computation.

That property is what makes it fit a microcontroller. The esp32-ai project splits the model across all three memory tiers by what each part actually needs:

The dense math the chip runs on every token is tiny, about 559K parameters, and it fits in fast SRAM. The 25M parameters of extra capacity sit in flash as a memory-mapped table. Per token the model reads about 450 bytes from that table, six rows, in roughly 0.12 milliseconds. The author measures the table at about 0.7 percent of per-token memory time. You get the model quality of 28.9M parameters while paying, on the hot path, for about half a million.

This is the same family of idea as the memory-saving quantization tricks we covered in our Google TurboQuant writeup. The theme is consistent: on constrained hardware, the wall is memory, and the wins come from not holding things you do not need in the fast tier.

Does the table actually help, or is it plumbing?

A fair objection: maybe the gain comes from the extra wiring, not from the 25M-parameter table itself. The author ran the ablation to check, measuring perplexity (lower is better, it is how well the model predicts held-out text) across four configurations.

PLE cuts perplexity from 12.58 to 11.41, about 9.3 percent, while keeping the compute core the same size. FatEmbed, a simpler way to spend the same parameter budget, does worse. The control run, ple_notable, wires up the same plumbing but without a real lookup table, and it lands worse than baseline. The author's summary is the right one: "the table does the work, not the plumbing." And the gain survives quantization. The PLE advantage measured in full precision actually holds or grows after 4-bit post-training quantization, at 124 to 126 percent retention, so the technique is not a floating-point artifact that evaporates on the shipping format.

Where the time goes: the bottleneck moved, it did not vanish

Fitting the model is one problem. Running it fast enough to be bearable is another, and this is where the honest engineering shows. At ~9.5 tok/s a token takes about 105 milliseconds. Here is where that time goes, per the author's profiling on the dual-core chip.

Read that top row twice. The output head, the layer that turns the model's internal state into a probability over all 32,768 vocabulary tokens, eats 57.6 ms, more than half the budget. It lives in PSRAM, and PSRAM reads at about 60.7 MB/s. Scanning the 2.43MB of head weights once takes roughly 40 ms just to move the bytes, before any math. Compute gets the leftover ~17 ms. The flash lookup table everyone worries about? 8.5 ms, and most of that is not the table reads themselves.

So the PLE trick did not make the model free. It moved the bottleneck. The dense core no longer bounds you; PSRAM bandwidth does. The author's own math puts the theoretical ceiling around 58 tok/s if you were perfectly bandwidth-bound, and the current runtime sits well under that because the compute stages have not all been overlapped with the reads yet. This is the useful lesson for anyone building on constrained hardware: solve the memory-capacity wall and you often just meet the memory-bandwidth wall behind it.

Running it yourself

The project ships everything: the training and quantization code in Python, the ESP32 firmware, and the exported model. You do not need a training run to try it, the repo includes a pre-exported model you can flash. The firmware builds with the Arduino ESP32 core (3.3.10) driven by arduino-cli, not ESP-IDF or PlatformIO. The one wrinkle worth knowing is that the model binary is flashed to its own partition, separate from the firmware, so firmware-only changes do not force you to rewrite the 15MB model.

# 1. Export and verify the model against a golden reference on your host
cd src && uv run python export.py && cd ..
cc -O3 -o /tmp/esp32-llm-verify firmware/host_verify/verify.c -lm
/tmp/esp32-llm-verify firmware/model/model.bin firmware/model/golden.txt

# 2. Compile and upload the firmware to an ESP32-S3
arduino-cli compile --fqbn esp32:esp32:esp32s3 firmware/esp32_llm
arduino-cli upload   --fqbn esp32:esp32:esp32s3 -p /dev/ttyUSB0 firmware/esp32_llm

# 3. Write the model binary to its own flash partition (only needed after export)
esptool.py --chip esp32s3 write_flash 0x110000 firmware/model/model.bin

# 4. Watch it generate
arduino-cli monitor -p /dev/ttyUSB0 -c baudrate=115200

What comes out is a stream of short TinyStories-style prose at about 9.5 tokens per second, rendered on a small attached display. The repo has a demo GIF if you want to see it move before you buy a board. Treat the code and weights as an experiment: this is a research demo of a storage technique, not a maintained product, so pin your toolchain versions and expect to read the source.

What it proves, and what it does not

Here is the honest verdict, and it is two-sided on purpose.

What it proves: you can hold a 28.9M-parameter model on a $8 microcontroller and run it, if you are willing to store most of it as a memory-mapped flash table and read only the rows each token needs. Per-Layer Embeddings, pulled from a research idea into a working ESP32 firmware, is a real and reusable technique. If your problem is "this model is a bit too big for the fast memory I have," moving the embedding capacity to a slower tier you read sparsely is a lever worth knowing. The ablation shows the capacity is doing real work, and it survives 4-bit quantization.

What it does not prove: that a microcontroller can run a useful language model. This model writes toy stories. It will not answer a question, follow an instruction, call a tool, or know a single fact, because TinyStories does not teach any of that, and 28.9M parameters could not hold much of it anyway. The output category is exactly what part one described, just bigger and more coherent. And the moment you chase real capability you need a real vocabulary, a real instruction-tuned model, and a KV cache for context, and every one of those pushes you straight back off the microcontroller and onto a Raspberry Pi or a phone. Part one's cheat sheet still stands: put the reflex on the sensor, keep the brain where the memory is.

Who should care, and what to watch

If you build products on constrained hardware, this is not a component you ship. It is a technique you file away. The people who should actually clone the repo are hobbyists who want to see an LLM breathe on an $8 chip, and researchers hunting for ways to fit more capacity into fixed memory budgets. The one thing worth watching: whether anyone applies the same flash-resident PLE trick to a small instruction-tuned model rather than a story generator. That is the experiment that would test whether this is a curiosity or a path. Until then, it is a very good curiosity, and a clean demonstration that on tiny hardware the binding constraint is always memory.

A community fork put it on an 8MB board

Since this series is about what actually runs, here is a data point from the wild. A developer forked the project to get it onto a Seeed XIAO ESP32-S3, a board with only 8MB of flash instead of the 16MB the stock 14.9MB model needs. The full model does not fit, so the fork retrains a smaller variant with an 8,192-token vocabulary instead of 32,768. That drops the model to about 8 million parameters and a 4.12MB file, which fits an 8MB board with room to spare. Same Per-Layer Embeddings architecture, same TinyStories training, just a smaller tokenizer.

It works, and here is the twist: on the XIAO ESP32-S3 the fork reports about 11.97 tokens per second (82.9 ms per token), faster than the original's 9.5 tok/s. The reason is exactly the bottleneck described above. The output head is PSRAM-bandwidth-bound, and a smaller vocabulary means a smaller head: 0.82MB staged versus 2.43MB in the original. Shrink the thing that dominates the per-token memory scan and the whole model speeds up. It is this article's thesis confirmed on real silicon: on this hardware the binding constraint is memory bandwidth, so the size of the vocabulary head moves the number more than anything else.

A sample of what the 8MB build wrote, unedited:

Once upon a time, there was a little girl named Lily. She loved to play with her toys and run around in the park. One day, she found a big, red ball in the park. She wanted to play with it, but her mom said no.

The fork, with an 8MB partition table, a PlatformIO build, and the smaller re-export, is on GitHub: karamble/esp32-ai.

Sources and further reading

• esp32-ai repository (slvDev)
• RESULTS.md: methodology, ablations, and measurements
• TinyStories dataset (Hugging Face)
• Gemma 3 model card (Per-Layer Embeddings)
• ESP32-S3 datasheet (Espressif)
• Part one: edge AI on microcontrollers, what actually fits

The original 28.9M model's figures are author-reported from the repository's RESULTS.md (2026-07-21). The 8MB-board figures (Seeed XIAO ESP32-S3, vocab-8192 re-export, about 11.97 tok/s) are reported by the community fork linked above.
Date checked: 2026-07-24

This is part two of a two part thread. Part one built a website chatbot backed by markdown files. Everything there still stands: the same AI Agent, the same Chat Trigger, the same website widget. We change exactly one thing, the tool the agent uses to find information, and in return the chatbot scales from a dozen documents to thousands.

What Actually Changes

Because only the tool changes, you can run both side by side and migrate when you are ready. The website embed from part one does not change at all.

Prerequisites

You need n8n, Ollama, and Supabase. The local-ai-packaged stack ships all three together, which is why we recommend it for this series. Pull an embedding model alongside your chat model:

ollama pull nomic-embed-text
ollama pull qwen2.5:7b-instruct-q4_K_M

nomic-embed-text turns text into 768 dimensional vectors and runs comfortably on a CPU. It is the embedding model the reference workflows use, and the vector width matters in the next step.

Step 1: Turn Supabase Into a Vector Store

Supabase is Postgres, and Postgres becomes a vector database with the pgvector extension. In the Supabase SQL editor, enable the extension, create a table for embedded chunks, and add the search function n8n calls. Run this once:

create extension if not exists vector;

create table documents (
  id bigserial primary key,
  content text,
  metadata jsonb,
  embedding vector(768)
);

create function match_documents (
  query_embedding vector(768),
  match_count int default null,
  filter jsonb default '{}'
) returns table (
  id bigint,
  content text,
  metadata jsonb,
  similarity float
) language plpgsql as $$
begin
  return query
  select id, content, metadata,
    1 - (documents.embedding <=> query_embedding) as similarity
  from documents
  where metadata @> filter
  order by documents.embedding <=> query_embedding
  limit match_count;
end;
$$;

Two details decide whether this works. The vector width, 768, must match your embedding model exactly, or inserts fail. And match_documents is the function n8n's Supabase Vector Store node calls by name, so do not rename it without updating the node.

Step 2: The Ingestion Loop

Now build a second workflow whose only job is to get documents into that table. This is the part markdown lookup did not have. A production version, like the one behind the Decred knowledge assistant, walks a folder, skips anything it has already embedded, and re-embeds only what changed. The node chain looks like this:

[
  "Manual or Schedule Trigger",
  "Read Files from Disk (glob /data/shared/**/*.md)",
  "Loop Over Items (Split in Batches)",
  "Hash File (crypto, SHA-256)",
  "Check hash against document_metadata (Supabase)",
  "Switch: new file / unchanged / changed",
  "Extract Document Text",
  "Supabase Vector Store (insert)"
]

The Supabase Vector Store node in insert mode does the heavy lifting. Attach three sub-nodes to it:

• Embeddings Ollama pointed at nomic-embed-text. This turns each chunk into the 768 dimensional vector the table expects.
• Default Data Loader to carry document metadata (file name, source URL, hash) alongside the text.
• Recursive Character Text Splitter with a chunk size around 400 and a small overlap. Chunking is what makes semantic search precise: the agent retrieves a paragraph, not a whole file.

The dedup step is what keeps a growing knowledge base cheap. Hash each file, store the hash in a document_metadata table, and on the next run compare. Unchanged files are skipped, changed files get their old vectors deleted and re-inserted. Without this, every run re-embeds everything.

Step 3: Swap the Retrieval Tool

Back in the chatbot workflow from part one, delete the lookup_document tool and add a Supabase Vector Store node in retrieve-as-tool mode. Give it the same documents table, the match_documents query name, and its own Embeddings Ollama sub-node, again nomic-embed-text, so the query is embedded the same way the documents were.

Name the tool something the agent will understand, like search_knowledge, and set a topK of 4 or 5 to start. That is the number of chunks the search returns per question. The system prompt gets simpler than the markdown version, because the agent no longer needs a list of file names:

You answer questions about MySite from a knowledge base.
Use search_knowledge to find relevant information before answering.
Answer only from what the search returns. If it returns nothing useful,
say you do not have that information. Never invent product details.

The chat model, the memory with Session ID set to From input, and the Chat Trigger are untouched. Reactivate the workflow. The website widget from part one keeps talking to the same chat URL, and now every answer is backed by semantic search.

Why the Numbers Matter

Three settings shape answer quality, and all three are worth a moment.

If answers feel thin, raise topK before you touch chunk size. If answers wander, lower it. Retrieval tuning is empirical, so change one number at a time and re-test the same handful of questions.

Local or Cloud

Everything above runs locally on Ollama, which keeps your documents and your queries on your own hardware and costs nothing per call. If you would rather trade privacy for a bit more accuracy, swap the two Ollama sub-nodes for OpenAI equivalents: an OpenAI Embeddings node (remember to change the table to vector(1536) to match) and an OpenAI Chat Model. The rest of the workflow, and the entire website embed, stay exactly the same. That is the point of building it this way: the pieces are independent, so you can upgrade one without rebuilding the others.

Where You Land

You now have a chatbot on your website that searches a real vector database by meaning, dedupes its own ingestion, and runs on hardware you control. It scales to thousands of documents, answers from specific paragraphs rather than whole files, and shares its plumbing with anything else in your n8n instance that reads the same documents table.

The natural next step is to let other systems reach this knowledge base, not just the website widget. That is exactly what turning the workflow into an MCP tool does, so an AI agent elsewhere can query your knowledge base directly.

Sources and Further Reading

• Supabase vector columns and pgvector
• n8n Supabase Vector Store node
• nomic-embed-text on Ollama
• local-ai-packaged on GitHub

Back to the start of the thread: the simple markdown chatbot, if you skipped straight here. Build that first, then come back and swap in the vector tool. It is a fifteen minute change once the ingestion loop is running.

This is the first of a two part thread in the n8n Automation Stack series. Here we build the simple version: markdown files as the knowledge base, a chat widget on your site. The follow up swaps the markdown lookup for a Supabase vector database when your knowledge outgrows a handful of files. If you are new to n8n, start with what n8n is and how to install it.

What You Are Building

Three pieces, wired together in one n8n workflow:

No embeddings, no database migrations. The agent does keyword-level reasoning over whole documents, which is plenty when you have a dozen or two well-named files.

Prerequisites

You need a running n8n with a local model runtime. The easiest way to get both, plus everything the follow up article needs, is the bundled stack we covered in our local-ai-packaged guide. It brings up n8n, Ollama, and Postgres together in one command. Pull a small instruct model once it is running:

ollama pull qwen2.5:7b-instruct-q4_K_M

That is the model doing the talking in this build. Anything in the 7B to 14B instruct range works; smaller is faster, larger follows instructions more reliably.

Step 1: The Knowledge Base Is Just Files

Create a folder that n8n can read and drop one markdown file per topic into it. n8n containers usually mount a shared directory at /data/shared, so a clean layout looks like this:

/data/shared/mysite/docs/
  getting-started.md
  pricing.md
  returns-policy.md
  api-authentication.md

The file name is the lookup key, so name files after the questions people ask. There is no ingestion step and nothing to rebuild. Edit a file, save it, and the next answer uses the new text. That immediacy is the whole appeal of this approach.

Step 2: A Sub-Workflow That Reads One Document

The agent should not touch the filesystem directly. Instead, build a small second workflow that takes a document name and returns its text, then hand that workflow to the agent as a tool. Keeping it separate means the agent can only read files through this one narrow door, and you can test the door on its own.

The sub-workflow is three nodes behind an Execute Workflow Trigger whose input is a single field, document:

{
  "nodes": [
    { "name": "When Called", "type": "n8n-nodes-base.executeWorkflowTrigger" },
    { "name": "Build File Path", "type": "n8n-nodes-base.set" },
    { "name": "Read Document", "type": "n8n-nodes-base.readWriteFile" },
    { "name": "Extract Text", "type": "n8n-nodes-base.extractFromFile" }
  ]
}

The Set node turns the document name into a safe path. Sanitising the name matters: it stops the agent from walking out of your docs folder with a crafted value. Use an expression on a single string field named filePath:

=/data/shared/mysite/docs/{{ $json.document.toLowerCase().replace(/[^a-z0-9_-]/g, '-') }}.md

Read Document points at {{ $json.filePath }}, and Extract Text runs in text mode so the output is plain markdown the model can read. That is the entire tool.

Step 3: The Agent

Now the main workflow. Drop in an AI Agent node and give it three attachments: a chat model, a memory, and the tool you just built.

• Chat model: an Ollama Chat Model node pointed at qwen2.5:7b-instruct-q4_K_M.
• Memory: a Simple Memory (or Postgres Chat Memory) node with Session ID set to From input, so each website visitor keeps their own conversation.
• Tool: a Call n8n Workflow Tool node pointing at the sub-workflow from step 2. Name it lookup_document and describe it clearly.

The description is what the agent reads to decide when to call the tool, so it earns its keep. A system prompt on the agent ties it together:

You answer questions about MySite using a document library.
Available documents: getting-started, pricing, returns-policy, api-authentication.
To answer, call lookup_document with the single most relevant document name,
then answer only from what it returns. If no document fits, say so plainly.
Never invent product details.

Listing the available document names in the prompt is the trick that makes file-based lookup work. The model maps the question to a file, calls the tool, and answers from the returned text.

Step 4: The Front Door

Replace any manual trigger with a Chat Trigger node (When chat message received). This node is what the website widget talks to. Two settings matter:

Leave the response mode at the default so the Chat Trigger streams the agent's answer straight back to the widget. Turn off Append n8n Attribution in the node options for a clean embed. Activate the workflow, then open the node and copy its chat URL. It has this shape:

https://your-n8n-host/webhook/<chat-webhook-id>/chat

Test it from n8n's built-in chat panel before touching your website. Ask something that should hit a document and confirm the agent calls lookup_document and answers from the file.

Step 5: Put the Chat on Your Website

n8n publishes an official chat widget, @n8n/chat, under an MIT license. You load it from a CDN and initialise it with one function call. Add this to any page:

<link href="https://cdn.jsdelivr.net/npm/@n8n/chat/dist/style.css" rel="stylesheet">
<script type="module">
  import { createChat } from 'https://cdn.jsdelivr.net/npm/@n8n/chat/dist/chat.bundle.es.js';

  createChat({
    webhookUrl: 'https://your-n8n-host/webhook/<chat-webhook-id>/chat',
    mode: 'window',
    chatInputKey: 'chatInput',
    chatSessionKey: 'sessionId',
    showWelcomeScreen: false,
    initialMessages: ['Hi! Ask me anything about MySite.'],
    i18n: {
      en: {
        title: 'Ask us',
        subtitle: '',
        inputPlaceholder: 'Type your question...',
        getStarted: 'New conversation'
      }
    }
  });
</script>

That is the exact shape a real deployment uses. A few options are worth understanding:

If the widget loads but every message fails, the cause is almost always CORS. The origin you are embedding on must be listed in the Chat Trigger's Allowed Origins. That single field is the most common reason a working chat looks broken on the live site.

Where This Runs Out of Road

File-based lookup is honest about what it is. The agent matches the question to a file name and reads the whole file, so it works best when documents are short, topics are cleanly separated, and there are not too many of them. Past roughly a few dozen files, listing every name in the system prompt stops being practical and the model starts guessing the wrong file.

It also cannot answer across documents or find a paragraph buried in a long one. There is no semantic search, only whole-file retrieval. When you hit that wall, you do not throw this away: you keep the same agent, the same Chat Trigger, and the same website embed, and you swap the file-reading tool for a vector search tool. That is exactly what part two does with Supabase and embeddings.

Sources and Further Reading

• @n8n/chat widget on npm
• n8n Chat Trigger node documentation
• n8n AI Agent and tools tutorial
• local-ai-packaged on GitHub

Next in the series: scaling this chatbot with a Supabase vector database, so it can search thousands of documents by meaning instead of matching file names. The website widget does not change. Only the tool behind it does.

This is part two of the n8n Automation Stack series. Part one covered what n8n is and the four concepts behind it. Here we get it installed properly, including the environment variables that decide whether your webhooks work behind a reverse proxy. Part three depends on getting that last part right.

The Five Paths Compared

Path 1: Docker With a Volume

This is the right default for a single user. The only change from the throwaway command in part one is a named volume and dropping --rm:

docker volume create n8n_data

docker run -d --name n8n \
  -p 5678:5678 \
  -v n8n_data:/home/node/.n8n \
  docker.n8n.io/n8nio/n8n

Everything lives in that volume: the SQLite database, your credentials, and the encryption key. Back up the volume and you have backed up the instance.

Path 2: Docker Compose With Postgres

n8n defaults to SQLite. It supports PostgreSQL as the alternative, and MySQL and MariaDB were deprecated back in v1.0, so Postgres is the only real choice once you outgrow a single file.

Move to Postgres when you want concurrent writes, a backup story that is not a file copy, or queue mode later.

services:
  postgres:
    image: postgres:16
    restart: unless-stopped
    environment:
      POSTGRES_DB: n8n
      POSTGRES_USER: n8n
      POSTGRES_PASSWORD: change_me
    volumes:
      - pg_data:/var/lib/postgresql/data

  n8n:
    image: docker.n8n.io/n8nio/n8n
    restart: unless-stopped
    ports:
      - "5678:5678"
    environment:
      DB_TYPE: postgresdb
      DB_POSTGRESDB_DATABASE: n8n
      DB_POSTGRESDB_HOST: postgres
      DB_POSTGRESDB_USER: n8n
      DB_POSTGRESDB_PASSWORD: change_me
      N8N_ENCRYPTION_KEY: put_a_long_random_string_here
    volumes:
      - n8n_data:/home/node/.n8n
    depends_on:
      - postgres

volumes:
  pg_data:
  n8n_data:

Bring it up with docker compose up -d. One refinement worth knowing: n8n supports a _FILE suffix on individual variables, so DB_POSTGRESDB_PASSWORD_FILE can point at a file instead of putting the password in the Compose file.

Path 3: npm

Useful if you are writing custom nodes and want the code on your host rather than inside a container:

npm install n8n -g
n8n start

Data goes to ~/.n8n by default. You can move it with N8N_USER_FOLDER. You are responsible for the service manager, the Node version, and the upgrade path, which is why most people end up on Docker anyway.

Back Up the Encryption Key Before You Do Anything Else

This is the single highest-consequence detail on this page.

n8n encrypts stored credentials with a key. If you do not set N8N_ENCRYPTION_KEY, n8n generates a random one on first launch and writes it into the user folder. Restore your database onto a fresh instance without that key and every stored credential is unreadable. The workflows come back, the connections do not.

Set it explicitly from the start and store it wherever you keep secrets. Note also that key rotation exists but is gated behind N8N_ENV_FEAT_ENCRYPTION_KEY_ROTATION, and the docs describe it as a one-way change that wants a full database backup first. Getting it right on day one is much cheaper.

The Variables That Decide Whether Webhooks Work

Run n8n on localhost and everything works. Put it behind nginx, Caddy, or Apache with TLS, and webhooks start handing out URLs nobody can reach. The cause is always the same: n8n does not know its own public address.

For an instance served at https://automation.example.com, the working set looks like this:

N8N_HOST=automation.example.com
N8N_PROTOCOL=https
N8N_PORT=5678
WEBHOOK_URL=https://automation.example.com/
N8N_EDITOR_BASE_URL=https://automation.example.com/

TLS terminates at the proxy. n8n keeps speaking plain HTTP on 5678 behind it, and N8N_PROTOCOL=https only tells n8n what to advertise in the URLs it generates.

One more proxy setting matters, and it is the one that breaks part three of this series: response buffering must be off. n8n's MCP server transport uses Server-Sent Events, and a buffering proxy will hold the stream until it times out. In nginx that is proxy_buffering off; on the relevant location block.

Path 4: local-ai-packaged

If you want n8n and a local model runtime together, local-ai-packaged bundles n8n, Ollama, Postgres, Qdrant, and a few other services behind a single command. You get a working local RAG setup without wiring five containers yourself.

The trade-off is that you inherit the whole stack, including parts you may not use, and its upgrade cadence rather than n8n's. We walked through the full install, the hardware acceleration profiles, and a first local RAG agent in our local-ai-packaged guide.

Path 5: ODS

ODS takes the same idea further and installs a private AI server in one command, with hardware auto-detection choosing a model tier for you. n8n is one of the services it brings up, alongside llama-server, Open WebUI, Whisper, Qdrant, and SearXNG.

Pick this when the goal is a private AI server that happens to include automation, rather than an automation server that happens to include AI. Details, including what the installer actually does, are in our ODS write-up.

Lock It Down Before You Expose It

An n8n instance holds credentials for every service it touches. Treat it as one of the more sensitive boxes you run.

Do not publish port 5678 to the internet directly. Bind it to localhost or a private network and let the reverse proxy be the only public listener. In Compose that means 127.0.0.1:5678:5678 instead of 5678:5678.

Three settings are worth knowing about:

That second one deserves emphasis. With the default setting, anyone who can edit a workflow can read your environment variables, which is where your database password lives.

Upgrading and Rolling Back

On Docker, an upgrade is a pull and a recreate:

docker compose pull
docker compose up -d

Two habits make this safe. Pin a specific image tag rather than tracking latest, so an upgrade is a deliberate act you can reverse by editing one line. And snapshot the database before a major version jump, because n8n runs schema migrations on start and those do not roll back cleanly.

Which Path Should You Pick

Running it for yourself on a homelab box: Docker with a volume. Anything other people depend on: Compose with Postgres, an explicit encryption key, and pinned tags. Writing custom nodes: npm. Already planning to run local models: one of the bundled stacks, which saves you a day of container wiring.

Whichever you choose, set N8N_ENCRYPTION_KEY now and put it somewhere you will still be able to find it in a year.

Sources and Further Reading

• n8n self-hosting documentation
• n8n environment variable reference
• n8n database configuration
• local-ai-packaged on GitHub

Next in the series: turning one of these workflows into an MCP tool an AI agent can call directly. If your instance sits behind a proxy, check proxy_buffering before you start.

This is the first article in our n8n Automation Stack series. It covers what n8n is, the four concepts you need before anything else makes sense, what changed in 2026, and when reaching for n8n is the wrong call. No workflow building yet. That starts in the next two parts.

What n8n Actually Is, and What It Is Not

n8n is a node-based automation engine with several hundred built-in integrations. You self-host it, your credentials stay in your database, and every workflow is JSON you can export and commit to git. That last property is the one that matters most for developers, and it is the main thing hosted automation tools do not give you.

Here is the part most introductions skip: n8n is not open source. It is source-available under the Sustainable Use License v1.0. You can read the code, modify it, and self-host it, but the licence sets explicit boundaries:

You may use or modify the software only for your own internal business purposes or for non-commercial or personal use. You may distribute the software or provide it to others only if you do so free of charge for non-commercial purposes.

In practice that means running n8n for your own company or your own projects is fine. Wrapping it up and reselling it as a hosted service is not. Separately, any source file with .ee. in its filename or .ee in its directory name sits outside that licence entirely and needs a commercial Enterprise licence.

We cover open-weights models and genuinely open tooling here, so it is worth being precise rather than calling everything self-hostable "open source". n8n is fair-code. For most readers that distinction changes nothing. If you are building a product on top of it, read the licence before you write the business plan.

The Four Concepts That Make n8n Click

Almost every point of confusion for new users comes from missing one of these four ideas.

Nodes

A node is one step. It might call an HTTP endpoint, run a snippet of JavaScript or Python, branch on a condition, or talk to Postgres. Data flows between nodes as an array of JSON items, and this is the single most important thing to internalise: a node does not receive one object, it receives a list. If your workflow mysteriously runs five times, it is because the node upstream handed it five items.

Triggers

A trigger is how a workflow starts. The common ones are schedule (cron), webhook (an inbound HTTP request), and manual execution from the editor. A workflow has exactly one active trigger path. Getting the trigger right is most of the design work.

Credentials

Credentials are stored separately from workflows and referenced by name. This is why you can export a workflow to JSON and share it without leaking your API keys. It also means the encryption key that protects those credentials is the single most important thing to back up. Lose it and every stored credential becomes unreadable, even with an intact database. We come back to this in part two.

Executions

Every run is recorded with its input and output at each node. When something breaks at 3am, the execution log tells you which node failed and exactly what data it received. This is the feature that makes n8n worth using over a shell script, and it is the one people discover last.

Test Runs and Production Runs Are Not the Same Thing

n8n distinguishes between executing a workflow from the editor and running a published one. They use different URLs, and this trips up almost everyone the first time they build a webhook.

A test execution shows live data in the editor, which is what you want while building. A production execution does not display in the editor at all. You inspect it afterwards through the Executions tab. If you have ever pointed an external service at an n8n webhook and watched nothing happen, you were almost certainly still on the test URL, which only listens for a single request.

Remember this one. It becomes load-bearing in part three, when we expose a workflow as a tool over the network.

What Changed in n8n During 2026

If your mental model of n8n is from 2024, several things are different.

• The AI Agent node was rebuilt with tool calling across Claude, GPT-4o, Gemini, Mistral, Groq, and any OpenAI-compatible endpoint. That last one is what lets you point it at a local model.
• Four memory node types shipped: in-memory, Redis, Postgres, and Motorhead. Conversation state no longer has to be something you hand-roll.
• The Canvas UI arrived in version 1.30 and replaced the flat left-to-right builder. Nodes can be grouped into labelled clusters and collapsed, which matters once a workflow passes about 20 nodes.
• Per-node retry with exponential backoff is now built in, instead of being something you construct out of error branches.
• Execution replay landed in June 2026 and lets you trace variables line by line through JavaScript and Python nodes on a failed run.
• MCP support means n8n can act as a Model Context Protocol server or client. This is the subject of part three.

When n8n Is the Right Tool

It is not always. Being honest about this saves you a rewrite later.

The honest failure mode of n8n is complex branching business logic. Once a workflow has 40 nodes and six conditional paths, it is harder to read than the equivalent 100 lines of code, and much harder to diff in review. Use it as glue between systems, not as a programming language.

What Self-Hosting Actually Costs

n8n itself is free to self-host under the terms above. The community edition has no execution limits, so the cost is your hardware and your time.

A small instance handling a few thousand executions a month runs comfortably on 1 vCPU and 1 GB of RAM. Add Postgres instead of the default SQLite once you care about concurrent writes or want backups that are not a file copy. If you plan to run local models alongside it, the memory budget is dominated by the model, not by n8n. Our full local AI stack guide covers those numbers in detail.

Your First Ten Minutes

The fastest way to get a feel for the editor is a throwaway container. This is not how you should run it permanently, and part two explains why, but it is enough to click around:

docker run -it --rm --name n8n -p 5678:5678 docker.n8n.io/n8nio/n8n

Open http://localhost:5678, create the owner account, and build the smallest possible workflow: a Schedule Trigger into a Code node that returns { hello: "world" }. Execute it, then open the Executions tab and look at the recorded input and output. That loop of build, run, inspect is the whole tool in miniature.

Note the --rm flag. Everything you build in that container disappears when you stop it, credentials included. That is deliberate for a first look and unacceptable for anything else.

What Comes Next in This Series

Part two covers setting n8n up properly: the barebones Docker and npm paths, the environment variables that decide whether your webhooks work behind a reverse proxy, and the two bundled harnesses that install n8n alongside a local model stack. Part three turns a workflow into an MCP tool that an AI agent can call directly.

If you would rather jump straight to building something, our AI generator tutorial wires n8n to Ollama and the Claude API end to end.

Sources and Further Reading

• n8n documentation
• n8n Sustainable Use License
• n8n self-hosting documentation
• n8n release notes

This is part three of the n8n Automation Stack series. Part one covered the core concepts, part two covered installing n8n properly. That second one matters here more than usual, because the MCP transport is the one thing a default reverse proxy configuration will silently break.

What You Get

• One n8n workflow exposed as an MCP tool with a typed input schema.
• Bearer-authenticated access from Claude Desktop, Cursor, or any MCP client.
• The four self-hosting failures that stop it working, and the fix for each.
• A tested look at the second, instance-level MCP server: what it exposes, what it costs to run, and the security boundary you are trusting.

Two Directions, Do Not Mix Them Up

n8n sits on both sides of MCP, and the node names are easy to confuse.

This article builds the first one. The second gets a paragraph at the end.

Prerequisites

A self-hosted n8n instance you can reach over HTTPS, an MCP client, and a workflow worth exposing. Pick something with a clear input and a clear output. A lookup against a database, a status check against an internal API, or a search over your own content all work well. Anything that takes ten parameters and returns prose does not.

Step 1: Add the MCP Server Trigger

Create a new workflow and add the MCP Server Trigger node. It replaces the usual trigger, so this workflow will not have a Schedule or Webhook node.

The node gives you a path, which becomes part of the URL agents connect to, and an authentication choice. Two options are available, both configured through HTTP request credentials:

• Bearer auth: a token in the Authorization header. Use this unless you have a reason not to.
• Header auth: a custom header name and value, for fitting an existing scheme.

Set a path you will recognise later. The node then shows you two URLs, and the difference between them is the same test versus production split from part one.

Step 2: Attach a Tool

The trigger on its own exposes nothing. Tools attach to it as sub-nodes, and two kinds can connect:

• Custom n8n Workflow Tool, which exposes another of your workflows as a callable tool.
• MCP Client Tool, which re-exposes tools from another MCP server.

Use the first. Attach a Custom n8n Workflow Tool, point it at the workflow doing the actual work, and then spend your time on the two fields that decide whether an agent uses your tool correctly: the name and the description.

The description is not documentation. It is the prompt the model reads when deciding whether to call your tool. Write it for the model:

{
  "name": "get_order_status",
  "description": "Look up the current status of a customer order by its ID. Returns status, carrier and tracking number. Use when the user asks where an order is.",
  "parameters": {
    "order_id": "The order ID, format ORD-12345"
  }
}

Vague descriptions produce agents that either never call the tool or call it constantly with nonsense arguments. Say what it returns and when to use it.

Step 3: Test Before You Publish

Click Listen for Test Event and the node hands you a test URL. Point your MCP client at it and call the tool once.

The test URL displays live data in the editor, which is the whole reason to use it. You can see the arguments the model sent and the data each node returned. Note that it listens for a single event, so if a second call appears to hang, that is why.

Step 4: Publish and Connect for Real

Publishing the workflow activates the production URL. Production executions do not display in the editor at all. To inspect them you open the Executions tab, which is where all your debugging happens from this point on.

Point your MCP client at the production URL with the same credentials. In Claude Desktop that is an entry in the MCP servers section of the config; in Cursor and other clients it is equivalent. The tool should appear in the client's tool list with the name and description you set in step 2.

Step 5: The Self-Hosting Reality Check

This is the part that is not in most walkthroughs, and it is where a self-hosted instance behind a proxy stops working.

There is no stdio transport

Many MCP servers run as a local subprocess and speak over standard input and output. The n8n MCP Server Trigger does not support stdio. It is SSE and HTTP only, meaning your n8n instance has to be reachable over the network from wherever the client runs. If your client only supports stdio, you need a bridge process, not this node.

Proxy buffering breaks it

Server-Sent Events depend on the server pushing bytes as they happen. A reverse proxy that buffers responses will hold those bytes until the buffer fills or the connection times out, and the symptom is a connection that appears to open and then never delivers anything. The n8n documentation is explicit that a reverse proxy in front of the MCP endpoint must have buffering disabled and compression settings adjusted.

In nginx that is proxy_buffering off; on the location handling the MCP path. In Apache, SetEnv proxy-sendchunked 1 and no output filters on that path. This is the same setting part two flagged, and it is the first thing to check when a connection opens but no tools appear.

Multiple replicas break it differently

If you run n8n with more than one webhook replica behind a load balancer, SSE connections fail when a follow-up request lands on a different replica than the one holding the stream. The fix is routing: send every request matching /mcp* to one dedicated replica. Single-container deployments are unaffected, which is most self-hosted setups.

Fire tool calls one at a time

We connected a real MCP client to a self-hosted instance and the first thing that broke was concurrency. Sending several tool calls in one parallel batch returned transport errors, some coming back as a bare "not connected" even though the server was up. Running the same calls one after another worked every time.

This is the SSE transport doing exactly what the buffering and replica warnings above predict: a single streamed connection does not want several requests racing through it at once. If your client or your agent framework likes to fan out tool calls in parallel, throttle it to one in flight. A call that fails this way is worth a single sequential retry before you assume anything is actually wrong with the server.

The Other Direction: Consuming MCP Servers

The MCP Client Tool node is the mirror image. Attach it to an n8n AI Agent and the agent gains the tools of an external MCP server. It connects over an SSE endpoint, and it supports bearer, single or multiple custom headers, OAuth2, or no authentication at all.

The useful detail is tool filtering. You can expose All tools from the server, a Selected subset, or All Except a blocklist. Handing a model 40 tools when it needs 3 measurably degrades tool choice, so select deliberately.

A Third Thing, Which Is Not This

n8n also has an instance-level MCP server, a different feature with a confusingly similar name. It lets an MCP client connect to n8n itself at /mcp-server/http and build, search, test, and publish workflows by prompting. Authentication is OAuth2 or a personal access token. We wired one into a client and used it, so the rest of this section is from that rather than from the docs.

This is a development tool, not a runtime one. It is how you build workflows by talking to a model, not how an agent calls your workflow in production.

What it actually exposes

Connecting turns n8n into a toolbox with four kinds of tool, and the difference between them is entirely about blast radius:

That last row is the one to respect. An execute call is not a dry run. If the workflow it triggers calls a paid model API, hits a database, or posts to a webhook, it does all of that for real. Point an agent at this server and "test my workflow" can mean spending money and sending outbound requests.

Two things it does right

Listing credentials returns names and types only, never secret values. You can let a model see that an OpenAI credential and a Postgres credential exist, and reference them by ID when building, without ever exposing the keys. That is the correct design and it held up in practice.

It also refuses to let you guess. The server makes you pull the SDK reference and the best-practices guidance for a technique before it will accept workflow code, so a model cannot hallucinate node parameters straight into your instance. Building is slower as a result, and correct more often.

The opt-in is real, and stricter than you expect

Workflows are exposed individually. On the instance we connected to, a small minority of the total were flagged available over MCP; the rest were invisible to the connected client even though the same token could list them through the management tools. The exposure is also not client-scoped: every client using that token sees the same enabled set. Treat "enabled for MCP" as a deliberate per-workflow decision, not a default.

The token is the whole security boundary

A personal access token here is a long-lived bearer credential. The one we generated was a JWT with no expiry claim, so it does not time out on its own, and it grants read, build, write, and execute across everything the issuing user can reach. It sits in your client's config file in plain text.

Which means: store it like a password, do not commit the config that holds it, and rotate it in n8n's settings if it ever leaks. Anyone who reads that file can drive your instance.

One more caveat. Several third-party guides cite an enablement environment variable and a minimum version that do not match the official documentation, which describes the module as enabled by default and disabled with N8N_DISABLED_MODULES=mcp. Check your own instance before copying either claim.

What This Is Actually Good For

Two directions, two audiences.

The Server Trigger from steps one through five is for exposing what you already built. If you run n8n and have workflows an agent should be able to trigger, an internal lookup, a status check, a search over your own content, wrapping them as tools is a thin layer over work that already exists.

The instance-level server is the inversion: it is for building and auditing n8n from a chat window. Asking a model which workflows are active, which have not run since last quarter, or which are exposed over MCP is a fast way to take stock of an instance that has grown past what you remember. Building a new workflow through it means the model discovers nodes and validates them against your real credentials before anything is saved. It is the most useful when your instance is large enough that clicking through the editor has become the slow path.

Limitations and Gotchas

• No stdio transport. The instance must be network-reachable from the client.
• SSE needs an unbuffered path end to end, including any CDN in front.
• Concurrent tool calls over the SSE transport fail. Send one at a time.
• Production executions are invisible in the editor. Use the Executions tab.
• The tool description is prompt text and drives model behaviour. Treat it as code.
• An execute call on the instance-level server runs the real workflow with real credentials. It is not a dry run.
• The bearer token is the whole security boundary, is long-lived, and sits in a config file in plain text. Anyone who reads it can drive your instance.

When Not To Bother

Skip the Server Trigger if you only need an agent to call one HTTP endpoint. An MCP server wrapping a single API call adds a hop and a service to maintain. Point the agent at the API.

Skip the instance-level server on a small instance you already know by heart. Its payoff scales with how many workflows you have forgotten about, and on a handful of workflows the editor is faster than a chat window.

Sources and Further Reading

• n8n MCP Server Trigger node docs
• n8n MCP Client Tool node docs
• Connect to the n8n MCP server
• Model Context Protocol specification

Ten minute version: add an MCP Server Trigger to any existing workflow, set bearer auth, attach a Custom n8n Workflow Tool, and call the test URL from your client. If nothing arrives, check proxy_buffering first.

First, the name confusion

If you went looking for this project after seeing it in a July roundup, you were probably sent to a repo called DreamServer under an org called Light-Heart-Labs. That link still works, but it is a redirect. The project renamed itself: github.com/Light-Heart-Labs/DreamServer now returns an HTTP 301 to github.com/Osmantic/ODS, which stands for Osmantic Deployment System. Several aggregators have not caught up. If a tutorial tells you to clone DreamServer, it is stale but harmless.

What you get

ODS is a Docker Compose stack with a CLI wrapped around it. The pitch is turning a PC, Mac or Linux box into a private AI server, and the component list is genuinely broad:

Most of these we have covered individually. OpenCode is in there as the coding assistant, and the inference layer is the same llama.cpp lineage that sits under Ollama. The value here is not any single component, it is that they come pre-wired with a gateway in front and one config file behind.

The part that matters: what the installer does

The documented install is a pipe from the internet into your shell, which should always make you pause:

curl -fsSL https://install.osmantic.com/ods.sh | bash

So we downloaded it and read it instead of running it. It is 507 lines of Bash. Here is the honest audit.

Network calls. The only external hosts in the entire script are github.com (to clone the repo), install.osmantic.com (itself), and git-scm.com in a help message. There is no telemetry. We grepped for the usual suspects, analytics endpoints, PostHog, Segment, and any stray curl to a reporting URL. Nothing.

Privilege. It runs as your user. sudo appears in exactly two situations: installing git if you do not have it, and force-removing root-owned container data during cleanup. It does not ask for root to run the stack.

Where it lands. The install directory and repo URL are both environment variables, so you can redirect it before it touches anything:

# Read it first, then run it deliberately
curl -fsSL https://install.osmantic.com/ods.sh -o ods.sh
less ods.sh

# Point it wherever you want
ODS_INSTALL_DIR="$HOME/ai/ods" bash ods.sh

That is a clean bill of health for a curl-pipe-bash installer, which is not something we get to write often. It still deserves the same treatment as any other: download, read, then run.

Hardware tiers

ODS detects your accelerator and picks a default model rather than making you guess a quantization. The mapping, from the project's own table:

Tiers as documented by the project. We did not verify model selection on each backend.

Bootstrap mode is the nice touch: it starts you on a small model so the chat UI answers within a couple of minutes, then pulls the full tier model in the background.

Driving it

The ods CLI is the reason this beats a hand-rolled compose file. Services are toggles, not YAML edits:

ods status              # health checks and GPU state
ods list                # every service and whether it is up
ods enable n8n          # turn a service on
ods disable whisper     # turn one off
ods logs llm            # tail a service
ods model swap T3       # jump to another hardware tier
ods mode local          # local, cloud or hybrid inference
ods preset save         # snapshot the whole config

Open WebUI lands on localhost:3000. The llama-server API is on 11434 under Docker on Linux, or 8080 when it runs natively on macOS and Windows, and every port is overridable through the environment.

Local is the default mode. Cloud providers are opt-in, either with --cloud at install time or ods mode later, and the project is explicit that your prompts stay on your machine unless you choose otherwise. Given how the local-first label has been stretched lately, including by Ollama shipping a cloud-hosted default agent in v0.32, an installer that ships local-only and makes you ask for the cloud is worth noting.

Requirements and gotchas

• Docker is mandatory everywhere. On Windows that means Docker Desktop with the WSL2 backend; on macOS, Docker Desktop plus Apple Silicon, since Intel Macs are not supported.
• On macOS, llama-server runs natively for Metal while everything else runs in containers, so it is a hybrid deployment rather than a pure container stack.
• The tier selector deliberately avoids Qwen3 Coder Next on unified-memory hosts, citing correctness issues on those backends.
• 187 open issues against 3,446 stars. Normal for a young project moving fast, but this is not a set-and-forget appliance yet.
• The last tagged release is v2.5.3 from May 26, while commits landed as recently as the day we published. Running from main gets you fixes the tag does not have, with the usual tradeoff.
• Disk usage is not documented, and it will be substantial: a dozen container images plus a quantized model in the tens of gigabytes.
• No GPU means CPU fallback, which works but is slow enough that the project recommends cloud mode instead.

Who should run it

Run ODS if you want the whole private-AI stack rather than one piece of it, and you would rather spend the evening using it than wiring it. It suits a homelab box, a spare workstation with a decent GPU, or an Apple Silicon machine with memory to spare. The service toggles make it a reasonable way to try Perplexica or Qdrant without committing to another compose file you will maintain forever.

Skip it if you only need inference, where plain llama.cpp or Ollama is less machinery for the same result, or if you are on hardware without an accelerator, where the whole point evaporates. And if you are on a shared or production server, read those 507 lines yourself before running them. We found nothing alarming, but "some blog checked it" is not an audit.

Sources and further reading

• GitHub: Osmantic/ODS (formerly DreamServer)
• The install script, in full
• ODS releases (latest tag v2.5.3)
• Open WebUI documentation
• llama.cpp, the inference engine underneath

Tested on: not deployed. We statically audited the 507-line installer (network calls, privilege escalation, telemetry, install paths) on Linux 6.1, and verified the repository metadata, redirect and release history against the GitHub API on 2026-07-22. We did not run the full stack: this bench has 8 GB of RAM and no GPU, which is below the lowest documented tier, so any performance numbers from it would have been misleading. Component lists, hardware tiers and port defaults are as documented by the project.
Date checked: 2026-07-22