# SingularityByte (full text)

> Full-text companion to /llms.txt. The most recent SingularityByte articles per section, with each body inlined as plain text so a model can read the actual content without fetching the HTML pages.

For the link map only (titles, URLs, descriptions), see [/llms.txt](https://singularitybyte.com/llms.txt). Coverage is open-weights-first for developers, indie hackers, AI engineers, researchers, and technical founders who run local models. This file is generated by the CMS from live content.

## Tutorials

# Open-Source Video Models Are 262 Elo Behind. One Weight Drop Closes It.
Source: https://singularitybyte.com/tutorials/open-weight-video-models-262-elo-gap.html

This is the fifth entry in our open-source AI video series. [Part four](/tutorials/open-source-video-models-mid-2026-what-changed.html) ended with a line we expected to sit on for months: if Alibaba opens Wan 2.7, that becomes the story. It has not happened yet.

Nine days later, two other labs made the same promise. Black Forest Labs announced FLUX 3 on July 23 with an open-weight backbone to follow. MiniMax announced H3 on July 31 and said the weights ship "in the coming days." Neither has published a file.

That sounds like more of the same, and in one sense it is. But the arithmetic underneath changed, and it is worth looking at directly, because the distance between open and closed video is now small enough to be measured in a single number.

## TL;DR

- The open-weights video gap is 262 Elo. The best downloadable text-to-video model with audio is LTX-2.3 Fast at 980 on the Artificial Analysis arena. MiniMax H3, announced last week, sits at 1242.

- Three labs are holding the trigger. FLUX 3 Dev, MiniMax H3, and Wan 3.0 are all announced as open-weight. As of today, none of the three exists on Hugging Face.

- One weight drop closes almost the entire gap. If MiniMax ships what it announced, open-weight video goes from "good enough to iterate with" to second place in the world overnight.

- The catch is the license, not the capability. Nobody at the frontier is shipping Apache 2.0 anymore. Revenue-capped community licenses are the new default.

## The 262-Elo gap in open-weight video models, measured

Artificial Analysis runs a blind arena for video: two clips from the same prompt, users vote, Elo comes out the other end. It has a filter for open weights, which makes it the cleanest available read on how far behind downloadable models are.

Comparing like for like on the with-audio board, because H3 and FLUX 3 both generate native audio:

Model | Elo | Weights
Gemini Omni Flash | 1245 | Closed
MiniMax H3 | 1242 | Announced as open, not shipped
Dreamina Seedance 2.0 720p | 1225 | Closed
Wan2.7-260612 | 1163 | API only
HappyHorse-1.1 | 1151 | Closed
Kling 3.0 1080p (Pro) | 1113 | Closed
SkyReels V4 | 1109 | Closed
LTX-2.3 Fast | 980 | Downloadable today
LTX-2.3 Pro | 963 | Downloadable today
LTX-2 Fast | 948 | Downloadable today
Two things stand out. The top ten is entirely closed or unshipped. And the entire open-weights leaderboard, every position of it, is Lightricks. There is no second open lab in the running right now.

1242 minus 980 is 262. That is the number this article is about. It is the amount of quality that would transfer from the closed column to the open column if exactly one company followed through on exactly one announcement.

For scale: 262 Elo in a preference arena is not a rounding error and not an unbridgeable chasm. It is roughly the distance between "usable for previz and social clips" and "usable for the shot you actually deliver."

## Three promises, zero files

Part four published a one-command check for whether a model is genuinely released. Here it is pointed at all three claims at once, run on the date below:

```

for q in Wan3 Wan2.7 MiniMax-H3 FLUX.3; do
echo -n "$q -> "
curl -s "https://huggingface.co/api/models?search=$q&limit=5" \
| python3 -c "import sys,json; d=json.load(sys.stdin); print(len(d), [m['modelId'] for m in d])"
done

```

Every one of them comes back empty, or with unrelated community repos that happen to match the string. Checking what the three labs have actually published, newest first:

Organization | Newest published model | Date
Wan-AI | Wan-Dancer-14B (plus the Wan 2.2 family) | 2026-07-17
MiniMaxAI | MiniMax-M3, a language model. No video model at all. | 2026-07-23
black-forest-labs | FLUX.2-small-decoder | 2026-04-07
Lightricks | LTX-2.3-22b-IC-LoRA-DubIt | 2026-07-30
That last row is the contrast worth sitting with. While three labs were announcing open weights, Lightricks pushed 17 repository updates in July alone, including eight new IC-LoRAs in the final four days of the month: dubbing, relighting, in/outpainting, spatial upscaling, deblur, decompression, water simulation, and one that shaves beards. Announcements are cheap. Adapters are not.

## What each drop would actually change

### MiniMax H3: the one that moves the number

H3 is a multimodal video model that takes text, images, video, and audio in one context and returns up to 15 seconds at 2K with native stereo sound. Artificial Analysis ranks it first in the world for video editing and second for text-to-video with audio.

MiniMax has committed to the MiniMax Community License: free for non-commercial use, free for commercial use by organizations under 20 million dollars in annual revenue, with attribution required. As of today there is no H3 repository on the MiniMaxAI org.

If it ships, this is the whole story. Everything else in this article is a footnote to it.

### FLUX 3: the one that changes the shape

FLUX 3 is Black Forest Labs' first video model, and it is not only a video model. It is jointly trained on image, video, audio, and action prediction in one architecture, which is why it also powers FLUX-mimic for robotics. Video output runs up to 20 seconds with native audio.

The open-weight piece is called FLUX 3 Dev, described in BFL's own announcement as "open-weight access to a multimodal backbone, for content creation (video, audio and image) and action prediction." What the announcement does not contain is a date or a license name.

Worth being precise about what this would mean if it lands. It would be the first downloadable model that generates video, audio, images, and robot actions from one set of weights. That is a different kind of release from H3, and for anyone building agents that need to both see and act, it is arguably the more interesting one. It is also the vaguest of the three promises.

### Wan 3.0: the one to discount

Wan 3.0 is pre-announced as an Apache 2.0 open-weight release for mid-2026. Treat that with more caution than the other two, for reasons that are entirely evidence-based.

We could not find a primary Alibaba source for Wan 3.0 at all. The specifications circulating contradict each other, with the same "confirmed" model described as 60B dense in one place and a 27B mixture-of-experts with 14B active in another. Those pages trace back to single-purpose domains built around the model name, the same pattern part four documented for Wan 2.7.

The track record does not help either. Wan 2.5 was pre-announced as open and never appeared. Wan 2.6 shipped closed. Wan 2.7 is on the arena at 1163 and remains API only. Official Wan open weights still stop at 2.2, where they have been since August 2025.

None of that makes Wan 3.0 fake. It makes it unverifiable, which for planning purposes is the same thing.

## The licenses are the real story

The reflex reading of an open-weights wave is that the field is getting more permissive. It is getting more available, which is not the same thing.

Model | License | Free commercial use up to
Wan 2.2 | Apache-2.0 | No limit
LTX-2.3 | LTX-2 Community License | 10M dollars ARR
MiniMax H3 (announced) | MiniMax Community License | 20M dollars revenue, attribution required
FLUX 3 Dev (announced) | Not yet named | Unknown
The only genuinely permissive entry is the oldest one. Every model that has landed or been announced since is a revenue-capped community license, and the caps come with details worth reading before you assume you are under them. The LTX threshold counts subsidiaries, affiliates, and companies under common control collectively, so a small studio inside a large group may already be over it.

For most people reading this, a 10 or 20 million dollar cap is functionally no cap. That is exactly why it works as a strategy, and exactly why it is worth naming: the industry has found a license that feels open to almost everyone who downloads it while staying commercially closed to everyone who matters to revenue.

## The lawsuit nobody is pricing in

One complication sits underneath the MiniMax promise specifically.

Disney, Marvel, Lucasfilm, Twentieth Century Fox, Universal, DreamWorks Animation, and Warner Bros. Discovery sued MiniMax on 2025-09-16 in the Central District of California, alleging Hailuo was trained on unauthorized copies of their work and reproduces recognizable characters on simple prompts. On 2026-05-26, Judge Stanley Blumenfeld denied MiniMax's motion to dismiss, finding the studios had plausibly alleged widespread intentional infringement. The case is live.

So the scenario on the table is a lab open-sourcing the weights of a video model whose predecessor is under active infringement litigation from most of Hollywood. We are not aware of a precedent for that, and the honest answer about downstream exposure for people who fine-tune and ship products on those weights is that nobody knows yet.

Factor it into planning, not into panic. It is a reason to read the license text when it lands rather than a reason to skip the download.

## What you can actually run today

One model in this entire article is downloadable, and it is the same one as last time. LTX-2.3: 22 billion parameters, synchronized audio and video from a single model, 2.13 million downloads on the base repository.

The shortest path from nothing to a clip:

```

pip install -U diffusers transformers accelerate

python3 - <<'PY'
import torch
from diffusers import DiffusionPipeline
from diffusers.utils import load_image, export_to_video

pipe = DiffusionPipeline.from_pretrained(
"Lightricks/LTX-2.3", torch_dtype=torch.bfloat16, device_map="cuda"
)
image = load_image("your-still.png")
out = pipe(image=image, prompt="A man with short gray hair plays a red electric guitar.").frames[0]
export_to_video(out, "output.mp4")
PY

```

Budget 24GB of VRAM with the int8 build for 1080p work. Community reports put a 10-second 1080p clip at roughly 4 to 6 minutes on an RTX 4090. Two constraints will bite you on the first run: width and height must be divisible by 32, and frame count must be divisible by 8 plus 1. Ask for 100 frames and you get an error rather than a video.

If you are on Apple Silicon, our [LTX-2 on Mac walkthrough](/tutorials/run-ltx-2-locally-mac-pinokio-phosphene.html) still applies, and [the hardware stack guide](/tutorials/self-hosted-ai-video-generator-stack-hardware-guide-2026.html) covers where a 22B model sits in memory terms.

## What to watch in the next 90 days

Three checks, in descending order of how much they matter:

- Does MiniMaxAI/MiniMax-H3 appear on Hugging Face? "Coming days" was July 31. If it is still empty at the end of August, the promise has entered Wan territory.

- Does FLUX 3 Dev get a date and a license name? The capability claim is the most interesting of the three. The commitment behind it is currently the thinnest.

- Does Lightricks stay alone at the top? If both promises slip, the open-weights leaderboard remains a single-vendor board for another quarter, and the 262 stays where it is.

The optimistic case here is real, and it is narrower than the coverage suggests. Open-weight video does not need three labs to follow through. It needs one. The gap has never been this close to closing on a single upload.

Set a reminder for two weeks out and run the check yourself. It takes ten seconds:

```

curl -s "https://huggingface.co/api/models?author=MiniMaxAI&sort=lastModified&direction=-1&limit=5" \
| python3 -c "import sys,json; [print(m['modelId'], str(m['lastModified'])[:10]) for m in json.load(sys.stdin)]"

```

## Sources and further reading

- [Part four: what actually shipped in mid-2026, and what did not](/tutorials/open-source-video-models-mid-2026-what-changed.html)

- [Part one: our tested open-source video model comparison](/tutorials/ai-video-generator-comparison-2026-open-source-models-tested.html)

- [Artificial Analysis open-weights text-to-video leaderboard](https://artificialanalysis.ai/video/leaderboard/text-to-video/open-weights)

- [Artificial Analysis text-to-video leaderboard (all models)](https://artificialanalysis.ai/video/leaderboard/text-to-video)

- [Black Forest Labs: FLUX 3 announcement](https://bfl.ai/blog/flux-3)

- [SCMP: MiniMax challenges ByteDance with open weights for H3](https://www.scmp.com/tech/article/3362540/video-ai-minimax-challenges-bytedance-low-price-open-weights-new-h3-model)

- [Disney Enterprises, Inc. v. MiniMax, 2:25-cv-08768 (C.D. Cal.)](https://www.courtlistener.com/docket/71357247/disney-enterprises-inc-v-minimax/)

- [Lightricks/LTX-2.3 on Hugging Face](https://huggingface.co/Lightricks/LTX-2.3)

- [The LTX-2 Community License Agreement](https://github.com/Lightricks/LTX-2/blob/main/LICENSE)

Tested on: not independently tested. We did not run LTX-2.3, H3, or FLUX 3. Every release claim above was verified directly against the public Hugging Face API on the date below, using the commands published in this article: model existence, organization publishing history, download counts, and last-modified dates. Elo figures are read from the Artificial Analysis video arena on the same date and are user-vote preference scores, not capability benchmarks. Capability descriptions, license terms, and release intentions are the projects' own. VRAM and generation-time figures for LTX-2.3 are community-reported.

Date checked: 2026-08-01

---


# Ollama vs llama.cpp vs vLLM vs LM Studio: Which Local AI Server Should You Run?
Source: https://singularitybyte.com/tutorials/ollama-vs-llama-cpp-vs-vllm-local-inference-server-2026.html

This is another entry in our Local AI Stack series. We have covered [running a full local stack](/tutorials/run-full-local-ai-stack-with-local-ai-packaged.html), [turning a PC into a private AI server](/tutorials/ods-private-ai-server-one-command.html), and plenty of models to run on them. What we have never answered is the question that comes first: which server actually runs the model.

Search for that and you get the same list every time. Ollama, llama.cpp, LM Studio, vLLM, SGLang, ranked by tokens per second as though they were five competitors doing the same job. They are not. Understanding why makes the choice obvious in about a minute.

## TL;DR

- There are two families, not five competitors. Ollama and LM Studio both embed llama.cpp, so they are close cousins. vLLM and SGLang are a different architecture entirely.

- Picking between Ollama and LM Studio is a UX decision. Picking between Ollama and vLLM is an architecture decision. Only one of those is worth agonising over.

- Everything here is MIT or Apache 2.0 except LM Studio, which is closed source but free, including at work.

You are | Run this | Why
One developer, any OS, want it working now | Ollama | One command to pull and run. Sane defaults.
One person who wants a GUI and a model browser | LM Studio or Jan | LM Studio is more polished, Jan is open source.
Serving concurrent users in production | vLLM | Continuous batching. This is the actual dividing line.
Serving RAG or heavy shared-prefix traffic | SGLang | Optimised for reused prompt prefixes.
Odd hardware, embedded, or CPU-only | llama.cpp | Runs where nothing else will.
On Apple Silicon | MLX, or LM Studio | Built for the unified memory architecture.
## The split that explains everything

Family one is built on llama.cpp. llama.cpp is the inference engine: a C++ implementation that runs GGUF-quantised models on almost anything, including CPUs and Apple Silicon. Ollama wraps it in a daemon with a model registry. LM Studio wraps it in a desktop GUI.

This is why throughput comparisons between Ollama and LM Studio are close to meaningless. Community benchmarks put them within roughly five percent of each other, and that is exactly what you would expect from two programs running the same engine underneath. The differences that matter are ergonomics and how quickly each one ships upstream llama.cpp improvements, not speed.

Family two is built for concurrency. vLLM and SGLang use continuous batching and PagedAttention. Instead of processing requests one at a time, they keep the GPU saturated by continuously slotting new requests into the batch as older ones finish, and they manage the KV cache in pages rather than contiguous blocks, which cuts the memory waste that otherwise limits how many requests fit at once.

Ollama and LM Studio do not do this. For a single user that costs you nothing, because there is only one request in flight. Under concurrent load it is the whole ballgame. Community benchmarks report vLLM at more than an order of magnitude above Ollama's concurrent throughput, and while the exact multiple depends entirely on your workload, the mechanism is not in dispute.

So the honest framing is: if you are one person on one machine, benchmark differences between these tools are noise. If you are serving multiple users, Ollama is the wrong shape, not the slower option.

## Family one: the llama.cpp lineage

### llama.cpp

The engine everything else in this family is built on, at 121,567 stars and MIT licensed. Note it now lives under the ggml-org organisation rather than the original ggerganov account.

Use it directly when you need control or when your hardware is unusual. It runs on CPUs, on Apple Silicon, on modest GPUs, and on things that are not really meant to run language models at all. Its quantisation format, GGUF, is the one you keep seeing on Hugging Face, and if you want to understand what those suffixes mean, we broke that down in [quantization formats explained](/tutorials/quantization-formats-explained.html).

The cost is ergonomics. You manage model files, flags and server processes yourself.

### Ollama

The most popular project in this entire comparison at 176,859 stars, MIT licensed, and the reason most people's first local model ever ran. It hides llama.cpp behind a daemon and a registry:

```

ollama run qwen3

```

That is the whole onboarding. It also exposes an OpenAI-compatible endpoint, which is why nearly every local AI tutorial, including our own [n8n generator build](/tutorials/build-your-own-ai-generator-n8n-ollama-claude-api-2026.html), assumes it. It has been moving fast lately, picking up an [interactive agent mode in v0.32](/news/ollama-v0-32-interactive-agent-2026.html).

Its limitation is the one above: no continuous batching. Excellent for one user, wrong tool for fifty.

### LM Studio, and Jan

LM Studio is the polished desktop app: browse models, click, chat, with a local server when you want one. It bundles both llama.cpp and MLX, so it handles Apple Silicon natively.

It is also the one closed-source tool here, and that deserves a precise statement rather than a warning. LM Studio is proprietary, developed by Element Labs, but it is free for personal and commercial use. The separate work licence was removed in July 2025, so using it at your job requires no form and no fee. Its lms CLI is MIT.

So the reason to prefer an alternative is auditability, not cost. If you want a GUI and open source, Jan is the Apache 2.0 option covering the same ground. Closed source does not mean paid here, and it is worth being clear about which objection actually applies to you.

## Family two: built for concurrency

### vLLM

The default choice for serving, at 87,140 stars and Apache 2.0. It originated PagedAttention, supports a very wide range of model architectures, and exposes an OpenAI-compatible API so most clients work unchanged.

It is what you reach for when requests arrive in parallel. It is also what the wider ecosystem assumes: both the [NemoClaw agent stack](/tutorials/run-open-agent-stack-openshell-langchain-deep-agents-2026.html) and [Qwen3-ASR](/tutorials/qwen3-asr-open-whisper-alternative-local-2026.html) document vLLM as their serving path.

The tradeoff is that it wants a real GPU and more setup than ollama run. There is no CPU-only story worth having.

### SGLang

The younger sibling at 30,736 stars, also Apache 2.0, also continuously batched. Its specialisation is workloads where many requests share a long common prefix, which describes RAG almost exactly: the same retrieved context and system prompt in front of many different questions. On that shape it can edge out vLLM.

For general serving with wide model support, vLLM remains the shorter path. Reach for SGLang when your traffic actually has that prefix-heavy shape rather than on principle.

## The one that quietly fell behind

Hugging Face's Text Generation Inference used to appear in every list like this. It is worth checking whether it still should. Repository activity, read from the GitHub API on the date below:

Project | Stars | License | Last pushed
Ollama | 176,859 | MIT | 2026-07-25
llama.cpp | 121,567 | MIT | 2026-07-25
vLLM | 87,140 | Apache-2.0 | 2026-07-25
SGLang | 30,736 | Apache-2.0 | 2026-07-25
Text Generation Inference | 10,882 | Apache-2.0 | 2026-03-21
Four of the five pushed commits the same day we checked. TGI last saw a push four months earlier and sits at roughly an eighth of vLLM's stars. We are not calling it dead, and Apache 2.0 code does not stop working. But if you are choosing a serving stack to build on in 2026, that row is the one to think hardest about, and the dates say more than any opinion would.

## Apple Silicon

MLX is Apple's array framework built for unified memory, and mlx-lm runs language models on it. On an M-series Mac it is generally the fastest path, because it is designed for the architecture rather than ported to it.

You do not necessarily need to touch it directly. LM Studio bundles MLX alongside llama.cpp and picks appropriately, which is a large part of why it is the common recommendation for Mac users who do not want to think about any of this. We used a Mac-native path in [running LTX-2 locally on Mac](/tutorials/run-ltx-2-locally-mac-pinokio-phosphene.html) for the same reason.

## So which one

Just starting, or one developer: Ollama. The onboarding is genuinely one command and the OpenAI-compatible endpoint means nothing you build against it is wasted if you migrate later.

You want a GUI: LM Studio if you want the most polished experience and do not mind closed source, Jan if you would rather stay open. Both are free.

Building a product with real users: vLLM. Not because Ollama is slow, but because concurrent serving is a different problem and Ollama does not attempt to solve it. Migrating later is not painful, since both speak the same API shape, so starting on Ollama and moving when you have traffic is a reasonable plan rather than a mistake.

RAG at volume: benchmark SGLang against vLLM on your own traffic before committing. The prefix-sharing advantage is real but workload-specific.

Weird hardware, or no GPU: llama.cpp, and see [AirLLM](/tools/airllm-running-massive-70b-llms-on-a-4gb-gpu.html) for the extreme end of low-VRAM tricks.

The ten-minute move: if you have never run a local model, install Ollama and pull one. If you already have and are wondering whether to switch, the useful question is not which is faster but whether you have concurrent users. If you do not, you are already on the right tool.

## Sources and further reading

- [Quantization formats explained: what GGUF and the rest actually mean](/tutorials/quantization-formats-explained.html)

- [Run a full local AI stack on your own hardware](/tutorials/run-full-local-ai-stack-with-local-ai-packaged.html)

- [ggml-org/llama.cpp on GitHub (MIT)](https://github.com/ggml-org/llama.cpp)

- [ollama/ollama on GitHub (MIT)](https://github.com/ollama/ollama)

- [vllm-project/vllm on GitHub (Apache 2.0)](https://github.com/vllm-project/vllm)

- [sgl-project/sglang on GitHub (Apache 2.0)](https://github.com/sgl-project/sglang)

- [LM Studio: free for use at work](https://lmstudio.ai/blog/free-for-work)

- [Apple MLX on GitHub](https://github.com/ml-explore/mlx)

Tested on: not independently tested. Our bench has no NVIDIA GPU, so we ran no throughput benchmarks and quote none as our own. The star counts, licences and last-push dates in the table above were read directly from the GitHub API on the date below and are the load-bearing facts in this article. Throughput comparisons are community-reported and workload-dependent; we describe the architectural mechanism, continuous batching and PagedAttention, rather than repeating a specific multiplier as though it were universal. LM Studio's licensing is quoted from its own announcement.

Date checked: 2026-07-25

---


# Qwen3-ASR: The Open Whisper Alternative You Can Run Locally
Source: https://singularitybyte.com/tutorials/qwen3-asr-open-whisper-alternative-local-2026.html

This is another entry in our Local AI Stack series. In [the Whisper guide](/tutorials/whisper-ai-local-transcription-guide.html) we set up local speech-to-text with the model everyone reaches for by default. Whisper is still excellent. It is also no longer the only serious open option, and for a lot of real audio it is no longer the best one.

Qwen3-ASR is Apache 2.0, comes in 0.6B and 1.7B sizes, and the two Transformers-native builds have pulled over 206,000 downloads. The headline you will see is "it beats Whisper." That is too simple, and the real picture is more useful.

## TL;DR

- Two models, Apache 2.0, 0.6B and 1.7B, covering 52 languages and dialects (30 languages plus 22 Chinese dialects).

- It is not a Whisper killer, it is a Whisper complement. Whisper still edges it on pristine English audio. Qwen wins on noisy audio, accents, Mandarin and Cantonese, sometimes by enormous margins.

- Not new, despite the recent buzz: released 2026-01-29. What changed is native Transformers support on 2026-06-26, which is what made it a two-line install and drove the download numbers.

## What Qwen3-ASR actually is

Two all-in-one speech recognition models from the Qwen team, built on the Qwen3-Omni foundation model. They do language identification and transcription in a single pass, and they handle more than clean dictation: speech, singing voice, and songs with background music are all in scope. Offline and streaming inference run through one unified architecture rather than separate models.

Sizes and adoption, from the Hugging Face API:

Model | Downloads | What it is for
Qwen3-ASR-0.6B | 135,557 | The efficiency pick. 0.8B params on disk. Claims 2000x throughput at a concurrency of 128.
Qwen3-ASR-1.7B | 71,136 | The accuracy pick, and the one in the benchmark table below.
Qwen3-ForcedAligner-0.6B | 20,422 | Separate model for word-level timestamps and alignment.
On the Hugging Face Open ASR Leaderboard the 0.6B posts a mean WER of 6.31 percent and the 1.7B posts 5.59 percent. Those are aggregate English numbers, and on their own they undersell the model, because the interesting behaviour is in the spread rather than the mean.

## Where it beats Whisper, and where it does not

Here is the project's own benchmark table against Whisper-large-v3. Word error rate, so lower is better, and the winner in each row is bold.

Benchmark | Whisper-large-v3 | Qwen3-ASR-1.7B
Librispeech clean | 1.51 | 1.63
Librispeech other | 3.97 | 3.38
GigaSpeech | 9.76 | 8.45
Accented English | 21.30 | 16.07
AISHELL-2 (Mandarin) | 5.06 | 2.71
Mandarin | 10.61 | 3.81
Cantonese | 31.04 | 4.12
The two Librispeech rows are the ones worth sitting with, because they are the same corpus split by difficulty. "Clean" is studio-quality read speech, and Whisper wins it. "Other" is the harder, noisier split, and Qwen wins that. That single pair tells you the shape of the whole comparison: Whisper is marginally better on pristine audio, and Qwen is better as soon as the audio gets difficult.

Then look at Cantonese. Whisper posts 31.04 percent WER, which is roughly one word in three wrong, meaning it is not usable. Qwen posts 4.12 percent. That is not an incremental gain, it is the difference between a feature you can ship and one you cannot. Mandarin shows the same pattern less dramatically, and accented English sits in between.

So the honest summary is not "Qwen3-ASR is better." It is that Whisper was trained in a way that made clean English its strongest case, and Qwen3-ASR is much more robust everywhere else. If your audio is podcast-quality American English, your existing Whisper pipeline is fine. If it is phone calls, accented speakers, background noise, or anything Chinese, this is a meaningful upgrade.

## Running it

Native Transformers support is the whole reason this is now easy. One install:

```

pip install "transformers>=5.13.0"

```

Then transcribe. The processor has a dedicated transcription-request helper, so you do not hand-build the prompt:

```

from transformers import AutoProcessor, AutoModelForMultimodalLM

model_id = "Qwen/Qwen3-ASR-0.6B-hf" # or Qwen/Qwen3-ASR-1.7B-hf
processor = AutoProcessor.from_pretrained(model_id)
model = AutoModelForMultimodalLM.from_pretrained(model_id, device_map="auto")

inputs = processor.apply_transcription_request(
audio="https://example.com/audio.wav",
).to(model.device, model.dtype)

output_ids = model.generate(**inputs, max_new_tokens=256)
generated_ids = output_ids[:, inputs["input_ids"].shape[1]:]

transcription = processor.decode(
generated_ids, return_format="transcription_only",
)[0]
print(transcription)

```

audio= takes a local path as readily as a URL. Language identification is automatic, so there is no language flag to set unless you want to pin one.

If you are pushing volume, the project documents torch.compile giving roughly a 2.4x speedup on an A100 at batch size 4, and supports vLLM for serving.

### Word-level timestamps

The ASR models transcribe but do not align. If you need per-word timings, for subtitles or for seeking into a recording, that is the separate Qwen3-ForcedAligner-0.6B model. This is a design decision worth knowing before you plan a subtitle pipeline: Whisper gives you segment timestamps in the same pass, whereas Qwen splits transcription and alignment into two models. Two steps, but the alignment model is small and you only pay for it when you need it.

## Hardware reality

Be careful here, because the parameter counts invite a wrong assumption. A 0.6B model sounds like something that runs comfortably on a laptop CPU, and it may well. But the project does not document CPU inference at all. Every piece of guidance is GPU-shaped: cuda:0 in the examples, vLLM for serving, FlashAttention 2 for speed, A100 numbers for the benchmarks.

That does not mean CPU is impossible. It means nobody has published what to expect, so if your plan is CPU-only transcription, budget time to find out for yourself rather than treating it as a supported path. The 0.6B is the obvious candidate if you try.

For comparison, this is a real advantage Whisper retains: years of community work has produced [whisper.cpp](https://github.com/ggml-org/whisper.cpp) and a mature quantized CPU story. Qwen3-ASR has the better model on hard audio; Whisper has the better ecosystem on constrained hardware. That trade is the actual decision.

## Which should you use

Stay on Whisper if your audio is clean English, if you are running CPU-only or on a Raspberry Pi class device, or if you depend on the surrounding tooling (whisper.cpp, faster-whisper, the many wrappers). None of that is displaced.

Switch to Qwen3-ASR if you transcribe Mandarin or Cantonese, where the gap is not close. Also switch if your English audio is accented, noisy, or recorded over a phone, where the Librispeech "other" and accented-English rows say you will measurably do better.

Run both if you are building a product. They are both Apache 2.0, both small, and routing by detected language costs you almost nothing.

The ten-minute move: take the worst piece of audio you have, the one your current pipeline mangles, and run it through the 0.6B with the snippet above. The aggregate benchmarks will not tell you whether this helps your data. That one file will.

## Sources and further reading

- [Our guide to running Whisper locally](/tutorials/whisper-ai-local-transcription-guide.html)

- [QwenLM/Qwen3-ASR on GitHub (Apache 2.0, benchmark tables)](https://github.com/QwenLM/Qwen3-ASR)

- [Qwen3-ASR-0.6B on Hugging Face](https://huggingface.co/Qwen/Qwen3-ASR-0.6B-hf)

- [Qwen3-ASR-1.7B on Hugging Face](https://huggingface.co/Qwen/Qwen3-ASR-1.7B-hf)

- [Qwen3-ForcedAligner-0.6B on Hugging Face](https://huggingface.co/Qwen/Qwen3-ForcedAligner-0.6B-hf)

- [Qwen3-ASR technical report (arXiv 2601.21337)](https://arxiv.org/html/2601.21337v1)

Tested on: not independently tested. Our bench has no NVIDIA GPU, and the project documents no CPU path, so we did not run inference. Every WER figure above is quoted from the project's own published benchmark table against Whisper-large-v3, not from a third-party summary, and the download counts and Apache 2.0 licensing were read directly from the Hugging Face API on the date below. Release dates come from the repository's own changelog.

Date checked: 2026-07-25

---


# Open-Source Video Models Mid-2026: What Actually Shipped, and What Did Not
Source: https://singularitybyte.com/tutorials/open-source-video-models-mid-2026-what-changed.html

This is the fourth entry in our open-source AI video series. In [the April comparison](/tutorials/ai-video-generator-comparison-2026-open-source-models-tested.html) we tested Wan 2.1 and 2.2, HunyuanVideo and Mochi on real hardware and named a winner per use case. Three months on, the obvious follow-up was "here are the new models." We started writing that piece and the facts did not cooperate.

The headline change since April is not a new open model. It is that the web filled up with confident, well-ranked claims about a model that was never opened at all.

## TL;DR

- Wan 2.7 is not open source. Official Wan open weights still stop at 2.2. There is no Wan2.7 repository and no weights on Hugging Face, despite a cluster of sites saying otherwise.

- LTX-2.3 is the real story. 22B parameters, synchronized audio and video from one model, and over 2.1 million downloads. It is the most-downloaded open video model we can find by a wide margin.

- None of the leaders is Apache 2.0 at the frontier. The "they all went permissive" claim is wrong in three different ways, laid out below.

## The Wan 2.7 problem

Search for open-source video models today and you will be told, repeatedly and confidently, that Wan 2.7 shipped under Apache 2.0 in April 2026 with first/last-frame control and 9-grid image input. Several of those pages give install instructions and download links.

We went looking for the weights. They are not there:

Check | Result
Hugging Face search for Wan2.7 | 0 models
Wan-Video/Wan2.7 on GitHub | 404
Newest video model in the Wan-AI org | Wan-Dancer-14B, plus the Wan 2.2 family
Newest official Wan repo | Wan-Video/Wan2.2, Apache-2.0, last pushed 2026-03-17
Wan 2.7 does exist. It is a real model with real capabilities, available through Alibaba's cloud and API. What it is not is downloadable. Alibaba's pattern with Wan has been cloud first, open weights later, and for 2.7 the second half has not happened.

The sites saying otherwise share a tell: many are single-purpose domains built around the model name, publishing "open source guide" and "how to download" pages for something with nothing behind them. They rank well because nobody else was competing for those terms.

The practical consequence for readers of our April piece is reassuring. We tested Wan 2.1 and 2.2 because that was the open Wan, and 2.2 is still the open Wan. That article did not age out. The newer content did.

## LTX-2.3 is what actually shipped

The model that genuinely changed the picture came from Lightricks. LTX-2 was open-sourced in January 2026; the current release is LTX-2.3, a 22-billion-parameter diffusion transformer that generates video and synchronized audio together from a single model rather than bolting a soundtrack on afterwards. It handles text-to-video, image-to-video, and video-to-video.

The adoption numbers are what stand out. Set against the other open contenders:

Model | Hugging Face downloads | Last updated
Lightricks/LTX-2.3 | 2,117,254 | 2026-07-09
Wan-AI/Wan2.2-TI2V-5B-Diffusers | 153,577 | 2025-08-09
Wan-AI/Wan2.2-I2V-A14B-Diffusers | 128,077 | 2025-08-09
tencent/HunyuanVideo-1.5 | 4,009 | 2025-12-25
Downloads measure interest rather than quality, and the Wan totals are split across many repository variants, so do not read this as a capability ranking. But a 500x gap over HunyuanVideo-1.5 is not noise, and it lines up with the other signal: an active adapter ecosystem. Lightricks has shipped LoRAs through June and July for video-to-audio foley, cinemagraphs, in/outpainting, clean-plate removal, and spatial upscaling. That is what a model people actually build on looks like.

### Requirements

LTX-2.3 wants Python 3.12 or newer, CUDA above 12.7, and PyTorch 2.7 or later. Resolution has to be divisible by 32, and frame count divisible by 8 plus 1, which will bite you the first time you request 100 frames and get an error.

## The licensing claim, corrected

The tidiest version of the mid-2026 story is that the open video leaders converged on Apache 2.0. We were ready to write that. It is wrong three times over:

Model | Actual license | The catch
Wan 2.2 | Apache-2.0 | Genuinely permissive, but this is 2.2. The newer 2.7 has no weights at all.
LTX-2.3 | LTX-2 Community License Agreement | Not Apache. Free for research and for commercial use under 10 million dollars ARR; above that you negotiate.
HunyuanVideo-1.5 | Other (GitHub reports NOASSERTION) | A Tencent community license, not a standard open-source one.
The LTX threshold has a detail worth reading before you assume you are under it: subsidiaries, affiliates, and companies under common control are counted collectively. A small studio owned by a large group may be over the line. Lightricks states there are no retroactive charges, so crossing the threshold is a conversation rather than a penalty.

For most readers of this site, "free under 10 million ARR" is functionally free. It is still not Apache 2.0, and if you are choosing a model on license terms the difference matters.

## Check any model claim yourself in about a minute

The useful takeaway is not the specific facts above, which will move again. It is that both Hugging Face and GitHub expose public APIs that settle "is this actually released" in one command, with no key and no account.

Does the model exist on Hugging Face?

```

curl -s "https://huggingface.co/api/models?search=LTX-2.3&limit=5" \
| python3 -c "import sys,json; [print(m['modelId'], m.get('downloads')) for m in json.load(sys.stdin)]"

```

An empty result is your answer. That single check is what showed us Wan 2.7 had no weights.

What has an organization actually published, most recent first?

```

curl -s "https://huggingface.co/api/models?author=Wan-AI&sort=lastModified&direction=-1&limit=10" \
| python3 -c "import sys,json; [print(m['modelId'], str(m['lastModified'])[:10]) for m in json.load(sys.stdin)]"

```

And the repository's real license, rather than what a blog says it is:

```

curl -s https://api.github.com/repos/Wan-Video/Wan2.2 \
| python3 -c "import sys,json; j=json.load(sys.stdin); print(j['license']['spdx_id'], j['pushed_at'])"

```

A NOASSERTION here means GitHub could not match the LICENSE file to a standard license, which is a strong hint you are looking at a custom community license rather than Apache or MIT. That is exactly what HunyuanVideo-1.5 returns.

## What this changes about the April comparison

Less than you would expect, which is the point.

Our tested numbers for Wan 2.2 and HunyuanVideo still describe the current open field, because the open field did not move as much as the coverage suggests. The genuine gap is LTX-2.3, which we have not benchmarked and which did not appear in that article at all. If you want the hardware side, [the self-hosted stack guide](/tutorials/self-hosted-ai-video-generator-stack-hardware-guide-2026.html) still applies; a 22B model sits above the Wan 2.2 5B variant and below the heavier A14B configurations in memory terms.

## What we would test next

Three things for the next entry. LTX-2.3 against Wan 2.2 on the same prompts and hardware as the April run, so the comparison is like for like. Whether the single-model synchronized audio actually beats generating video and audio separately, which is LTX's core architectural bet. And whether the foley and upscaler LoRAs hold up outside the demos, since an adapter ecosystem is only worth something if the adapters work.

If Alibaba does open Wan 2.7, that becomes the story and we will cover it. It has not happened yet.

## Sources and further reading

- [Part one: our tested open-source video model comparison](/tutorials/ai-video-generator-comparison-2026-open-source-models-tested.html)

- [The self-hosted video stack hardware guide](/tutorials/self-hosted-ai-video-generator-stack-hardware-guide-2026.html)

- [Lightricks/LTX-2.3 on Hugging Face](https://huggingface.co/Lightricks/LTX-2.3)

- [The LTX-2 Community License Agreement](https://github.com/Lightricks/LTX-2/blob/main/LICENSE)

- [Wan-Video/Wan2.2 on GitHub (Apache-2.0)](https://github.com/Wan-Video/Wan2.2)

- [Tencent HunyuanVideo-1.5 on GitHub](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5)

Tested on: not independently tested. We did not run LTX-2.3, which needs an NVIDIA GPU our bench does not have. Every release claim above was verified directly against the Hugging Face and GitHub public APIs on the date below, using the exact commands published in this article: model existence and download counts, organization publishing history, repository licenses, and last-push dates. Capability descriptions are the projects' own. Where our findings contradict widely-published claims, we have shown the check so you can repeat it.

Date checked: 2026-07-25

---


# StableDAW Is Now theDAW: What Changed and How to Move Your Setup
Source: https://singularitybyte.com/tutorials/stabledaw-is-now-thedaw-whats-new-2026.html

This is part two of our local AI music studio series. In [part one](/tools/stabledaw-ai-music-studio-pinokio.html) we covered StableDAW, the browser-based AI music studio built on Stable Audio 3 that installed in one click through Pinokio and ran the small model on a plain CPU. Nine days after that article went up, the project was archived. It did not die, it moved and grew: the repo is now [gantasmo/theDAW](https://github.com/gantasmo/theDAW), it went from seven workspaces to ten, and it picked up VST3 hosting, stem separation, and MIDI transcription along the way.

There is a catch, and it is the reason this article exists. The one-click install path still works, and that is the problem. It silently gives you the old build.

## TL;DR

- StableDAW is archived and superseded by theDAW. Same author, same MIT license, roughly twice the application.

- The Pinokio launcher has not been updated. It still clones the archived repo, so the install succeeds and hands you the June build. There is no Pinokio path to theDAW right now.

- Latest release is v0.1.3 (2026-07-06). Use the Windows script, a release installer, the container image, or a source checkout instead.

Workspace | Status | What it does
MAKE | Carried over | Text-to-audio, audio-to-audio, inpainting, continuation, Chimera fusion
EDIT | Carried over | Multitrack timeline, waveform editing, trims and fades
MIX | Expanded | 25 FFmpeg effects, now hosts VST3 and .gan web-plugins
DJ | Expanded | Two decks, plus stems, key-lock and automix
VJ | Carried over | WebGL reactive visuals, shaders, camera sources
LEARN | Carried over | Genealogy graph of track lineage
TRAIN | Renamed to Underfit | LoRA adapter training, eight adapter types
Perform | New | Live clip launcher driven from imported projects
Foundry | New | Plugin-interface designer, exports .gan web-plugins
Audimate | New | Node-graph editor for generation pipelines
## What happened to StableDAW

The numbers tell the story plainly. The StableDAW repo took its final commit on 2026-06-13 and was archived later that month, sitting at 44 stars. theDAW is a separate, active repo: last push 2026-07-10, 116 stars, and four tagged releases in July. Both are MIT.

This was not a GitHub rename, which matters more than it sounds. A rename leaves a redirect, so old clone URLs quietly follow you to the new project. Here the old repo still exists as a read-only archive at its original address. Anything pointing at it keeps working and keeps serving June code.

Not sure which one you are running? The fastest check is the tab bar. Seven tabs ending in TRAIN means the archived build. Ten tabs including Foundry and Audimate means theDAW. Or ask git directly:

```

cd /path/to/your/install
git remote -v
# .../gantasmo/stabledaw -> archived June build
# .../gantasmo/theDAW -> current

```

## The Pinokio problem

Part one told you to install through Pinokio, because at the time that was the best path available. That advice is now stale, and it fails in the most annoying way possible: quietly.

The launcher at cocktailpeanut/stabledaw.pinokio took its last commit on 2026-06-12, the day before StableDAW's final commit and before the archive. It has not been touched since. Its install step is a single hardcoded clone:

```

git clone https://github.com/gantasmo/stabledaw app

```

Because the archived repo is still readable, that clone returns 0. Pinokio reports a clean install. You get a working music studio. It is simply the June build, missing Foundry, Audimate, Perform, VST3 hosting, Demucs stems and MIDI transcription, and it will never update.

We checked for a replacement and there is not one yet: cocktailpeanut/thedaw.pinokio and the obvious variants all 404 as of 2026-07-25. So if you want theDAW today, you install it another way. Below are the paths that actually land on the current code.

## Installing theDAW

Four options, roughly easiest first. Requirements across all of them: Python 3.10 or newer, Node.js 20.19+ or 22.12+, FFmpeg on your PATH, and for the Medium model or Magenta RealTime 2, an NVIDIA driver at 550 or later. The Small model still runs on CPU, Apple Silicon included.

### Windows: the batch script

Double-click theDAW.bat. It inspects the machine, installs whatever is missing after one confirmation, and opens the app in your browser. There is also theDAW-desktop.bat for the Electron shell rather than a browser tab, and install/setup.ps1 if you would rather drive PowerShell yourself.

### Release installers

Every GitHub release ships a Windows installer and a macOS disk image, which is new since part one. Grab theDAW-Setup-<version>.exe or theDAW-<version>.dmg from the releases page. Current version is v0.1.3.

### Container

There is now an official image, which is the cleanest option if you already run containers and do not want Python and Node on your host:

```

docker pull ghcr.io/gantasmo/thedaw
# the repo also ships a docker-compose.yml

```

### From source

The one to use if you want to track development. Note the submodules flag, the checkout is incomplete without it:

```

git clone --recurse-submodules https://github.com/gantasmo/theDAW
cd theDAW

# backend
uv run uvicorn backend.server:app --host 0.0.0.0 --port 8600 --reload

# frontend, in a second terminal
cd frontend && npm run dev

```

Port 8600 is not negotiable: the frontend proxies /api/* to it and that address is baked into the Vite config. The frontend takes 5173, and the launcher also clears 5187. Same port story as part one, so if you had those free before, you still do.

## The four new workspaces

Underfit is TRAIN with a new name and more depth. It fits LoRA adapters to the Stable Audio 3 RF base models across eight adapter types: lora, dora-rows, dora-cols, bora, and an -xs variant of each. You can filter which layers get adapted with --include and --exclude, adjust adapter strength at runtime rather than retraining, and stack multiple adapters additively. Still needs an NVIDIA GPU.

Foundry is the one with no obvious equivalent anywhere else. It is an infinite-canvas designer for plugin interfaces: you lay out the controls, and the finished design exports as a .gan web-plugin that hosts inside the MIX chain right next to VST3 plugins and the built-in effects. Designing your own plugin UI is not something local AI audio tools normally let you do.

Audimate is a node-graph editor for building generation pipelines. Nodes cover library sources, generation (Stable Audio or Magenta), effects, merges and feedback stages, wired together with bezier edges on a pannable canvas. If you have used ComfyUI for images, the shape will be familiar.

Perform is a live clip launcher fed from imported projects, aimed at playing a set rather than arranging one. Together with the DJ tab's new stems, key-lock and automix, the live half of the app got most of this release's attention.

## What else came along

Three additions are worth knowing about even though they do not have their own tabs. theDAW ships a CUDA port of Google's Magenta RealTime 2, which the project describes as the first non-Mac port. Demucs handles stem separation, so you can pull a track apart rather than only generating new material. And audio can now be transcribed to MIDI and rendered as notation, which closes the loop between generated audio and something you can actually edit as music.

## Moving your existing setup

There is no migration tool, so treat this as a fresh install alongside the old one rather than an upgrade in place. Keep the archived install until you have confirmed the new one works.

Your generated audio is just files, so copy them out of the old install's data directory. LoRA adapters you trained in TRAIN target the same Stable Audio 3 RF base models that Underfit uses, so they should carry over, though verify strength settings after loading rather than assuming they match. The one thing you cannot bring is the LEARN genealogy graph, which is tied to the old install's library database.

## Limitations and gotchas

- No Pinokio path. The headline issue. Until someone updates the launcher, the one-click route lands on archived code.

- Early version numbers. v0.1.3, with four releases in two days, and the repo carries a file named ERRORS-AND-FIXES-2026-07-05.md. This moves fast and breaks.

- The GPU line moved. More of the interesting surface now needs NVIDIA: Medium model, Magenta RT2, Underfit training. CPU users still get the Small model and the whole edit, mix and perform side.

- Licensing is unchanged. App code MIT, model weights under the Stability AI Community License. Commercial use is fine under 1 million dollars in annual revenue. Still no vocals: Stable Audio 3 makes instrumental music and sound effects.

- Submodules. A plain git clone gives you a broken checkout. Use --recurse-submodules.

## Who should move, and when

If you installed through Pinokio after part one and have been happy, you are on the June build and you are missing a lot. Moving is worth it, but do it as a parallel install.

If you never installed it, skip StableDAW entirely and start at theDAW with a release installer or the container.

If you only ever wanted a local text-to-audio sketchpad on a laptop with no GPU, the archived build genuinely still does that. There is no urgency.

The ten-minute move: run git remote -v in your install directory. If it says stabledaw, you know where you stand, and the container pull above gets you current without touching your existing setup.

## Sources and further reading

- [Part one: StableDAW, the one-click AI music studio](/tools/stabledaw-ai-music-studio-pinokio.html)

- [gantasmo/theDAW on GitHub (MIT)](https://github.com/gantasmo/theDAW)

- [theDAW releases](https://github.com/gantasmo/theDAW/releases)

- [gantasmo/StableDAW, archived](https://github.com/gantasmo/StableDAW)

- [The unmaintained Pinokio launcher](https://github.com/cocktailpeanut/stabledaw.pinokio)

- [Stable Audio 3 Medium on Hugging Face](https://huggingface.co/stabilityai/stable-audio-3-medium)

Tested on: not independently tested. Our bench has no NVIDIA GPU, so we did not run theDAW end to end. Everything above comes from the project's own repositories and release artifacts, verified against the GitHub API on the date below: archive status and commit dates on both repos, the release tags and timestamps, the launcher's clone target read from its install.js, and the absence of a theDAW Pinokio launcher. Feature descriptions are the project's own.

Date checked: 2026-07-25

---


# Run a Fully Open Agent Stack: OpenShell, LangChain Deep Agents, and Nemotron 3
Source: https://singularitybyte.com/tutorials/run-open-agent-stack-openshell-langchain-deep-agents-2026.html

This is part two of our Nemotron series. In [part one](/models/nvidia-nemotron-3-ultra.html) we looked at Nemotron 3 Ultra and argued that the interesting thing was not the 550 billion parameters, it was the license: NVIDIA shipped the weights, the pretraining data, the post-training recipes, and the RL environments under OpenMDW-1.1. We ended that piece by telling you to point a LangChain Deep Agents loop at the endpoint. On July 8, 2026, NVIDIA and LangChain did exactly that and packaged it. The result is called NemoClaw, and it is the first time every layer of an agent stack (model, harness, and runtime) has been open at the same time.

## TL;DR

- NemoClaw is a blueprint, not a product: Nemotron 3 Ultra for the model, LangChain Deep Agents Code for the harness, NVIDIA OpenShell for the sandboxed runtime.

- All three layers are open, and OpenShell is Apache 2.0. Despite the NVIDIA branding, none of it requires an NVIDIA GPU, because GPU support in OpenShell is optional and the model can sit behind any OpenAI-compatible endpoint.

- The headline "10x cheaper" number comes from LangChain's own eval suite, run by the two companies announcing the product, against an unnamed competitor. Treat it as vendor-reported.

Layer | Component | License
Model | Nemotron 3 Ultra (550B total, 55B active, 1M context) | OpenMDW-1.1
Harness | LangChain Deep Agents Code (dcode) | Open source
Runtime | NVIDIA OpenShell | Apache 2.0
## What changed since part one

Part one covered a model. This covers the two pieces that were missing around it.

An agent is not a model. It is a model plus a loop that plans, calls tools, remembers things, and keeps going for hours. That loop is the harness. And because the loop runs shell commands and touches files, it needs somewhere safe to do that, which is the runtime. Until now you could get an open model easily and an open harness fairly easily, but the runtime layer was where you quietly ended up on somebody's hosted platform.

NemoClaw fills that gap. The harness is LangChain's Deep Agents Code, shipped as a terminal agent called dcode, in the same category as Claude Code or Codex. The runtime is NVIDIA OpenShell. Both are open, both are self-hostable, and that is the whole story.

## OpenShell is the part worth paying attention to

The model gets the headlines, but OpenShell is the piece that did not exist before.

It is a sandbox for autonomous agents, written in Rust, that runs a K3s Kubernetes cluster inside a single Docker container. You do not install Kubernetes separately. Agents run unmodified inside it while OpenShell enforces filesystem, network, and process rules from a declarative YAML policy, and it keeps an audit trail of every allow and deny decision it makes.

That last part matters more than it sounds. Most people running coding agents today either give them full access to a real machine and hope, or run them in a container and lose track of what they actually did. An audit log of denied actions tells you what your agent tried to do, which is exactly the thing you want to know before you let it run unattended for eight hours.

It is also honest about its maturity. NVIDIA describes it as alpha and calls it "single-player mode": one developer, one environment, one gateway. That is not a stack you put in front of a team this quarter.

## Getting it running

OpenShell supports Linux, Apple Silicon macOS, and WSL2 (experimental). There are two install paths. The PyPI one:

```

uv tool install -U openshell

```

And the official installer, which fetches a release package from GitHub:

```

curl -LsSf https://raw.githubusercontent.com/NVIDIA/OpenShell/main/install.sh | sh

```

Then you create a sandbox and run an agent inside it:

```

openshell sandbox create -- claude
openshell sandbox connect
openshell policy set default --policy policy.yaml

```

### The gotcha nobody documents

The two install paths do not have the same requirements, and we hit this the hard way. The PyPI wheels are built for manylinux_2_39, which means glibc 2.39 or newer. On Debian 11 (glibc 2.31) uv refuses to resolve:

```

× No solution found when resolving dependencies:
╰─▶ Because all versions of openshell have no wheels with a matching
platform tag (e.g., `manylinux_2_31_x86_64`) ...
hint: Wheels are available for `openshell` (v0.0.91) on the following
platforms: `manylinux_2_39_aarch64`, `manylinux_2_39_x86_64`,
`macosx_13_0_arm64`

```

The shell installer is more forgiving. Read the script and it sets its own floor at glibc 2.28, because it installs a Debian or RPM package rather than a Python wheel. So on an older distro the uv path fails and the install.sh path works. The tradeoff is that the package install needs root and registers a systemd user service, which is a bigger commitment than a user-local tool.

If you are on Ubuntu 24.04 or later, or a current Fedora, both paths work and this never comes up.

## Pointing the harness at Nemotron

Deep Agents Code installs with its own script:

```

curl -LsSf https://langch.in/dcode | bash

```

It is model-agnostic, which is the useful bit. Any provider that speaks tool calling works, and providers are declared in config.toml. Since Nemotron 3 Ultra is served over an OpenAI-compatible API, you register it like any other provider. Adapted from LangChain's documented schema:

```

[models]
default = "openrouter:nvidia/nemotron-3-ultra-550b-a55b:free"

[models.providers.openrouter]
display_name = "OpenRouter"
api_key_env = "OPENROUTER_API_KEY"
base_url = "https://openrouter.ai/api/v1"
models = ["nvidia/nemotron-3-ultra-550b-a55b:free"]
enabled = true

[models.providers.openrouter.params]
temperature = 0

```

One useful detail: dcode checks DEEPAGENTS_CODE_{NAME} before falling back to {NAME} for credentials. So you can set DEEPAGENTS_CODE_OPENROUTER_API_KEY and give the agent its own key without touching what every other tool on your machine uses.

### Putting the harness inside the sandbox

Here is the part the announcement glosses over. OpenShell's four headline agents (Claude Code, OpenCode, Codex, Copilot CLI) are not a whitelist, they are the ones that come pre-installed in the base image with credentials auto-discovered from your shell. Everything after -- on sandbox create is passed verbatim to the container, so any binary runs. dcode simply is not one of the pre-baked ones, which means you put it in the image yourself.

That is what --from is for. It takes a community image name, a local directory with a Dockerfile, or any container image reference:

```

# build a sandbox image that has dcode installed
openshell sandbox create --from ./my-dcode-sandbox --name nemo -- dcode

# register the model endpoint explicitly, since auto-discovery
# only covers the four pre-installed agents
openshell provider create

```

Two flags worth knowing while you are here. --policy attaches your YAML rules at creation time, and --gpu requests GPU passthrough, which is opt-in. That last one is the clearest evidence that the GPU is genuinely optional rather than quietly assumed.

You do not need to self-host 550B to try this. OpenRouter carries a free Nemotron 3 Ultra endpoint at 200 requests per day with the full 1M context, which is enough to see whether the harness fits how you work. Paid pricing is $0.50 per million input tokens and $2.20 per million output. The free tier logs requests, so keep anything confidential off it.

## About that 10x number

The announcement's headline claim is that Nemotron 3 Ultra under Deep Agents scored an aggregate 0.86 at $4.48 of inference cost, against $43.48 for "the next closest model."

Read that carefully. It is LangChain's own eval suite. It was run by the two companies announcing the product. The competitor is not named, and no third party has replicated it. None of that makes it false, and the underlying economics are plausible, since a 55B-active MoE really is cheaper to serve than a dense frontier model. But an unnamed baseline in a vendor benchmark is a number you quote with attribution, not one you plan a budget around.

The claim we would actually stand behind is the boring one: you can run this stack yourself, inspect every layer, and swap any of them out. That does not need a benchmark.

## Limitations and gotchas

- OpenShell is alpha. NVIDIA says so directly. Single developer, single environment, single gateway.

- The glibc split above. Old distro, use the shell installer, expect to need root.

- dcode is not a pre-installed OpenShell agent, though it still runs. Only Claude Code, OpenCode, Codex and Copilot CLI ship in the base image with credentials auto-discovered, so getting the harness into the sandbox is on you. See the section above.

- Self-hosting the model is a datacenter problem. Nemotron 3 Ultra needs a multi-GPU node even at NVFP4, as covered in part one. The realistic setup for most people is open harness and open runtime locally, model over an API.

- Free-tier limits. 200 requests per day disappears fast when an agent runs a long task with many tool calls.

## Who should use it

If you are building agents you intend to run unattended, on data you care about, OpenShell is worth an afternoon on its own, independent of anything NVIDIA. It is Apache 2.0 and it works with agents you already use.

If you are shopping for an agent stack you can audit end to end, with no layer you cannot inspect or replace, NemoClaw is currently the only complete answer. Just size the alpha warning honestly.

If you want a coding agent that works today, this is not that. Use what you already have and revisit when OpenShell leaves alpha.

The ten-minute move: install dcode, point it at the free Nemotron endpoint with the config block above, and give it one real task. That tells you whether the harness suits you before you invest in the runtime.

## What we are watching next

Three things for the next entry in this series. Whether OpenShell picks up multi-user support, which is what turns it from a developer tool into infrastructure. Whether anyone independently replicates the cost claim. And whether the next Nemotron ships with the same open data and recipes, because that, not the parameter count, is what made part one worth writing.

## Sources and further reading

- [Part one: Nemotron 3 Ultra, a fully open 550B agent model](/models/nvidia-nemotron-3-ultra.html)

- [LangChain: NemoClaw Deep Agents blueprint announcement](https://www.langchain.com/blog/langchain-and-nvidia-launch-the-nemoclaw-deep-agents-blueprint)

- [NVIDIA: Nemotron with the LangChain Deep Agents harness](https://blogs.nvidia.com/blog/nemotron-langchain-agents-open-stack/)

- [NVIDIA OpenShell on GitHub (Apache 2.0)](https://github.com/NVIDIA/OpenShell)

- [OpenShell documentation](https://docs.nvidia.com/openshell/about/overview)

- [OpenShell: supported agents and base image contents](https://docs.nvidia.com/openshell/about/supported-agents)

- [OpenShell: managing sandboxes and the --from flag](https://docs.nvidia.com/openshell/sandboxes/manage-sandboxes)

- [LangChain deepagents on GitHub](https://github.com/langchain-ai/deepagents)

- [Deep Agents Code documentation](https://docs.langchain.com/oss/python/deepagents/code/overview)

- [Nemotron 3 Ultra free endpoint on OpenRouter](https://openrouter.ai/nvidia/nemotron-3-ultra-550b-a55b:free)

Tested on: not independently tested end to end. We verified the install paths on Debian 11 (Intel i7-10510U, 6 cores, no GPU) and confirmed the glibc 2.39 wheel requirement and the installer's 2.28 floor from the release script. We did not run an agent under OpenShell: the PyPI path will not resolve on glibc 2.31, and the package path requires a root install of alpha software. Benchmark figures are vendor-reported by LangChain and NVIDIA and are labeled as such above.

Date tested: 2026-07-25

---


# How a 28.9M-Parameter LLM Runs on an $8 ESP32-S3 Microcontroller
Source: https://singularitybyte.com/tutorials/run-28m-llm-on-esp32-s3-microcontroller-2026.html

This is part two of our microcontroller AI series. In [part one](/tutorials/edge-ai-on-microcontrollers.html) we argued the honest version of "AI on a microcontroller": a real sensor node runs TinyML classifiers, not language models, and the closest anyone had come to an LLM on an ESP32 was a 260K-parameter toy that "generates story fragments" and little else. Well, someone just moved that line. The developer slvDev shipped [esp32-ai](https://github.com/slvDev/esp32-ai), a 28.9-million-parameter language model that runs on an $8 ESP32-S3 at about 9.5 tokens per second and writes coherent little stories. That is roughly 100 times bigger than the toy. Here is the trick that makes it fit, what the numbers actually say, and why it still does not overturn part one.

## What changed since part one

Part one drew a hard line between two things people both call "edge." A microcontroller like the ESP32 has kilobytes to low megabytes of usable memory and runs TinyML: tiny quantized classifiers that spot a keyword, a sound, or an anomaly. A single-board computer like a Raspberry Pi has gigabytes and can run a small language model on its CPU. The gap between them is three orders of magnitude, and no amount of quantization closes it.

The esp32-ai project does not break that rule. It bends the memory question instead. The previous ESP32 language-model demo we cited held a 260K-parameter network. This one holds 28.9M parameters, coherent enough to finish a sentence and keep a short story on track, on the same class of $8 chip. The output is the same category as before, short synthetic stories, but the model behind it is two orders of magnitude larger. The interesting part is not the story. It is how 28.9M parameters fit somewhere that cannot hold them in RAM.

Spec | Value
Total parameters | 28.9M (25M live in flash)
Dense compute core | ~559K params (273KB at 4-bit, fits SRAM)
Config | d_model 96, 6 layers, ple_dim 128, vocab 32,768
Model file | 14.9MB (group-128 ragged-int4)
Board | ESP32-S3 N16R8 (16MB flash / 8MB PSRAM), ~$8
Speed | ~9.5 tok/s end to end (author-reported)
Training data | TinyStories
## The trick: Per-Layer Embeddings from flash

Start with the constraint. Transformer inference wants its weights in fast memory, because it touches them on every token. The ESP32-S3 has 512KB of on-chip SRAM. A 28.9M-parameter model at 4-bit is roughly 14.5MB. It cannot live in RAM. Bolting on the 8MB of PSRAM does not fix it either, and PSRAM is slow anyway, which matters later.

The move is to notice that not every parameter is a compute parameter. Per-Layer Embeddings (PLE) is a design from Google's Gemma models. The idea: give each token, at each layer, its own learned embedding vector pulled from a big lookup table, instead of folding all that capacity into the dense weights the math runs over. The table is huge in parameter count but you only ever read the handful of rows a given token needs. It is a dictionary, not a computation.

That property is what makes it fit a microcontroller. The esp32-ai project splits the model across all three memory tiers by what each part actually needs:

Memory tier | Size | What lives there | Why
SRAM | 512KB | Dense compute core (~559K params, 273KB at 4-bit) | Touched every token, must be fast
PSRAM | 8MB | Output head and working buffers | Big, scanned once per token
Flash | 16MB | 25M-param PLE lookup table (12MB), memory-mapped | Only a few rows read per token
The dense math the chip runs on every token is tiny, about 559K parameters, and it fits in fast SRAM. The 25M parameters of extra capacity sit in flash as a memory-mapped table. Per token the model reads about 450 bytes from that table, six rows, in roughly 0.12 milliseconds. The author measures the table at about 0.7 percent of per-token memory time. You get the model quality of 28.9M parameters while paying, on the hot path, for about half a million.

This is the same family of idea as the memory-saving quantization tricks we covered in our [Google TurboQuant writeup](/tools/google-turboquant.html). The theme is consistent: on constrained hardware, the wall is memory, and the wins come from not holding things you do not need in the fast tier.

## Does the table actually help, or is it plumbing?

A fair objection: maybe the gain comes from the extra wiring, not from the 25M-parameter table itself. The author ran the ablation to check, measuring perplexity (lower is better, it is how well the model predicts held-out text) across four configurations.

Configuration | Compute core | Total params | Perplexity
Baseline | 559K | 3.7M | 12.58
PLE | 558K | 28.9M | 11.41
FatEmbed | 559K | 28.9M | 11.94
ple_notable (control) | 558K | 3.7M | worse than baseline
PLE cuts perplexity from 12.58 to 11.41, about 9.3 percent, while keeping the compute core the same size. FatEmbed, a simpler way to spend the same parameter budget, does worse. The control run, ple_notable, wires up the same plumbing but without a real lookup table, and it lands worse than baseline. The author's summary is the right one: "the table does the work, not the plumbing." And the gain survives quantization. The PLE advantage measured in full precision actually holds or grows after 4-bit post-training quantization, at 124 to 126 percent retention, so the technique is not a floating-point artifact that evaporates on the shipping format.

## Where the time goes: the bottleneck moved, it did not vanish

Fitting the model is one problem. Running it fast enough to be bearable is another, and this is where the honest engineering shows. At ~9.5 tok/s a token takes about 105 milliseconds. Here is where that time goes, per the author's profiling on the dual-core chip.

Stage | Time per token | Bound by
Output head | 57.6 ms | PSRAM bandwidth
Attention | 25.6 ms | Compute
PLE lookup | 8.5 ms | Flash reads
Feed-forward | 6.9 ms | Compute
Input processing | 4.4 ms | Compute
Read that top row twice. The output head, the layer that turns the model's internal state into a probability over all 32,768 vocabulary tokens, eats 57.6 ms, more than half the budget. It lives in PSRAM, and PSRAM reads at about 60.7 MB/s. Scanning the 2.43MB of head weights once takes roughly 40 ms just to move the bytes, before any math. Compute gets the leftover ~17 ms. The flash lookup table everyone worries about? 8.5 ms, and most of that is not the table reads themselves.

So the PLE trick did not make the model free. It moved the bottleneck. The dense core no longer bounds you; PSRAM bandwidth does. The author's own math puts the theoretical ceiling around 58 tok/s if you were perfectly bandwidth-bound, and the current runtime sits well under that because the compute stages have not all been overlapped with the reads yet. This is the useful lesson for anyone building on constrained hardware: solve the memory-capacity wall and you often just meet the memory-bandwidth wall behind it.

## Running it yourself

The project ships everything: the training and quantization code in Python, the ESP32 firmware, and the exported model. You do not need a training run to try it, the repo includes a pre-exported model you can flash. The firmware builds with the Arduino ESP32 core (3.3.10) driven by arduino-cli, not ESP-IDF or PlatformIO. The one wrinkle worth knowing is that the model binary is flashed to its own partition, separate from the firmware, so firmware-only changes do not force you to rewrite the 15MB model.

```
# 1. Export and verify the model against a golden reference on your host
cd src && uv run python export.py && cd ..
cc -O3 -o /tmp/esp32-llm-verify firmware/host_verify/verify.c -lm
/tmp/esp32-llm-verify firmware/model/model.bin firmware/model/golden.txt

# 2. Compile and upload the firmware to an ESP32-S3
arduino-cli compile --fqbn esp32:esp32:esp32s3 firmware/esp32_llm
arduino-cli upload --fqbn esp32:esp32:esp32s3 -p /dev/ttyUSB0 firmware/esp32_llm

# 3. Write the model binary to its own flash partition (only needed after export)
esptool.py --chip esp32s3 write_flash 0x110000 firmware/model/model.bin

# 4. Watch it generate
arduino-cli monitor -p /dev/ttyUSB0 -c baudrate=115200
```

What comes out is a stream of short TinyStories-style prose at about 9.5 tokens per second, rendered on a small attached display. The repo has a demo GIF if you want to see it move before you buy a board. Treat the code and weights as an experiment: this is a research demo of a storage technique, not a maintained product, so pin your toolchain versions and expect to read the source.

## What it proves, and what it does not

Here is the honest verdict, and it is two-sided on purpose.

What it proves: you can hold a 28.9M-parameter model on a $8 microcontroller and run it, if you are willing to store most of it as a memory-mapped flash table and read only the rows each token needs. Per-Layer Embeddings, pulled from a research idea into a working ESP32 firmware, is a real and reusable technique. If your problem is "this model is a bit too big for the fast memory I have," moving the embedding capacity to a slower tier you read sparsely is a lever worth knowing. The ablation shows the capacity is doing real work, and it survives 4-bit quantization.

What it does not prove: that a microcontroller can run a useful language model. This model writes toy stories. It will not answer a question, follow an instruction, call a tool, or know a single fact, because TinyStories does not teach any of that, and 28.9M parameters could not hold much of it anyway. The output category is exactly what part one described, just bigger and more coherent. And the moment you chase real capability you need a real vocabulary, a real instruction-tuned model, and a KV cache for context, and every one of those pushes you straight back off the microcontroller and onto a Raspberry Pi or a phone. Part one's cheat sheet still stands: put the reflex on the sensor, keep the brain where the memory is.

## Who should care, and what to watch

If you build products on constrained hardware, this is not a component you ship. It is a technique you file away. The people who should actually clone the repo are hobbyists who want to see an LLM breathe on an $8 chip, and researchers hunting for ways to fit more capacity into fixed memory budgets. The one thing worth watching: whether anyone applies the same flash-resident PLE trick to a small instruction-tuned model rather than a story generator. That is the experiment that would test whether this is a curiosity or a path. Until then, it is a very good curiosity, and a clean demonstration that on tiny hardware the binding constraint is always memory.

## A community fork put it on an 8MB board

Since this series is about what actually runs, here is a data point from the wild. A developer forked the project to get it onto a Seeed XIAO ESP32-S3, a board with only 8MB of flash instead of the 16MB the stock 14.9MB model needs. The full model does not fit, so the fork retrains a smaller variant with an 8,192-token vocabulary instead of 32,768. That drops the model to about 8 million parameters and a 4.12MB file, which fits an 8MB board with room to spare. Same Per-Layer Embeddings architecture, same TinyStories training, just a smaller tokenizer.

It works, and here is the twist: on the XIAO ESP32-S3 the fork reports about 11.97 tokens per second (82.9 ms per token), faster than the original's 9.5 tok/s. The reason is exactly the bottleneck described above. The output head is PSRAM-bandwidth-bound, and a smaller vocabulary means a smaller head: 0.82MB staged versus 2.43MB in the original. Shrink the thing that dominates the per-token memory scan and the whole model speeds up. It is this article's thesis confirmed on real silicon: on this hardware the binding constraint is memory bandwidth, so the size of the vocabulary head moves the number more than anything else.

A sample of what the 8MB build wrote, unedited:

Once upon a time, there was a little girl named Lily. She loved to play with her toys and run around in the park. One day, she found a big, red ball in the park. She wanted to play with it, but her mom said no.

The fork, with an 8MB partition table, a PlatformIO build, and the smaller re-export, is on GitHub: [karamble/esp32-ai](https://github.com/karamble/esp32-ai/tree/xiao-esp32s3-8mb-variant).

## Sources and further reading

- [esp32-ai repository (slvDev)](https://github.com/slvDev/esp32-ai)

- [RESULTS.md: methodology, ablations, and measurements](https://github.com/slvDev/esp32-ai/blob/main/RESULTS.md)

- [TinyStories dataset (Hugging Face)](https://huggingface.co/datasets/roneneldan/TinyStories)

- [Gemma 3 model card (Per-Layer Embeddings)](https://ai.google.dev/gemma/docs/core/model_card_3)

- [ESP32-S3 datasheet (Espressif)](https://documentation.espressif.com/esp32-s3_datasheet_en.html)

- [Part one: edge AI on microcontrollers, what actually fits](/tutorials/edge-ai-on-microcontrollers.html)

The original 28.9M model's figures are author-reported from the repository's RESULTS.md (2026-07-21). The 8MB-board figures (Seeed XIAO ESP32-S3, vocab-8192 re-export, about 11.97 tok/s) are reported by the community fork linked above.

Date checked: 2026-07-24

---


# Scale Your n8n Chatbot With a Supabase Vector Database (RAG)
Source: https://singularitybyte.com/tutorials/scale-n8n-chatbot-supabase-vector-database-rag-2026.html

A folder of markdown files makes a fine chatbot right up until it does not. Add enough documents and the agent can no longer keep every file name in its head, and it starts answering from the wrong one. The fix is not a bigger prompt. It is retrieval by meaning: embed your documents into vectors, store them in a database, and let the agent search for the most relevant chunks instead of guessing a file name. This is retrieval-augmented generation, and Supabase gives you the vector database for it without leaving the stack you already run.


This is part two of a two part thread. Part one built [a website chatbot backed by markdown files](tutorials/build-website-knowledge-chatbot-n8n-markdown-2026.html). Everything there still stands: the same AI Agent, the same Chat Trigger, the same website widget. We change exactly one thing, the tool the agent uses to find information, and in return the chatbot scales from a dozen documents to thousands.



## What Actually Changes





Part
Markdown chatbot
Vector RAG chatbot
Knowledge store
Files on disk
Supabase table of embedded chunks
Ingestion
None, just edit files
A workflow that embeds and upserts
Retrieval
Read a whole named file
Semantic search for top matching chunks
Agent, memory, Chat Trigger, website embed
Identical. Only the retrieval tool is swapped.
Because only the tool changes, you can run both side by side and migrate when you are ready. The website embed from part one does not change at all.



## Prerequisites


You need n8n, Ollama, and Supabase. The [local-ai-packaged stack](tutorials/run-full-local-ai-stack-with-local-ai-packaged.html) ships all three together, which is why we recommend it for this series. Pull an embedding model alongside your chat model:



```
ollama pull nomic-embed-text
ollama pull qwen2.5:7b-instruct-q4_K_M
```


nomic-embed-text turns text into 768 dimensional vectors and runs comfortably on a CPU. It is the embedding model the reference workflows use, and the vector width matters in the next step.



## Step 1: Turn Supabase Into a Vector Store


Supabase is Postgres, and Postgres becomes a vector database with the pgvector extension. In the Supabase SQL editor, enable the extension, create a table for embedded chunks, and add the search function n8n calls. Run this once:



```
create extension if not exists vector;

create table documents (
id bigserial primary key,
content text,
metadata jsonb,
embedding vector(768)
);

create function match_documents (
query_embedding vector(768),
match_count int default null,
filter jsonb default '{}'
) returns table (
id bigint,
content text,
metadata jsonb,
similarity float
) language plpgsql as $$
begin
return query
select id, content, metadata,
1 - (documents.embedding <=> query_embedding) as similarity
from documents
where metadata @> filter
order by documents.embedding <=> query_embedding
limit match_count;
end;
$$;
```


Two details decide whether this works. The vector width, 768, must match your embedding model exactly, or inserts fail. And match_documents is the function n8n's Supabase Vector Store node calls by name, so do not rename it without updating the node.



## Step 2: The Ingestion Loop


Now build a second workflow whose only job is to get documents into that table. This is the part markdown lookup did not have. A production version, like the one behind the Decred knowledge assistant, walks a folder, skips anything it has already embedded, and re-embeds only what changed. The node chain looks like this:



```
[
"Manual or Schedule Trigger",
"Read Files from Disk (glob /data/shared/**/*.md)",
"Loop Over Items (Split in Batches)",
"Hash File (crypto, SHA-256)",
"Check hash against document_metadata (Supabase)",
"Switch: new file / unchanged / changed",
"Extract Document Text",
"Supabase Vector Store (insert)"
]
```


The Supabase Vector Store node in insert mode does the heavy lifting. Attach three sub-nodes to it:




- Embeddings Ollama pointed at nomic-embed-text. This turns each chunk into the 768 dimensional vector the table expects.

- Default Data Loader to carry document metadata (file name, source URL, hash) alongside the text.

- Recursive Character Text Splitter with a chunk size around 400 and a small overlap. Chunking is what makes semantic search precise: the agent retrieves a paragraph, not a whole file.


The dedup step is what keeps a growing knowledge base cheap. Hash each file, store the hash in a document_metadata table, and on the next run compare. Unchanged files are skipped, changed files get their old vectors deleted and re-inserted. Without this, every run re-embeds everything.



## Step 3: Swap the Retrieval Tool


Back in the chatbot workflow from part one, delete the lookup_document tool and add a Supabase Vector Store node in retrieve-as-tool mode. Give it the same documents table, the match_documents query name, and its own Embeddings Ollama sub-node, again nomic-embed-text, so the query is embedded the same way the documents were.


Name the tool something the agent will understand, like search_knowledge, and set a topK of 4 or 5 to start. That is the number of chunks the search returns per question. The system prompt gets simpler than the markdown version, because the agent no longer needs a list of file names:



```
You answer questions about MySite from a knowledge base.
Use search_knowledge to find relevant information before answering.
Answer only from what the search returns. If it returns nothing useful,
say you do not have that information. Never invent product details.
```


The chat model, the memory with Session ID set to From input, and the Chat Trigger are untouched. Reactivate the workflow. The website widget from part one keeps talking to the same chat URL, and now every answer is backed by semantic search.



## Why the Numbers Matter


Three settings shape answer quality, and all three are worth a moment.





Setting
Start with
What it trades
Chunk size
400 characters
Smaller is more precise but loses context; larger keeps context but blurs the match
Chunk overlap
40 to 80 characters
Overlap stops ideas from being cut in half at a boundary
topK
4 to 5
More chunks give the model more to work with, but add noise and tokens
If answers feel thin, raise topK before you touch chunk size. If answers wander, lower it. Retrieval tuning is empirical, so change one number at a time and re-test the same handful of questions.



## Local or Cloud


Everything above runs locally on Ollama, which keeps your documents and your queries on your own hardware and costs nothing per call. If you would rather trade privacy for a bit more accuracy, swap the two Ollama sub-nodes for OpenAI equivalents: an OpenAI Embeddings node (remember to change the table to vector(1536) to match) and an OpenAI Chat Model. The rest of the workflow, and the entire website embed, stay exactly the same. That is the point of building it this way: the pieces are independent, so you can upgrade one without rebuilding the others.



## Where You Land


You now have a chatbot on your website that searches a real vector database by meaning, dedupes its own ingestion, and runs on hardware you control. It scales to thousands of documents, answers from specific paragraphs rather than whole files, and shares its plumbing with anything else in your n8n instance that reads the same documents table.


The natural next step is to let other systems reach this knowledge base, not just the website widget. That is exactly what [turning the workflow into an MCP tool](tutorials/n8n-mcp-server-expose-workflow-as-ai-tool-2026.html) does, so an AI agent elsewhere can query your knowledge base directly.



## Sources and Further Reading




- [Supabase vector columns and pgvector](https://supabase.com/docs/guides/ai/vector-columns)

- [n8n Supabase Vector Store node](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.vectorstoresupabase/)

- [nomic-embed-text on Ollama](https://ollama.com/library/nomic-embed-text)

- [local-ai-packaged on GitHub](https://github.com/coleam00/local-ai-packaged)


Back to the start of the thread: [the simple markdown chatbot](tutorials/build-website-knowledge-chatbot-n8n-markdown-2026.html), if you skipped straight here. Build that first, then come back and swap in the vector tool. It is a fifteen minute change once the ingestion loop is running.

---


## Models

# MotionBricks
Source: https://singularitybyte.com/models/nvidia-motionbricks-real-time-motion-generation.html

Assassin's Creed ships with over 15,000 hand-authored animation clips. Every one of them is wired into a state machine by hand: this walk blends into that turn, that turn blends into this vault, and if you add a new gait you touch a hundred transitions. NVIDIA Research just published a model that replaces that graph. MotionBricks is a single 224M-parameter neural backbone covering more than 350,000 motion skills, running at 15,000 FPS with 2 ms latency, and it drives a Unitree G1 humanoid from the same weights that drive a game character. The code is Apache 2.0, the checkpoints are downloadable, and there is a licensing catch that nobody is talking about. Here is what shipped, how it works, and what is actually open.

## What actually shipped

MotionBricks lives in NVIDIA's [GR00T-WholeBodyControl](https://github.com/NVlabs/GR00T-WholeBodyControl/tree/main/motionbricks) repository, in a motionbricks/ subdirectory. The repo sits at roughly 3.2k stars. The paper was accepted to SIGGRAPH 2026 and published in ACM Transactions on Graphics, with the preprint posted to arXiv on 2026-04-27.

The release is dual-licensed. Source code is Apache 2.0. Pretrained weights fall under the NVIDIA Open Model License, which permits commercial use with attribution, subject to NVIDIA's trustworthy-AI terms. The checkpoints ship through Git LFS rather than Hugging Face, and total about 2.2 GB:

- motionbricks_pose, 1.6 GB

- motionbricks_root, 391 MB

- motionbricks_vqvae, 273 MB

- G1-clip.ckpt, 7.5 MB

One thing to set expectations on early: this is a preview. The repo says full integration into NVIDIA's GEAR-SONIC pipeline was targeted for roughly one month after the April release. It is August now, and MotionBricks is still labelled preview. You get the interactive demo and the checkpoints; you do not yet get the fully embedded production pipeline.

## The problem: animation graphs do not scale

If you have never shipped a character controller, here is the shape of the problem. Traditional real-time animation runs on a state machine, often called an animation graph. Each node is a clip or a blend space. Each edge is a transition with conditions, blend durations, and priority rules. Walking to running is one edge. Walking to running while injured while carrying a crate while on a slope is a combinatorial mess.

The cost is not the clips. Mocap is cheap now. The cost is the wiring, and the wiring grows faster than the content. Adding a zombie gait to a finished game is not one clip, it is every transition into and out of that gait, plus the QA pass to catch the ones you missed. This is why animation teams grow superlinearly with game scope.

Robotics hit the same wall from the opposite direction. A humanoid like the Unitree G1 does not have 15,000 curated clips; it has a controller that is either hand-tuned per behaviour or trained per task with reinforcement learning. Both approaches are per-skill. Neither gives you one policy that walks, crouches, picks something up, and transitions between all three without a human specifying the transitions.

MotionBricks proposes the same fix for both: stop authoring the graph, and learn the whole space of motions in one generative model that you query at runtime.

## How it works: tokenizer, root, pose

The system is three modules, and the total is small enough to be surprising. About 224M parameters covers all 350,000 skills. For a site that mostly writes about 100B-class releases, that number is worth sitting with.

### 1. The motion tokenizer (23.5M params)

Motion is continuous, but the generative backbone works on discrete tokens, so the first job is converting one into the other. MotionBricks uses a VQ-VAE, a vector-quantized autoencoder: it compresses input into a latent space, snaps each latent vector to the nearest entry in a learned codebook of discrete symbols, then reconstructs from those symbols. The codebook entries become your vocabulary, and motion becomes a sequence you can model like text.

The encoder is a 1D convolutional U-Net with 1024 channels and three residual conv layers per level, downsampling at 2x and 4x so that T input frames become T/4 tokens. It encodes joint positions and rotations but deliberately excludes root information. The decoder mirrors it and reinjects the root trajectory through skip connections at every level.

The quantization is multi-head: instead of one codebook per latent, each embedding is split along the feature dimension and quantized separately against K parallel codebooks. That gives you a combinatorial vocabulary from small codebooks, which is how you get 350,000 skills out of something this size.

### 2. The root module (50M params)

The root is where the character actually goes in world space. It is a transformer, 512 dimensions, 12 heads, 12 layers, and it runs in two steps. First it predicts timing: given the target, how many frames should this motion take? It outputs a distribution over frame counts at 4-frame resolution. Then it predicts the actual root trajectory over 16 learnable frame-slot embeddings, covering up to 64 frames.

Splitting timing from trajectory matters. It is why a reach can take longer when the target is farther away without anyone authoring a speed curve.

### 3. The pose module (150M params)

The largest module, and the one that does the animation. A transformer at 1024 dimensions, 16 heads, 16 layers, modelling the distribution over pose tokens conditioned on the root trajectory and any keyframe constraints. Training uses masked token modelling with a cosine-scheduled curriculum: you hide a fraction of the tokens and train the model to fill them back in, starting easy and masking progressively harder. It is BERT-style pretraining applied to motion, and it is what makes the model good at in-betweening, since filling gaps is literally the training objective.

Training cost, for reference: 32 H100s across 4 nodes, 2 million updates, batch size 256 per GPU, Adam at 5e-5 decaying to 2e-6. Roughly 7 days for the tokenizer, 3 for the root module, 7 for the pose module.

## Smart primitives: the part you actually program against

The architecture is the interesting half. The interface is the useful half. NVIDIA calls the control layer "smart primitives," and it comes in two flavours.

Smart Locomotion is the navigation interface. You give it velocity, heading, and a style, and it produces the gait. Styles include stylized variants like zombie, injured, skipping, and strafing, and you can switch between them continuously at runtime instead of triggering a transition clip. Under the hood the root trajectory gets refined in stages: a critically damped spring model produces a first pass, a neural refinement pass improves it, and the decoder refines it once more. That layering is why the output does not foot-skate when you yank the stick.

Smart Objects is the interaction interface, and it is the one that eats the most authoring time in a traditional pipeline. You place a proxy keyframe describing intent, for example the hand pose at the moment of grabbing a handle, and the backbone fills in the approach, the contact, and the follow-through. You do not author the reach.

Keyframe enforcement has a dial. Setting the tolerance to zero makes the keyframe a hard constraint, so the hand lands exactly there, which is what you want for a door handle or a ladder rung. Setting it above zero makes it soft guidance, so the model gets close but prioritizes natural motion, which is what you want for a gesture. Objects bind through collision tracing for detection, a socket system for placement, and keyframe anchoring, and each behaviour can use a different number of keyframes.

## Benchmarks

The paper evaluates against six baselines on its 350k dataset. The headline table:

Method | FPS | Latency | FID | MMD | Win rate | Jnt jitter | Foot skate
MotionBricks | 15,000 | 2 ms | 1.054 | 0.1056 | 86.5% | 3.38 | 0.003
Cond. in-betweening | 27,000 | 2.4 ms | 1.594 | 0.1093 | 0.8% | 16.88 | 0.018
CondMDI | 1,930 | 33.2 ms | 1.213 | 0.1080 | 15.6% | 16.19 | 0.012
MMM | 3,600 | 18.1 ms | 1.544 | 0.1176 | 19.9% | 5.40 | 0.005
Closd-DiP | 4,200 | 15.3 ms | 1.292 | 0.1076 | 15.1% | 14.03 | 0.015
Read the first two rows together, because they are the honest story. Conditional in-betweening is faster: 27,000 FPS against 15,000. It also has a 0.8% human win rate, five times the joint jitter, and six times the foot skate. Speed was never the hard part of this problem. Quality at speed was, and that is where the gap sits.

The metrics, briefly: FID measures distribution distance from real motion, lower is better. MMD is another distributional distance. Joint jitter and foot skate are the two artifacts that make generated animation read as fake, high-frequency limb noise and feet sliding on the ground while planted. Win rate is a human preference study.

The same ordering holds on HumanML3D and LaFAN1-G1, which are public benchmarks, so this is not purely a home-field result. Note the FPS figures are measured on an RTX 5090.

## Does it scale? The 350k versus 70k question

The paper trains on four datasets, and the pair that matters is the internal 350k corpus and its 70k subset. The full set is 700 hours, 315k training clips, 36 categories, roughly 9,300 unique skills. The 70k subset is 140 hours and 62k training clips, and exists specifically as a scaling benchmark.

The scaling result is the argument for the whole approach: quality improves with corpus size, which means this is a data problem rather than an architecture problem. Given that the architecture is 224M parameters, the implied roadmap is obvious. Do not scale the model, scale the mocap.

Which raises the question of where you get the mocap.

## The dataset catch: what "open" means here

This is the part worth reading carefully before you plan around MotionBricks.

The 350k corpus the shipped checkpoints were trained on is proprietary. It is production-grade mocap from real actors, and it was not released. What was released, on 2026-03-16, is BONES-SEED: 142,220 annotated human motions, roughly 288 hours, in SOMA and Unitree G1 formats, with natural-language descriptions, temporal segmentation, and skeletal metadata. It is a genuinely large and well-annotated dataset.

It is also gated. Access on Hugging Face is restricted, and the terms cover academic use and qualifying startups; anything else goes through l&#105;&#x63;&#101;&#x6e;&#115;&#105;&#x6e;&#x67;&#x40;&#98;&#111;n&#x65;&#115;&#46;&#x73;&#116;&#x75;&#100;&#x69;&#x6f;. So the practical position for a commercial team is this asymmetry:

- You can ship the pretrained weights in a commercial product, under the NVIDIA Open Model License, with attribution.

- You cannot reproduce those weights, because the corpus behind them is not public.

- You cannot straightforwardly retrain on the open substitute either, because BONES-SEED is gated for commercial use, and it is 288 hours against the 700 the model actually saw.

The training scripts underline the point. They ship and they run, but they default to synthetic data. Real training means bringing your own corpus or clearing a license.

That is a meaningfully different posture from NVIDIA's own [Nemotron 3 Ultra](/models/nvidia-nemotron-3-ultra.html), which shipped weights, data, and recipes under a permissive license and can be reproduced end to end. MotionBricks is open-weights and open-code, not open-pipeline. Both are legitimate releases. They are not the same thing, and the difference decides whether you can fine-tune on your own studio's motion library or only consume what NVIDIA trained.

## Two audiences, one repo

MotionBricks is aimed at game and animation developers on one side and robotics teams on the other, and the two get quite different value today.

Game and animation developers get the strongest immediate case. The state machine problem is real, the smart primitives map cleanly onto how character controllers already work, and 2 ms latency fits inside a frame budget with room to spare. What you do not get is an engine plugin. There is no Unreal or Unity integration in the box, so shipping this means writing the bridge yourself and validating it against your rig. The retargeting caveat below bites here.

Robotics teams get a G1-specific path: the interactive G1 demo, G1 skeleton meshes, and BONES-SEED's G1 MuJoCo trajectories. The gap is that MotionBricks is kinematic, not physics-simulated. It generates poses, not torques, and it has no awareness of whether a motion is dynamically feasible or within the robot's joint limits. It also assumes ground-truth object poses and terrain geometry, which a real robot does not have. You need a perception stack and a physical feasibility layer between MotionBricks and hardware. That work is not small, and it is exactly what the SONIC side of the repo is for.

If you follow the site's physical-AI coverage, the split is familiar. [Cosmos 3](/models/nvidia-cosmos-3.html) predicts what the world does next; MotionBricks decides how a body moves through it. They sit at different layers of the same stack, and neither one closes the perception gap on its own.

## Limitations and gotchas

- Preview status. Full GEAR-SONIC integration was targeted for about a month after the April release and has not landed. Plan around the demo and checkpoints, not the production pipeline.

- Kinematic, not physical. The authors state the model can produce physically implausible motion, including self-collisions and motions exceeding hardware constraints. Nothing in the model prevents this.

- No visual planning. It assumes ground-truth object poses and terrain geometry. Real robots need a vision-driven kinematic planner in front of it.

- Retargeting is unsolved. Adapting motion across different body proportions still trades runtime speed against quality. If your character is not human-proportioned, expect work.

- Dataset coverage. The authors call 350,000 motions "small" with limited coverage of rare motion types. Unusual or highly stylized motion is where it will thin out.

- Linux and X11 only, in practice. The demo uses a keyboard key-grab workaround that conflicts on Wayland, macOS, and Windows.

- Crawling modes lack side-only directions. A small but documented hole in the locomotion coverage.

- NVIDIA GPU required. CUDA only. There is no Apple Silicon or AMD path.

## Who should use it, and who should wait

Use it now if you are a technical animator or gameplay engineer who wants to prototype what a graph-free character controller feels like, or a robotics researcher already working with a Unitree G1 who wants a motion prior instead of per-skill policies. Also use it if you are writing a paper: the architecture is small, the code is Apache 2.0, and public benchmarks are supported.

Wait if you need a production engine integration, if you need to retrain on your own motion library and cannot clear a BONES-SEED license, or if you need physically guaranteed output for hardware. And skip it entirely if you are not doing character motion. This is a narrow, deep tool, not a general model.

## Run the demo in about 10 minutes

You need a CUDA GPU, Python 3.10 or newer, and Git LFS. Budget 2.2 GB for the checkpoints. These commands are transcribed from the NVLabs README and are not independently tested here.

```

# Git LFS first, or the checkpoints arrive as pointer files
git lfs install

git clone https://github.com/NVlabs/GR00T-WholeBodyControl.git
cd GR00T-WholeBodyControl

# Pull only what the demo needs (~2.2 GB), not the whole repo's LFS objects
git lfs pull --include="motionbricks/out/**" --exclude=""
git lfs pull --include="motionbricks/assets/skeletons/g1/meshes/**" --exclude=""

cd motionbricks
conda create -n motionbricks python=3.10 -y
conda activate motionbricks
pip install -e .
pip install pynput python-xlib # Linux only, for the keyboard grab

```

Then launch the interactive G1 demo. WASD moves, the mouse controls the camera, and V/Z/X/B/R/T/C/E/F/G/Q switch motion styles at runtime. Switching styles mid-stride is the thing to try first, because that transition is exactly what you would otherwise hand-author.

```

DISPLAY=:1 python scripts/interactive_demo_g1.py

```

If you want to go past the demo, the training pipeline is three scripts run in order. The tokenizer comes first because both transformers consume its codebook:

```

# 1. Motion tokenizer (VQ-VAE). Everything downstream needs this.
python scripts/train_vqvae.py

# 2. Pose model, conditioned on the tokenizer's codebook
python scripts/train_pose.py

# 3. Root model (timing plus trajectory)
python scripts/train_root.py

# Note: these default to SYNTHETIC data. Real training needs your own
# corpus or a cleared BONES-SEED license.

```

Short on GPU? The ten-minute version is the [project page](https://nvlabs.github.io/motionbricks/), which has video of the style transitions and the smart-object interactions. Watch the stylized gait switching and the object approach fill-in, then read section 4 of the paper on the multi-head quantization. That trick, splitting the latent across parallel codebooks, is the reason a 224M-parameter model covers 350,000 skills, and it generalizes well beyond motion.

## Sources and further reading

- [MotionBricks project page (NVLabs)](https://nvlabs.github.io/motionbricks/)

- [MotionBricks paper on arXiv (2604.24833)](https://arxiv.org/abs/2604.24833)

- [ACM Transactions on Graphics, SIGGRAPH 2026](https://dl.acm.org/doi/10.1145/3811334)

- [NVlabs/GR00T-WholeBodyControl on GitHub](https://github.com/NVlabs/GR00T-WholeBodyControl/tree/main/motionbricks)

- [BONES-SEED dataset on Hugging Face (gated)](https://huggingface.co/datasets/bones-studio/seed)

Tested on: not independently tested. MotionBricks needs a CUDA GPU, which is beyond our bench, and our standing policy is not to install AI tooling locally. All figures are as reported by the paper, the repository, and the project page; the 15,000 FPS and 2 ms latency numbers are NVIDIA-measured on an RTX 5090 with no independent reproduction. MotionBricks was still labelled a preview release at the date below.

Date checked: 2026-08-11

---


# Muse Glimmer 30B
Source: https://singularitybyte.com/models/meta-muse-glimmer-30b-local-agent-model.html

Meta released Muse Glimmer on August 10, 2026: a 30-billion-parameter open-weight model built for one job, running agents on your own machine. The license is Apache 2.0, not a Llama-style community agreement, and the quantized weights fit in under 20 GB. That combination, a real open license plus single-GPU agentic performance, is what makes this release worth your afternoon. Here is what shipped, where it wins, where Qwen still beats it, and how to have it running before your coffee goes cold.

## Muse Glimmer 30B: a local agent model with a real open license

Muse Glimmer is a 30B dense model. Every parameter activates for every token, so there is no Mixture-of-Experts routing (the technique where only a slice of a much larger network fires per token). Dense costs more compute per token than MoE, but it also means predictable memory use and no expert-routing overhead, which matters when the whole model has to live on one consumer GPU.

The model was not trained from scratch at this size. Meta distilled it from Muse Spark, its larger flagship, using logit distillation: the small model learns to match the big model's full output distribution instead of just its final answers. It also carries a dedicated perception encoder for image input, a context window of 120K+ tokens, and coverage of 100+ languages.

The headline for builders is the license. Apache 2.0 means commercial use, modification, and redistribution with no acceptable-use policy, no monthly-active-user cutoff, and no lawyer required. After years of "open-ish" Llama community licenses, Meta shipping a straight Apache 2.0 model is the actual news here.

## How a 30B model fits in under 20 GB

Full-precision Muse Glimmer needs 55+ GB of memory, which rules out every consumer card. Meta's answer is two official quantized variants, K-Quant-Dynamic and K-Quant-17GB, that compress the weights to roughly 4-bit precision and land under 20 GB with what Meta describes as minimal accuracy loss. That fits a 24 GB card like an RTX 3090 or 4090, or unified memory on an Apple Silicon Mac.

Speed comes from DFlash, a small companion drafter model for speculative decoding. The drafter guesses several tokens ahead and the main model verifies the batch in one pass, so you get identical output faster. Meta reports 3.1x faster generation on an RTX 5090, 1.8x on an M5-Max MacBook, and 1.5x on an M4-Max. NVIDIA separately reports 20+ tokens per second per GPU at BF16/NVF4 precision on its Blackwell Ultra hardware.

## Benchmarks: wins the agent tests, not everything

Meta compared Muse Glimmer against the two open-weight models in its size class, Gemma 4-31B and Qwen3.6-27B. These are Meta-reported numbers from the launch post; we have not independently tested them, and you should read them the way you read any vendor deck.

Benchmark (Meta-reported) | Muse Glimmer 30B | Gemma 4-31B | Qwen3.6-27B
MCP-Atlas (tool calling) | 75.5 | 54.2 | 62.5
DeepSearch QA (agentic search) | 74.6 | 61.7 | 71.1
AIME 2026 (math) | 94.7 | 89.2 | 94.1
SWE-Bench Verified (coding) | 76.0 | n/a | 77.2
The pattern is clear and Meta, to its credit, publishes it. Muse Glimmer wins the agentic benchmarks, tool calling by 13 points over its nearest rival, and agentic search by 3.5. On raw coding it does not: Qwen3.6-27B edges it on SWE-Bench Verified and also leads on OSWorld-Verified and TerminalBench 2.1 in Meta's own table. Meta additionally reports strong results on tau-Bench, its published agent-workflow suite.

So the honest summary: if your workload is an agent loop, browsing, searching, calling MCP tools, recovering from failures, this is now the model to beat at 30B. If your workload is pure code generation, [GLM-5.2](/models/glm-5-2.html) class coders and Qwen3.6 still have the edge.

## Get it running in under 10 minutes

Launch-day ecosystem support is unusually complete. Official weights, an official GGUF conversion, and an ExecuTorch build are on Hugging Face, Unsloth has its usual quant spread, and there is a first-party Ollama library entry. If you have [Ollama v0.32 or newer](/news/ollama-v0-32-interactive-agent-2026.html), this is the whole install:

```
ollama run muse-glimmer
```

For llama.cpp or LM Studio, pull a GGUF directly. The Unsloth repo carries the full quant ladder if you want something smaller than Meta's official 4-bit:

```
# official Meta GGUF
huggingface-cli download meta-models/Muse-Glimmer-30B-GGUF

# or Unsloth's quant spread
huggingface-cli download unsloth/Muse-Glimmer-30B-GGUF
```

Mac users get MLX support at launch, so LM Studio on an M-series machine works day one; Meta's own test hardware was an M4-Max and M5-Max. If you are serving the model rather than chatting with it, vLLM and SGLang recipes shipped alongside the release, and NVIDIA has a prebuilt NIM container plus NeMo AutoModel support for fine-tuning with SFT, LoRA, and RL. We covered the same local-first release pattern when [Kimi K3 GGUFs landed](/news/run-kimi-k3-locally-unsloth-gguf.html), but at 594 GB that one was a server toy. This one actually fits.

## Limitations and gotchas

A few things to know before you rearrange your stack. First, the 20 GB figure is the quantized model; full BF16 needs 55+ GB, so serious fine-tuning still wants workstation or cloud hardware. Second, the DFlash speedups assume you run the drafter alongside the main model, which costs extra memory; budget for that on a 24 GB card. Third, cards below 16 GB are out of luck locally, though Together AI, Fireworks, and OpenRouter all have hosted endpoints live.

And the benchmark caveat bears repeating: every number above is vendor-run. Independent replications will land within days for a release this big. We would treat the agentic wins as directionally real, since the margins are wide, and the coding parity claims as wait-and-see.

## Who should use it, and what comes next

Use it now if you are building local agent workflows: MCP tool servers, research agents, browser automation, anything with a loop and a failure mode. A 30B Apache 2.0 model that wins tool-calling benchmarks and runs on a 4090 is exactly the gap the open ecosystem had, sitting between small models like [DeepSeek-V4 Flash](/models/deepseek-v4-flash.html) and the 100B+ giants nobody runs at home.

Skip it if you mainly need a code-completion model, where Qwen3.6 is cheaper to run at 27B and slightly better, or if you need vision output rather than vision input.

What comes next is the bigger story. Meta says an open release of Muse Spark 1.2, the teacher model Glimmer was distilled from, is on the roadmap. Zuckerberg framed the release around distributing capable models widely rather than centralizing them, and whatever you think of the framing, Apache 2.0 weights are the receipts. If Spark ships under the same license, the open-weight landscape shifts again.

Your under-10-minute move: run ollama run muse-glimmer, point it at an MCP server you already use, and see whether that 75.5 tool-calling score survives contact with your own stack.

## Sources and further reading

- [Meta: Introducing Muse Glimmer](https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model)

- [Muse Glimmer 30B on Hugging Face](https://huggingface.co/meta-models/Muse-Glimmer-30B)

- [Unsloth GGUF quantizations](https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF)

- [Muse Glimmer on Ollama](https://ollama.com/library/muse-glimmer)

- [NVIDIA: Muse Glimmer local agent workflows](https://developer.nvidia.com/blog/run-local-agentic-ai-workflows-with-metas-muse-glimmer-on-nvidia/)

- [Constellation Research: Muse Spark 1.2 on tap](https://www.constellationr.com/insights/news/meta-releases-open-weight-muse-glimmer-model-open-muse-spark-12-tap)

Tested on: not independently tested. All benchmark and performance figures are Meta- or NVIDIA-reported from the launch materials linked above.

Date checked: 2026-08-10

---


# DeepSeek-V4-Flash
Source: https://singularitybyte.com/models/deepseek-v4-flash.html

On July 31, 2026, DeepSeek promoted DeepSeek-V4-Flash out of preview. The official build, tagged V4-Flash-0731, keeps the exact same 284B-parameter architecture we covered in [our April V4 Preview breakdown](/models/deepseek-v4.html) and changes only one thing: the post-training. That one thing was apparently the whole story. On every one of the nine agent and coding benchmarks DeepSeek published, the retrained Flash now beats not just its own preview but V4-Pro-Preview, the 1.6T flagship it was supposed to sit under. Weights are on Hugging Face under MIT, API pricing stays at $0.14 in and $0.28 out per million tokens, and the model name is simply deepseek-v4-flash.

## TL;DR

- What it is: the official release of DeepSeek's small V4 variant, a 284B Mixture-of-Experts model with 13B active parameters and a native 1M-token context, re-post-trained for agents and released under MIT on [Hugging Face](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731).

- Why it matters: on DeepSeek's own benchmark table, the 0731 build beats V4-Pro-Preview everywhere, and third-party [Artificial Analysis](https://artificialanalysis.ai/models/deepseek-v4-flash) puts it one point behind OpenAI's budget GPT-5.6 Luna at roughly 60% lower cost per task.

- The catch: 284B total parameters still means server-class hardware for self-hosting, the preview's high hallucination rate has no published 0731 re-measurement yet, and the API is labeled public beta.

## What Actually Landed in DeepSeek-V4-Flash-0731

The [official changelog](https://api-docs.deepseek.com/updates/) is unusually direct about what changed: nothing in the architecture. "DeepSeek-V4-Flash-0731 keeps the same model architecture and size as DeepSeek-V4-Flash-Preview, and was only re-post-trained." Same hybrid CSA plus HCA attention, same FP4 expert weights, same 1M context. If you want the architecture tour, read [the April article](/models/deepseek-v4.html); all of it still applies.

Three things are genuinely new:

- A full agentic retraining pass. The gains below came entirely from post-training, which is a quiet argument that open-weight labs still have large headroom without touching pretraining budgets.

- Native Responses API support. The changelog notes the model "natively supports the Responses API format and is specifically adapted for Codex," so it drops into OpenAI-shaped agent harnesses without adapter glue.

- The DSpark checkpoint. The 0731 release ships with a [speculative-decoding module attached](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark), which is also why the Hugging Face parameter counter reads 304B instead of 284B: the extra ~20B is the draft module, not the model growing.

## Benchmarks: The Small Model Ate the Big One

All numbers below are DeepSeek-reported, from the [0731 model card](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731). We have not run these ourselves.

Benchmark (DeepSeek-reported) | V4-Flash-0731 | V4-Flash-Preview | V4-Pro-Preview | Opus 4.8
Terminal Bench 2.1 | 82.7 | 61.8 | 72.1 | 85.0
NL2Repo | 54.2 | 39.4 | 38.5 | 69.7
Cybergym | 76.7 | 38.7 | 52.7 | 83.1
DeepSWE | 54.4 | 7.3 | 12.8 | 58.0
Toolathlon-Verified | 70.3 | 49.7 | 55.9 | 76.2
Read the DeepSWE row twice. The preview scored 7.3. The same architecture, after retraining, scores 54.4. The changelog adds four more agent evals in the same pattern: DSBench-FullStack 68.7, DSBench-Hard 59.6, Toolathlon verified 70.3, and the still-hard tail of Agent Last Exam at 25.2 and Automation Bench (Public) at 25.1. Across all nine published benchmarks the 0731 build lands above V4-Pro-Preview, which makes the current DeepSeek lineup mildly absurd: the cheap model is the good one until V4-Pro gets the same retraining pass.

Third-party numbers point the same direction. [The Decoder reports](https://the-decoder.com/new-deepseek-flash-model-matches-openais-gpt-5-6-luna-at-roughly-60-percent-lower-cost/) an Artificial Analysis Intelligence Index of 50 versus 51 for GPT-5.6 Luna, "about 60 percent less per task, even after OpenAI's 80 percent price cut," a GDPval jump from 1,189 to 1,559 Elo, and 12% fewer tokens used than the preview. For context, when we covered the preview in April, V4-Pro led the open-weights GDPval board at 1,554. The Flash just walked past that too.

## Pricing: Still the Cheapest Seat at the Table

API pricing is unchanged from the preview, per the [DeepSeek platform docs](https://api-docs.deepseek.com/updates/):

Model | Input (cache miss) | Input (cache hit) | Output
deepseek-v4-flash | $0.14 / 1M | $0.0028 / 1M | $0.28 / 1M
The cache-hit price is a 98% discount, which matters more than it looks for agents: long-horizon loops re-read the same context constantly, so a well-structured agent pays the cache-hit rate for most of its input tokens. Combine that with the 1M-token window and the KV-cache compression from the V4 architecture, and this stays the cheapest way to run serious long-context agent workloads on someone else's GPUs.

## What It Takes to Run It Yourself

Honesty section: 13B active parameters does not mean 13B-sized hardware. The full FP4 plus FP8 checkpoint still wants a single big-memory accelerator or a small tensor-parallel group.

Setup | What you need | Notes
Hosted API | An API key | Public beta, OpenAI-compatible plus Responses API
Full weights, vLLM | 1x GB300-class (288GB) or 4x large GPUs | Official [vLLM recipe](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4-Flash), vLLM 0.20.0+, fused checkpoints need 0.25.0
4-bit community quants | ~80GB+ VRAM (H200, or 96GB+ unified memory) | Quality unverified, FP4-trained experts compress poorly below Q4
The vLLM single-GPU config from the official recipe is a useful reference for the moving parts (FP8 KV cache, the V4 tokenizer mode, and MTP speculative decoding at 3 draft tokens):

```

vllm serve deepseek-ai/DeepSeek-V4-Flash \
--tensor-parallel-size 1 --kv-cache-dtype fp8 \
--trust-remote-code --gpu-memory-utilization 0.92 \
--tokenizer-mode deepseek_v4 --tool-call-parser deepseek_v4 \
--enable-auto-tool-choice --reasoning-parser deepseek_v4 \
--speculative-config '{"method":"mtp","num_speculative_tokens":3}'

```

## Hands-On: Point Your Agent at It in Under 10 Minutes

The API is OpenAI-compatible, so switching an existing agent stack is a one-line change:

```

from openai import OpenAI

client = OpenAI(
api_key="YOUR_DEEPSEEK_API_KEY",
base_url="https://api.deepseek.com/v1",
)

response = client.chat.completions.create(
model="deepseek-v4-flash",
messages=[
{"role": "user", "content": "Read this repo and propose a fix plan."},
],
)
print(response.choices[0].message.content)

```

For self-hosting, grab the weights and follow the vLLM recipe above:

```

pip install -U "huggingface_hub[cli]"
huggingface-cli download deepseek-ai/DeepSeek-V4-Flash-0731

```

If you are on the Codex or Responses API side of the fence, the 0731 build is explicitly tuned for that harness, which is the first time DeepSeek has shipped a model adapted for a specific third-party agent frontend.

## Limitations and Gotchas

- Hallucination is an open question. The preview measured a 96% hallucination rate on Artificial Analysis's AA-Omniscience eval, among the worst of its peers. Nobody has published a 0731 re-measurement yet. Until someone does, keep retrieval grounding in fact-heavy pipelines.

- No Jinja chat template. Prompt encoding still goes through DeepSeek's Python encoder scripts in the repo. Day-one preview users tripped on this; nothing changed.

- The agent ceiling is real. Beating V4-Pro-Preview is impressive, but Agent Last Exam at 25.2 and Automation Bench at 25.1 are absolute scores. Long-horizon autonomy is still mostly unsolved, for everyone.

- Param-count confusion. Hugging Face shows 304B for the 0731 repo because the DSpark speculative-decoding module ships attached. The model itself is unchanged at 284B.

- Public beta API. DeepSeek labels the 0731 API a public beta, so expect endpoint behavior to shift.

## Who Should Use It

If you run coding or tool-use agents against a hosted API and your bill matters, this is now the default open-weight choice: near-Luna scores at 40% of the price, MIT weights as your exit hatch if the API terms ever change. That exit hatch is not theoretical; we made [the case for open-weight fallbacks](/news/anthropic-export-ban-case-for-open-weight-ai-2026.html) when a US export order switched off a closed frontier model overnight. If you self-host, V4-Flash on a single H200-class card with quantization remains the practical play, same as the preview. And if you were waiting on [the V3 line](/models/deepseek-v3-0324.html) to get a proper successor for agent work, this is it.

What to watch next: a V4-Pro build with the same post-training recipe. If retraining alone took Flash from 7.3 to 54.4 on DeepSWE, the same pass over the 1.6T model is the obvious next shoe to drop.

## Sources and Further Reading

- [deepseek-ai/DeepSeek-V4-Flash-0731 on Hugging Face (weights, benchmark table)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-0731)

- [DeepSeek-V4-Flash-DSpark (speculative-decoding checkpoint)](https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-DSpark)

- [DeepSeek official release announcement (V4-Flash-0731)](https://api-docs.deepseek.com/news/news260731)

- [DeepSeek API changelog (official release notes and benchmark list)](https://api-docs.deepseek.com/updates/)

- [DeepSeek-V4 technical report (arXiv)](https://arxiv.org/abs/2606.19348)

- [vLLM serving recipe for DeepSeek-V4-Flash](https://recipes.vllm.ai/deepseek-ai/DeepSeek-V4-Flash)

- [Artificial Analysis: DeepSeek V4 Flash 0731 analysis](https://artificialanalysis.ai/models/deepseek-v4-flash)

- [The Decoder: Flash matches GPT-5.6 Luna at 60% lower cost](https://the-decoder.com/new-deepseek-flash-model-matches-openais-gpt-5-6-luna-at-roughly-60-percent-lower-cost/)

- [SingularityByte: DeepSeek-V4 Preview breakdown (April launch, full architecture tour)](/models/deepseek-v4.html)

Tested on: not independently benchmarked. Every score in this article is DeepSeek-reported or third-party (Artificial Analysis, The Decoder) as labeled; we have not run the 0731 build locally.

Date checked: 2026-08-01

---


# Inkling
Source: https://singularitybyte.com/models/thinking-machines-inkling.html

On July 15, 2026, Thinking Machines Lab shipped Inkling: 975 billion parameters, Apache 2.0, weights on Hugging Face on day one. That makes it the largest open-weights model ever released by a US lab, and the first foundation model from the company Mira Murati founded after leaving OpenAI. It reasons natively over text, images, and audio, carries a 1-million-token context window, and exposes a thinking effort dial you can turn from 0.2 to 0.99. And in a refreshing break from launch-day tradition, the lab's own announcement admits it is not the strongest model available. Here is what Inkling actually is, what it scores, and how to run or fine-tune it today.



## TL;DR




- What it is: a 975B-parameter Mixture-of-Experts model with 41B active per token, multimodal input (text, image, audio), 1M-token context, trained on 45T tokens, released under a clean Apache 2.0 license with weights on Hugging Face at launch.

- Why it matters: it is the US answer to the Chinese open-weight wave, pitched honestly as the best open base for customization rather than a leaderboard king, with a token-saving effort dial and fine-tuning via the Tinker API.

- The catch: you need roughly 600 GB of VRAM even at 4-bit, llama.cpp support is still sitting in an unmerged PR, and on raw benchmarks it trails the closed frontier by a wide margin.



## What Thinking Machines actually shipped


Thinking Machines Lab launched in early 2025 with a large chunk of OpenAI's former leadership and a product, Tinker, that sells managed fine-tuning rather than chat subscriptions. Inkling is the model that strategy was waiting for: a big, permissively licensed base that customers can specialize on Tinker and then run anywhere. Two models were announced. Inkling itself, 975B total and 41B active, is downloadable now as a BF16 checkpoint plus an NVFP4 build for Blackwell systems. Inkling-Small, a 276B model with 12B active, was previewed for a later release and will be the one most of us can realistically host.


The positioning is the unusual part. The announcement states plainly that "Inkling is not the strongest overall model available today, open or closed" and sells it instead as a broad, honest base for customization. After a year of launch posts claiming frontier supremacy from every direction, a vendor leading with its own limitations is worth noticing.



## A DeepSeek-style recipe with real twists


A Mixture-of-Experts (MoE) model splits its feed-forward layers into many expert subnetworks and routes each token through only a few, so you hold 975B parameters on disk but pay inference for 41B. Inkling follows the recipe popularized by [DeepSeek-V4](models/deepseek-v4.html) and [GLM-5.2](models/glm-5-2.html): 66 layers, 256 routed experts plus 2 shared, 6 routed experts active per token, sigmoid routing with auxiliary-loss-free load balancing.


Then it deviates. There is no RoPE: Inkling uses a learned relative-position bias, which the lab claims extrapolates better to long sequences. Attention interleaves 512-token sliding-window layers with global layers at a 5:1 ratio. Short convolutions sit after the key and value projections, an inductive-bias trick you rarely see at this scale. And the multimodal path is encoder-free: images enter as raw 40x40 pixel patches through a four-layer MLP and audio as spectrogram tokens, no bolted-on vision tower. Sebastian Raschka's [architecture notes](https://sebastianraschka.com/blog/2026/inkling-architecture-benchmark-notes.html) are the best independent walkthrough of these choices, including a caution: with conventional grouped-query attention instead of DeepSeek-style MLA, raw decoding speed is probably not Inkling's advantage.



## Benchmarks, with the usual asterisk


All numbers below are first-party, reported at effort 0.99, and not reproduced on our bench. The comparison columns come from Thinking Machines' own model card. What stands out is not any single score but the shape: solid everywhere, dominant nowhere, and honest about the gap to the closed frontier.





Benchmark (reported)
Inkling
Claude Fable 5
GPT-5.6 Sol
SWE-Bench Verified
77.6%
95.0%
82.2%
Terminal Bench 2.1
63.8%
not listed
not listed
AIME 2026
97.1%
99.9%
99.9%
GPQA Diamond
87.2%
not listed
not listed
HLE (text only)
29.7%
53.3%
47.2%
SimpleQA Verified
43.9%
68.3%
71.6%
MMMU Pro (vision)
73.5%
not listed
not listed
VoiceBench (audio)
91.4%
not listed
not listed
All figures are self-reported by Thinking Machines at launch. Treat them as vendor claims until independently reproduced.


Within the open-weight field the picture is friendlier. 77.6% on SWE-Bench Verified clears Nvidia's [Nemotron 3 Ultra](models/nvidia-nemotron-3-ultra.html) at 71.9%, and the 91.4% VoiceBench score is unusual for a model whose audio path has no dedicated encoder. The clear weak spot is factual recall: 43.9% on SimpleQA means you want retrieval in front of it for anything knowledge-heavy. Against [Kimi K3](models/kimi-k3.html), announced the very next day at 2.8T parameters, Inkling loses the size contest but wins on availability: its weights are downloadable today, while Moonshot's are promised for July 27.



## The effort dial is the actual feature


Most reasoning models give you a binary: thinking on or off. Inkling exposes thinking effort as a continuous parameter from 0.2 to 0.99, and the lab's own charts show it matching Nemotron 3 Ultra on Terminal Bench 2.1 while spending roughly a third of the tokens at lower effort settings. For agent pipelines, that is a budget knob you control per request instead of a model-picker decision. Turn it down for routing and classification calls, up for the hard synthesis step, all against one deployed checkpoint. If the numbers hold up in independent testing, this is the feature other open labs copy next.



## Run it in about 10 minutes


Be honest with yourself about hardware first. The BF16 checkpoint wants about 2 TB of aggregated VRAM (think 16x H200), and even the NVFP4 build needs 600 GB or more across at least four Blackwell-class GPUs. This is not a laptop model. It is barely a single-server model. If you have the metal, vLLM works out of the box:



```
# Download the weights (BF16 ~2TB; NVFP4 for Blackwell systems)
huggingface-cli download thinkingmachines/Inkling

# Serve with vLLM on a multi-GPU node
vllm serve thinkingmachines/Inkling \
--tensor-parallel-size 8 \
--max-model-len 262144

# Community GGUF quants exist (unsloth/inkling-GGUF), but llama.cpp
# support is still in PR #25731. Until it merges, build from that PR
# and always pass --jinja so the chat template is applied.

```


Everyone else calls it hosted. Together, Fireworks, Modal, Databricks, and Baseten all serve Inkling, and any OpenAI-compatible client works:



```
# Inkling via Together's OpenAI-compatible endpoint.
from openai import OpenAI

client = OpenAI(
api_key="...", # your Together key
base_url="https://api.together.xyz/v1",
)

resp = client.chat.completions.create(
model="thinkingmachines/Inkling",
messages=[{"role": "user",
"content": "Summarize this repo's auth flow and list its weak points."}],
)
print(resp.choices[0].message.content)

```


Fine-tuning is the intended path, though: Inkling is live on Thinking Machines' Tinker platform with 64K and 256K context options at a 50% launch discount, plus a free playground for a limited time. Simon Willison's [first look](https://simonwillison.net/2026/Jul/16/inkling/) runs the multimodal loop through the Tinker API: he had Inkling draw an SVG pelican on a bicycle, then describe its own rendering. It analyzed the image coherently, then called its pelican a stork. Frontier is a spectrum.



## Limitations and gotchas




- Hardware floor: 600 GB VRAM minimum at 4-bit, about 2 TB at BF16. Self-hosting means a GPU server, not a workstation. Wait for Inkling-Small if you want local.

- llama.cpp support is not mainline yet. The GGUF quants from Unsloth and others require building llama.cpp from PR #25731 and passing --jinja for the chat template.

- Benchmarks are launch-day and first-party. The honest framing earns goodwill, but the numbers still deserve independent reproduction.

- SimpleQA at 43.9% is weak factual recall for a model this size. Pair it with retrieval for knowledge work.

- Output is text only. It reads images and audio but will not generate them.

- Training data disclosure is thin. The card acknowledges "content that may be subject to intellectual property protection" without detail, a point Willison flags too.



## Who should use it


Use Inkling if you fine-tune. That is what it is for: a permissive Apache 2.0 base with multimodal input, a 1M context, and a managed tuning platform attached, from a US lab if your compliance people care about that. It also makes sense as a hosted workhorse for agent pipelines where the effort dial can cut your reasoning-token bill against a single deployment.


Hold off if you want the strongest general model (the lab itself points you elsewhere), if you need something you can run on a workstation today (watch for Inkling-Small, or grab [GLM-5.2](models/glm-5-2.html)), or if your workload is pure coding, where [Kimi K2.7 Code](models/kimi-k2-7-code.html) gives you more per active parameter. For everyone else, the real story is strategic: the US open-weights scene finally has a frontier-scale entrant, after a year in which [Qwen](models/alibaba-qwen-3-5.html), DeepSeek, and Moonshot made the open frontier look like a one-country race.



## Sources and further reading




- [Thinking Machines: Introducing Inkling (official announcement)](https://thinkingmachines.ai/news/introducing-inkling/)

- [Inkling model card (hardware requirements, full benchmark table)](https://thinkingmachines.ai/model-card/inkling/)

- [Hugging Face: thinkingmachines/Inkling (BF16 + NVFP4 weights)](https://huggingface.co/thinkingmachines/Inkling)

- [Hugging Face blog: Welcome Inkling](https://huggingface.co/blog/thinkingmachines-inkling)

- [Simon Willison: first look at Inkling](https://simonwillison.net/2026/Jul/16/inkling/)

- [Sebastian Raschka: Inkling architecture and benchmark notes](https://sebastianraschka.com/blog/2026/inkling-architecture-benchmark-notes.html)

- [Unsloth: Inkling GGUF quantizations](https://huggingface.co/unsloth/inkling-GGUF)


Tested on: not independently benchmarked. Inkling needs 600+ GB of VRAM even quantized, which is beyond our local bench. All benchmark figures are Thinking Machines' launch-day numbers; hardware and access details are drawn from the model card and the sources above.
Date checked: 2026-07-22

---


# Kimi K3
Source: https://singularitybyte.com/models/kimi-k3.html

On July 16, 2026, Moonshot AI shipped Kimi K3, and the ceiling for open-weight models moved again. It is a 2.8-trillion-parameter Mixture-of-Experts model with a 1-million-token context window, and the full weights landed on Hugging Face on July 27. In blind developer testing it took first place on the Frontend Code Arena leaderboard, ahead of Claude Fable 5. It costs $3 in and $15 out per million tokens. In the same week, Anthropic is moving Fable 5 to $10 in and $50 out and pulling it out of subscription plans. If you build with models, that gap is the story. Here is what K3 is, what it actually scores, and how to point your existing agent at it today.

## TL;DR

- What it is: a 2.8T-parameter Mixture-of-Experts model (activates 16 of 896 experts per token), 1M-token context, native vision, always-on reasoning. Weights are public as of July 27, 2026, under the custom Kimi K3 License.

- Why it matters: it beats or trails the top closed US models by a hair on reported benchmarks, at roughly a third of Fable 5's price, with the weights now out for anyone to run.

- The catch: the headline scores are first-party, and self-hosting 2.8T parameters is a data-center job. The released weights measure 1,561 GB.

## What Moonshot actually shipped

One definition first, because it drives the cost math. A Mixture-of-Experts (MoE) model splits its feed-forward layers into many small "expert" subnetworks and routes each token to only a few of them. K3 holds 2.8 trillion parameters on disk but fires only 16 of its 896 experts per token, so you get the capacity of a very large model at the inference cost of a much smaller one. That is the same bargain behind other big open MoEs like [DeepSeek-V4](models/deepseek-v4.html) and [Qwen3.5](models/alibaba-qwen-3-5.html).

It takes text and images, carries a 1-million-token context window aimed at long-horizon coding and agent runs, and ships with an always-on reasoning mode Moonshot calls thinking mode. Pricing through the Kimi API is $3.00 per million input tokens and $15.00 per million output tokens, dropping to $0.30 per million on cached input. This is the same team behind [Kimi K2.7 Code](models/kimi-k2-7-code.html) and [Kimi K2.5](models/moonshot-kimi-k2-5.html), now aiming a general flagship at the very top of the table.

## Benchmarks, with the usual asterisk

Every number below is reported by Moonshot or by third-party arenas at launch, not reproduced on our bench. Read them as a starting point, not a measurement. The pattern is consistent though: K3 sits in the same band as the best closed US models, and leads several of them on agentic coding.

Benchmark (reported)
Kimi K3
Claude Fable 5 Max
GPT-5.6 Sol Max
Claude Opus 4.8
GDPval-AA v2 (real-world work, 44 occupations)
1,687
1,815
1,747.8
1,600
AA-Briefcase (long-horizon agentic)
1,527
1,587
1,495
not reported
On GDPval-AA v2, a test of real tasks across 44 occupations, K3 placed third overall, behind Fable 5 Max and GPT-5.6 Sol Max but ahead of Claude Opus 4.8. On AA-Briefcase, a private long-horizon agentic benchmark from Artificial Analysis, it climbed to second, past GPT-5.6 Sol Max and behind only Fable 5 Max. The one that got the most attention: Arena ranked K3 first on Frontend Code at 1,679 points in blind developer voting, ahead of Fable 5. Its overall Coding Index came in at 76.24. Independent testers report K3 leading on Terminal-Bench 2.1 and long-horizon SWE tasks, while Fable 5 keeps the edge on several other coding suites. Call it a tie at the frontier, which is the point: an open-weight model is now trading blows with the most expensive closed ones.

## The part the leaderboards skip: freedom and price

Here is why builders are paying attention beyond the score. K3 has no classifier sitting between your call and the model, and no quiet routing to a weaker fallback. Developers comparing the two report that the model you call is the model you get. That is not marketing spin from Moonshot, it is a design difference you can feel on long agent runs.

Contrast that with how a refusal works on the closed side. Anthropic's own API documentation describes a stop_reason of refusal, and states plainly that on Claude Fable 5, safety classifiers return this stop reason as a normal HTTP 200 response, not an error. In practice that means a request can look like it succeeded while the work simply did not happen, and your error monitoring will not flag it unless you check the stop reason. For agent pipelines that run unattended, a silent refusal is worse than a loud one.

Then there is the bill. In the same window K3 launched, Anthropic is pulling included Fable 5 access for Pro, Max, and Team subscribers on July 19, and switching it to metered usage credits on July 20 at $10 per million input tokens and $50 per million output tokens. That is double the rate of Claude Opus 4.8 and the most expensive pricing Anthropic has ever listed for a generally available model. Max subscribers are not exempt. So the same week one lab put its frontier model behind a higher paywall, another put a comparable one in the open at a third of the price.

Model
Input / Mtok
Output / Mtok
Weights
Kimi K3
$3.00 ($0.30 cached)
$15.00
Open (Kimi K3 License), since July 27
Claude Fable 5
$10.00
$50.00
Closed
The trend under all of this is not subtle. The frontier you can actually own, run, fine-tune, and audit is increasingly being set in the open, and right now a lot of it is being set by Chinese labs: Moonshot, DeepSeek, Qwen, Z.ai. You do not have to cheer for any flag to notice that open weights change your options. When the model is yours, nobody reprices it out from under you or reroutes it mid-run.

## Run it in about 10 minutes

You do not have to wait for the weights to try K3. Point any OpenAI-compatible client at the Kimi API and you are calling it in one request.

```
# Kimi K3 via the Moonshot API (OpenAI-compatible endpoint).
# Get a key at platform.moonshot.ai, then:
export MOONSHOT_API_KEY="sk-..."

curl https://api.moonshot.ai/v1/chat/completions \
-H "Authorization: Bearer $MOONSHOT_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "kimi-k3",
"messages": [
{"role": "user", "content": "Build a single-file browser desktop: draggable windows, a taskbar, and a working clock. Vanilla HTML, CSS, and JS only."}
]
}'

```

Because the endpoint is OpenAI-compatible, you can also drop it straight into your existing coding agent. Set the base URL and model, keep everything else.

```
# Reuse any OpenAI-style SDK; just swap the base_url and model.
from openai import OpenAI

client = OpenAI(
api_key="sk-...", # your Moonshot key
base_url="https://api.moonshot.ai/v1",
)

resp = client.chat.completions.create(
model="kimi-k3",
messages=[{"role": "user",
"content": "Refactor this repo's auth module and add tests."}],
)
print(resp.choices[0].message.content)

```

A browser desktop clone, a full HTML mockup of an old operating system UI, a from-scratch web app: these long, single-shot build tasks are exactly where a 1M-token context and cheap output tokens pay off. Give K3 the whole spec in one prompt and let it run. The weights are public now, so the same jobs can move to your own hardware if you have the memory for it. Our [breakdown of the weights drop](news/kimi-k3-open-weights.html) has the real repo contents, the license terms and the hardware math.

## Limitations and gotchas

- The weights arrived on July 27, 2026 at moonshotai/Kimi-K3: 114 files, 1,561 GB, under a custom Kimi K3 License rather than the Modified MIT of earlier releases. It carries revenue-triggered conditions for Model-as-a-Service operators, so read it before building on it.

- Benchmarks are first-party or arena-reported, not independently reproduced. Treat the table above as a vendor and community claim.

- It is enormous. At 2.8T parameters, self-hosting is a multi-node, data-center job even after quantization. This is not a laptop model, and it will not be for a long time.

- If you use the hosted API rather than the weights, you are calling a China-hosted endpoint. For sensitive code or data, read the Kimi terms and route accordingly.

- No safety classifier cuts both ways. Great for uninterrupted agent runs, but you own more of the guardrail decisions yourself.

## Who should use it

Use K3 if you want frontier-class coding and agent performance without frontier pricing, you value calling a model that does not silently refuse or downgrade, and you want the option to run the weights yourself, which is now available. It is a strong fit for long agent jobs, large-repo work, and anyone whose bill just doubled when Fable 5 left their plan.

Hold off if you cannot field a multi-node cluster, since the released weights need roughly 19 H100s just to load, if you require independently reproduced benchmarks before you trust a number, or if your data cannot leave your jurisdiction and the hosted API is your only option for now. For a smaller open coding model you can actually run this week, [Kimi K2.7 Code](models/kimi-k2-7-code.html) or [GLM-5.2](models/glm-5-2.html) are the saner starting points.

## Sources and further reading

- [Moonshot: official Kimi K3 announcement](https://www.kimi.com/blog/kimi-k3)

- [VentureBeat: Moonshot releases Kimi K3](https://venturebeat.com/technology/chinas-moonshot-ai-releases-kimi-k3-the-largest-open-source-model-ever-rivaling-top-u-s-systems)

- [Tom's Hardware: 2.8T Kimi K3 tops Frontend Code Arena](https://www.tomshardware.com/tech-industry/artificial-intelligence/moonshot-releases-2-8-trillion-parameter-kimi-k3)

- [OpenRouter: Kimi K3 API and benchmarks](https://openrouter.ai/moonshotai/kimi-k3)

- [Anthropic docs: the refusal stop reason (HTTP 200 on Fable 5)](https://platform.claude.com/docs/en/build-with-claude/stop-reasons-and-fallback)

- [BleepingComputer: Fable 5 subscription access and the July 19 cutoff](https://www.bleepingcomputer.com/news/artificial-intelligence/claude-fable-5-stays-free-for-paid-users-until-july-19-as-anthropic-buys-more-time/)

- [Android Authority: Fable 5 metered credits at $10 / $50](https://www.androidauthority.com/anthropic-claude-fable-5-credits-usage-july-3684840/)

Tested on: not independently benchmarked. Kimi K3 is a 2.8T-parameter MoE that requires multi-node inference, which is beyond our local bench. Open weights were published 2026-07-27. Every benchmark here is Moonshot-reported or arena-reported; pricing and access details are drawn from the sources linked above.
Date checked: 2026-07-18. Updated 2026-07-27 when the open weights were published.

---


# LingBot-Map
Source: https://singularitybyte.com/models/lingbot-map.html

Point a camera at the world, and LingBot-Map builds the 3D scene while you are still filming. Released in late April 2026 by Robbyant, the robotics arm of Ant Group, this Apache 2.0 model turns an ordinary RGB video stream into camera poses and a dense point cloud in real time: about 20 frames per second, sequences beyond 10,000 frames, and roughly 13.3 GB of VRAM. No LiDAR, no depth sensor, no hours of COLMAP crunching. If you have ever wanted photogrammetry that keeps up with a moving robot, a drone, or just your phone on a walk, this is the most usable open release yet.



## Streaming 3D reconstruction from video, explained


Classic 3D reconstruction is an offline job. You record everything first, then a pipeline like COLMAP or a batch model like VGGT chews on the full set of frames at once. That works for scanning a statue, but it is useless for a robot that needs to know where the wall is right now.


Streaming reconstruction flips the constraint: the model sees frames one at a time, in order, and must output geometry and camera pose immediately without ever revisiting old frames. The classic answer is SLAM (simultaneous localization and mapping), which tracks handcrafted feature points and runs constant optimization loops. LingBot-Map replaces that whole machinery with a single feed-forward transformer: frames go in, poses and points come out, no iterative optimization anywhere.


The catch that killed earlier attempts is drift. Streaming models like Spann3R or CUT3R accumulate small pose errors that compound over minutes of video, until the reconstruction folds in on itself. LingBot-Map's paper is essentially one long answer to the drift problem.



## How the Geometric Context Transformer works


The architecture is called a Geometric Context Transformer (GCT), and it is a pure autoregressive model, the same generation-one-step-at-a-time regime your LLM uses, applied to geometry. Each incoming frame attends to three kinds of context instead of one big history buffer.


First, an anchor context pins the global coordinate frame, so the scene does not slowly rotate out from under itself. Second, a pose-reference window holds the most recent frames for dense geometric cues, the local detail. Third, a trajectory memory keeps a compact record of where the camera has been, which is what corrects long-range drift when you loop back to a place you have already seen.


Because the state stays compact and attention runs over a paged KV cache (the same key-value caching trick that makes LLM serving fast), per-frame cost stays flat instead of growing with video length. The paper reports the bounded context cuts per-frame compute by roughly 80x versus keeping full history, which is how a 25,000-frame indoor walkthrough demo stays stable. Training covered 29 datasets across synthetic and real scenes.



## Benchmarks: paper-reported, but the gap is not subtle


All numbers below come from the [LingBot-Map paper](https://arxiv.org/abs/2604.14141), so treat them as vendor-reported until the community reproduces them. The team did release the full evaluation pipeline for KITTI, Oxford Spires, VBR, TUM-D, 7-Scenes, ETH3D, Tanks and Temples, and NRGBD, which makes checking their homework unusually easy.


On Oxford Spires with 320-frame sequences, LingBot-Map reports an absolute trajectory error (ATE, how far the estimated camera path deviates from ground truth) of 6.42 m at 20.29 FPS. The streaming competition is not close: CUT3R lands at 18.16 m and Wint3R at 21.10 m. It also beats the offline batch models VGGT (24.78 m) and DA3 (12.87 m), which get to see every frame at once.





Method
Type
Oxford Spires ATE (m)
ETH3D F1
FPS
LingBot-Map
Streaming
6.42
98.98
20.3
Wint3R
Streaming
21.10
77.28
3.9
CUT3R
Streaming
18.16
n/p
29.2
DA3
Offline
12.87
n/p
n/a
VGGT
Offline
24.78
n/p
n/a
All figures paper-reported (arXiv:2604.14141), not independently reproduced. "n/p" means not published in the paper's headline tables. Offline methods process the whole sequence at once, so FPS is not comparable.


The long-sequence test is the one that matters for real use. Stretched to 3,840 frames on Oxford Spires, LingBot-Map's error grows by just 0.69 m to 7.11 m, while CUT3R and Wint3R blow out past 32 m. Reconstruction quality (F1, the balance of accuracy and completeness of the recovered surface) tells the same story: 98.98 on ETH3D versus 77.28 for the runner-up, plus wins on 7-Scenes (80.39) and NRGBD (64.26).



## Run it in about 10 minutes


You need an NVIDIA GPU with around 14 GB of free VRAM and CUDA 12.8 drivers; a used RTX 3090 or any 4090-class card clears the bar. There is no Ollama route here, this is a Python research repo, but an unusually tidy one. If you are still deciding what box to build, our [local AI hardware guide](news/local-ai-computing-exploring-nvidia-dgx-spark-apple-m4-max-mac-studio-amd-ryzen-ai-max-395.html) covers the trade-offs.



```
# 1. Environment (Python 3.10 + PyTorch 2.8 with CUDA 12.8)
conda create -n lingbot-map python=3.10 -y
conda activate lingbot-map
pip install torch==2.8.0 torchvision==0.23.0 \
--index-url https://download.pytorch.org/whl/cu128

# 2. Clone and install, with the browser viewer extras
git clone https://github.com/Robbyant/lingbot-map.git
cd lingbot-map
pip install -e ".[vis]"
pip install flashinfer-python # recommended, big speedup

# 3. Grab a checkpoint (HF: robbyant/lingbot-map)
# lingbot-map-long is the one you want for real videos.

# 4. Reconstruct the bundled example scene
python demo.py --model_path /path/to/lingbot-map-long.pt \
--image_folder example/courthouse --mask_sky

```


The demo spins up a viser viewer at http://localhost:8080, and you can watch the point cloud grow frame by frame in your browser. Four example scenes ship in the repo (courthouse, university, a loop-closure trajectory, and an outdoor Oxford sequence). For your own footage, pass --video_path video.mp4 instead of an image folder, and add --mask_sky outdoors so the sky does not smear into the geometry (it pulls a small ONNX segmentation model on first run, so pip install onnxruntime first).


Two flags do most of the tuning work. --keyframe_interval 2 halves KV-cache memory on long videos, and --mode windowed --window_size 128 is mandatory past roughly 3,000 frames, resetting state in overlapping chunks so drift never accumulates. Tight on VRAM? --offload_to_cpu trades speed for headroom.



## Limitations and gotchas


First, the hard requirement: CUDA 12.8 on NVIDIA silicon. There is no Apple Silicon or ROCm path today, though a community [Mac desktop viewer](https://github.com/donalleniii/lingbot-desktop-mac) lets a Mac display a reconstruction streamed from a Linux GPU box.


Second, depth range is bounded by the training data distribution. Feed it aerial footage from 200 m up and the geometry gets vague, because the model never saw scenes at that scale. Third, quality degrades past about 320 frames of raw KV cache, so the keyframe and windowed modes are not optional extras for long captures, they are the operating manual. And the output is a point cloud with camera poses, not a textured mesh: for meshes you still hand the result to downstream tooling, or reach for a generative model like [Hunyuan3D-2](models/hunyuan3d-2.html) when the goal is an asset rather than a map.



## Who should use it


Robotics and embodied AI builders are the obvious audience, since real-time ego-motion plus geometry from one RGB camera is the input layer for navigation, and it slots naturally under world models like [NVIDIA Cosmos 3](models/nvidia-cosmos-3.html). Drone and scanning hobbyists get COLMAP-grade capture that finishes when the flight does. And 3D content pipelines can use the poses and points as fast initialization for NeRF or Gaussian splatting runs, the same way [LHM-1B](models/lhm-1b.html) shortcuts human reconstruction.


The 10-minute move: clone the repo, download lingbot-map-long, and run the courthouse example. Watching a building assemble itself in your browser at 20 FPS explains this release better than any benchmark table.



## Sources and further reading




- [LingBot-Map on GitHub](https://github.com/Robbyant/lingbot-map)

- [Model weights on Hugging Face](https://huggingface.co/robbyant/lingbot-map)

- [Model weights on ModelScope](https://modelscope.cn/models/Robbyant/lingbot-map)

- [Paper: Geometric Context Transformer for Streaming 3D Reconstruction (arXiv)](https://arxiv.org/abs/2604.14141)

- [Community Mac desktop viewer](https://github.com/donalleniii/lingbot-desktop-mac)


Tested on: not independently tested. LingBot-Map requires CUDA 12.8 on NVIDIA hardware, which we could not run for this article; all benchmark figures are paper-reported by the Robbyant team, and install steps are taken verbatim from the official repository.
Date checked: 2026-07-16

---


# Mistral Medium 3.5
Source: https://singularitybyte.com/models/mistral-medium-3-5.html

Mistral's middle child finally comes with weights. Mistral Medium 3.5, released April 29, 2026, is a 128-billion-parameter dense model you can download from Hugging Face today: no gate, no access form, no research-only clause. It posts 77.6% on SWE-bench Verified, takes text and images, carries a 256K context window, and ships under a modified MIT license that is free for everyone below a $20 million monthly revenue bar. If you skipped every "Mistral Medium" headline because the last one was API-only, this is the release worth reading about.



## What changed: Mistral's mid model is now open weights


The original Mistral Medium 3 (May 2025) was a closed, API-only model. No parameter count, no weights, and it is now deprecated, with retirement scheduled for August 31, 2026. Medium 3.5 replaces it and flips the distribution model: the full weights sit in the official [mistralai Hugging Face repo](https://huggingface.co/mistralai/Mistral-Medium-3.5-128B), in both FP8 (about 134 GB across three safetensors shards) and BF16 (about 250 GB). We opened the file tree and checked: tokenizer, config, LICENSE, all there, nothing gated.


That matters because Medium 3.5 is not a consolation-prize model. It is positioned for agentic and coding work, with a configurable per-request reasoning effort, and its official numbers land within a few points of the best open coding models on the planet.



## The Mistral naming decoder


Mistral's lineup names are a trap, and we fell into it ourselves while researching this piece. Here is the map so you don't have to reconstruct it:





Model
Size
License
Weights
[Mistral Small 3.1](models/mistral-small-3-1.html)
24B dense
Apache 2.0
Open
Mistral Medium 3 / 3.1
undisclosed
Proprietary
API only, deprecated
Mistral Medium 3.5
128B dense
Modified MIT
Open
[Mistral Large 3](models/mistral-large-3.html)
675B MoE (41B active)
Apache 2.0
Open
Yes, that means "Medium 3" and "Medium 3.5" sit on opposite sides of the open-weights line. If you see a blog calling Medium 3 an open model, close the tab.



## What it actually is


Medium 3.5 (version code 26.04) is a dense transformer: all 128B parameters are active on every token, unlike the Mixture-of-Experts design in Large 3. It is multimodal, taking text and images as input, with a vision encoder Mistral trained from scratch to handle variable image sizes and aspect ratios. The context window is 256K tokens, and you can dial reasoning effort up or down per request instead of choosing between separate instruct and reasoning variants.


Community reporting says Medium 3.5 merges what used to be separate specialist models (the Devstral coding line and Magistral reasoning line) into one set of weights. Mistral's own card does not state that, so treat it as plausible background, not fact.



## The license: MIT with a $20 million asterisk


The LICENSE file in the repo is standard MIT plus one carve-out, quoted directly: "You are not authorized to exercise any rights under this license if the global consolidated monthly revenue of your company (or that of your employer) exceeds $20 million." Above that bar, you email Mistral for a commercial license.


Practical reading: $20 million per month is roughly $240 million a year in revenue. Every indie hacker, startup, research lab, and mid-size company on earth is below it. You can use it commercially, fine-tune it, and redistribute it. It is still worth one critical note: within Mistral's own family, the smallest model (Small, Apache 2.0) and the biggest (Large 3, Apache 2.0) are more permissively licensed than this middle one. Mistral put the revenue cap exactly where the enterprise money is.



## Benchmarks: official numbers first


Mistral's model card states two headline results in plain text: 77.6% on SWE-bench Verified and 91.4% on the tau3-Telecom agentic benchmark. The rest of the card's comparisons ship as chart images, so exact MMLU and GPQA digits are not published as numbers anywhere we could verify.


How does 77.6% stack up? Here is the open-weights coding field as reported by community trackers and roundups (these cross-model numbers are community-reported, not Mistral's):





Model
SWE-bench Verified
License
Size
DeepSeek V4
80.6%
open
large MoE
Kimi K2.6
80.2%
open
large MoE
Mistral Medium 3.5
77.6% (official)
Modified MIT
128B dense
Qwen3.x 27B
77.2%
Apache 2.0
27B
The independent Artificial Analysis Intelligence Index scores Medium 3.5 at 30, second of 62 models tracked at the time of writing. Third-party API benchmarks measured about 102 tokens per second on La Plateforme, where pricing is $1.50 per million input tokens and $7.50 per million output.


The honest summary: the giant MoE models still hold the coding crown by about three points, but Medium 3.5 gets you within reach of them from a single-node, dense, self-hostable package.



## Run it locally, if your hardware can


Let's be direct: 128B dense is not a laptop model. Dense means every token streams all 128B weights, so your tokens per second are roughly memory bandwidth divided by model size. That is why MoE models fly on the same box while this one crawls. Mistral says it self-hosts on as few as 4 GPUs, which in practice means 80GB-class cards for the FP8 weights. The one measured community datapoint we found: two RTX PRO 6000 Blackwell cards (192GB total) running FP8 under vLLM deliver 26 to 35 tokens per second on prose and 37 to 43 on code. Below that tier, the community GGUF quants from [unsloth](https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF) are the realistic path:





Quant
File size
Runs on
UD-IQ2_XXS
34.9 GB
48GB Macs, 2x 24GB GPUs (quality hit)
Q2_K
46.6 GB
64GB unified memory
Q4_K_M
74.9 GB
128GB Mac Studio; Strix Halo 128GB and 4x 24GB GPUs fit it but run slow
Q5_K_M
88.3 GB
128GB unified memory
Q8_0
133 GB
192GB+ unified memory, multi-GPU servers
There is an official Ollama library entry, so the setup is one line if your machine qualifies:



```
# Ollama (needs ~80GB free RAM/VRAM for the default quant)
ollama run mistral-medium-3.5:128b

# Self-host FP8 with vLLM (official recipe; needs a vLLM NIGHTLY build as of July 2026)
vllm serve mistralai/Mistral-Medium-3.5-128B --tensor-parallel-size 8 \
--tokenizer_mode mistral --config_format mistral --load_format mistral \
--enable-auto-tool-choice --tool-call-parser mistral --reasoning-parser mistral

# No big iron? The hosted API. Set MISTRAL_API_KEY first.
curl https://api.mistral.ai/v1/chat/completions \
-H "Authorization: Bearer $MISTRAL_API_KEY" \
-H "Content-Type: application/json" \
-d '{"model":"mistral-medium-3-5","messages":[{"role":"user","content":"Refactor this function and explain why."}]}'

```



## Limitations and gotchas




- Dense means no free lunch. A 128B MoE with 40B active would run far faster on the same hardware. Every token here touches all 128B weights.

- The license is not Apache 2.0. The $20M/month carve-out is irrelevant for most of us but makes lawyers at big companies read the file twice.

- Mistral's chart-image benchmarks mean key numbers (MMLU, GPQA) are unverifiable in text form. We flagged what is official versus community-reported above.

- It trails DeepSeek V4 and Kimi K2.6 on SWE-bench by about three points, per community trackers. If leaderboard-max coding is the only goal, those remain ahead (and much heavier).

- Early GGUFs shipped with a broken YaRN long-context config (repetition and forgetting on long chats). It is fixed upstream, so if you downloaded a quant in the first days after launch, re-download it.

- Vision does not work through Ollama (the GGUF needs a separate mmproj file); use llama.cpp for image input, with a BF16 or F32 mmproj, not F16.

- vLLM support is nightly-only at the time of writing, and fine-tuning a 128B dense model is out of QLoRA range for typical rigs. If you want to fine-tune a Mistral, use [Small 3.1](models/mistral-small-3-1.html).



## Who should use it


Pick Medium 3.5 if you want near-frontier agentic coding from weights you control, and you have either a multi-GPU node, a 128GB unified-memory machine, or a tolerance for API pricing. It is the strongest open Mistral for coding and agent work, sitting between the local-friendly [Small 3.1](models/mistral-small-3-1.html) and the chat-focused [Large 3](models/mistral-large-3.html). If your box tops out at 24GB of VRAM, run Small 3.1 locally and call Medium 3.5 over the API when the task deserves it.



## Sources and further reading




- [Mistral: Medium 3.5 announcement](https://mistral.ai/news/vibe-remote-agents-mistral-medium-3-5/)

- [Official weights repo (Hugging Face)](https://huggingface.co/mistralai/Mistral-Medium-3.5-128B)

- [Mistral docs: Medium 3.5 model card](https://docs.mistral.ai/models/model-cards/mistral-medium-3-5-26-04)

- [unsloth GGUF quants](https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF)

- [Ollama library entry](https://ollama.com/library/mistral-medium-3.5:128b)

- [Artificial Analysis: independent benchmarks](https://artificialanalysis.ai/models/mistral-medium-3-5)


Tested on: not independently benchmarked. A 128B dense model exceeds our local bench, so all numbers above come from Mistral's model card (marked official) or community trackers (marked community-reported), with sources linked.
Date checked: 2026-07-10

---


# LongCat-2.0
Source: https://singularitybyte.com/models/longcat-2-0.html

Meituan is the company that delivers your dumplings in Beijing. On June 29, 2026, its LongCat team open-sourced LongCat-2.0, a 1.6-trillion-parameter agentic model under the MIT license, with a native 1M-token context window. Two more headlines ride along: the model had already spent two months anonymously topping OpenRouter's agent leaderboards as "Owl Alpha," and Meituan says the whole thing was trained and served on domestic Chinese AI chips, no NVIDIA hardware involved. Here is what shipped, the numbers with their asterisks, and what you can actually do with it today.

## What Meituan shipped

LongCat-2.0 is a sparse Mixture-of-Experts (MoE) model: 1.6 trillion total parameters, with roughly 48 billion active per token. An MoE splits its feed-forward layers into many small "expert" subnetworks and routes each token through only a few of them, so you pay storage for all 1.6T but compute for about 48B, the same bargain behind [DeepSeek-V4](/models/deepseek-v4.html) and [GLM-5.2](/models/glm-5-2.html). LongCat goes one step further: zero-computation experts and a PID controller let the activation float between about 33B and 56B per token depending on how hard the token is, per the [official README](https://github.com/meituan-longcat/LongCat-2.0).

The release is complete in the way we wish every open drop were: [BF16 weights](https://huggingface.co/meituan-longcat/LongCat-2.0), [FP8](https://huggingface.co/meituan-longcat/LongCat-2.0-FP8) and [INT8](https://huggingface.co/meituan-longcat/LongCat-2.0-INT8) checkpoints, SGLang deployment guides for both GPUs and NPUs, OpenAI- and Anthropic-compatible API endpoints, a thinking-mode toggle, and MIT for all of it. Pretraining ran on more than 35 trillion tokens, including hundreds of billions of tokens of 1M-context data, and Meituan claims the run finished with no rollbacks or loss spikes. Maximum output is 131,072 tokens.

This is not Meituan's first model. LongCat-Flash, a 560B MoE from 2025 ([tech report](https://arxiv.org/pdf/2509.01322)), introduced the zero-computation-expert trick, and the Flash-Thinking, Flash-Omni, and Flash-Lite variants followed. LongCat-2.0 is the trillion-scale successor, and it arrives in the same [June wave](/news/open-weight-wave-june-2026.html) that gave us GLM-5.2 and [Kimi K2.7 Code](/models/kimi-k2-7-code.html).

## Two months undercover as Owl Alpha

The best part of the launch story is that the market reviewed this model before anyone knew what it was. LongCat-2.0 ran anonymously on OpenRouter as "Owl Alpha" for about two months. During that stealth run it ranked first by monthly call volume on the Hermes Agent workspace, second on Claude Code, and third on OpenClaw, according to [Decrypt's report](https://decrypt.co/372579/longcat-2-0-meituan-ai-stealth-model-openrouter). Community trackers put its throughput above ten trillion tokens a month before the reveal, a figure Meituan itself has not confirmed, so treat it as community-reported.

Why that matters: developers routed real agent workloads to an unbranded model because it performed, not because of a launch blog. That is about as clean a blind test as this industry gets.

## The benchmarks, and the asterisk

Meituan's official table compares LongCat-2.0 only against closed frontier models. Values marked with an asterisk are cited from the competitor's own reports; the LongCat numbers are Meituan's and have not been independently reproduced. Notably, there is no official comparison against GLM-5.2 or DeepSeek-V4, so resist the urge to line those up from different tables.

Benchmark | LongCat-2.0 | GPT-5.5 | Gemini 3.1 Pro | Claude Opus 4.8
SWE-bench Pro | 59.5 | 58.6* | 54.2* | 69.2*
Terminal-Bench 2.1 | 70.8 | 73.8* | 70.7* | 78.9*
SWE-bench Multilingual | 77.3 | n/p | 76.9* | 84.8*
BrowseComp | 79.9 | 84.4* | 85.9* | 84.3*
GPQA-diamond | 88.9 | 93.6* | 94.3* | 92.4
All LongCat figures are Meituan-reported. Asterisked competitor scores are cited from those vendors' own publications; "n/p" means no public score. Source: the [official benchmark table](https://github.com/meituan-longcat/LongCat-2.0).

The honest read: near-frontier, not frontier-beating. On SWE-bench Pro, the agentic coding benchmark people actually watch, 59.5 edges GPT-5.5 (58.6) and clears Gemini 3.1 Pro (54.2), but the newer Claude Opus 4.7 (64.3) and 4.8 (69.2) stay ahead. For an MIT-licensed model you can download, modify, and redistribute, beating any current closed frontier model on any agentic benchmark is the story. Expecting it to beat all of them is not the right bar.

## The Chinese-chips claim

Meituan calls LongCat-2.0 the industry's first trillion-parameter model to complete full-process training and inference on domestic "AI ASIC superpods," with reporting citing a cluster of about 50,000 accelerator cards and zero NVIDIA silicon ([SiliconANGLE](https://siliconangle.com/2026/06/30/chinas-meituan-open-sources-massive-longcat-2-0-ai-model-saying-trained-domestic-chips/), [SCMP](https://www.scmp.com/tech/tech-trends/article/3358854/china-debuts-biggest-ai-model-trained-local-chips-meituan-releases-longcat-20)).

The part nobody can verify: which chips. Meituan never names the vendor. The community's guess is Huawei's Ascend line, inferred from interconnect and memory clues in the documentation, but that is speculation, not confirmation. What is checkable is the deployment code: the repo ships an NPU serving path alongside the GPU one, which at minimum proves non-NVIDIA inference is a first-class citizen, not a press-release afterthought. If the training claim holds up, the export-control debate we covered in [the open-weights export fight](/news/anthropic-export-ban-case-for-open-weight-ai-2026.html) just got a large new data point.

## Architecture for the curious

Two pieces are genuinely new. First, LongCat Sparse Attention (LSA), Meituan's answer to the sparse-attention design DeepSeek introduced: an indexer picks which past tokens each query actually attends to, and LSA layers three refinements on top (streaming-aware indexing, one index shared across every two layers, and a coarse-to-fine hierarchical pass). The payoff is long-context cost that scales closer to linear, which is what makes a native 1M window usable rather than theoretical.

Second, the on-disk weights are bigger than the headline: alongside the 1.6T MoE parameters sits a 135B N-gram Embedding block, inherited from LongCat-Flash-Lite, which is why the checkpoint folder describes itself as roughly 1.8T parameters. A 3-step multi-token prediction module handles speculative decoding, and post-training distills three separate teacher groups (agent, reasoning, interaction) into one student, a recipe Meituan calls MOPD.

## Limitations and gotchas

- You cannot run this at home. Even the FP8 checkpoint is a multi-node deployment; community estimates point at a 16x H20-class server as the floor, and that figure is itself unverified. This is a rent-a-cluster or use-the-API model.

- No Ollama build and no community GGUF exist yet, so there is no one-line local install to paste here.

- Benchmarks are self-reported, and the OpenRouter throughput legend is community-reported. The blind Owl Alpha traffic is the strongest independent signal we have.

- The chip vendor is undisclosed. "Trained on domestic silicon" is a Meituan claim with no third-party audit.

- API pricing (around $0.75 per million input tokens and $2.95 per million output at standard rates, with a launch promo roughly 60 percent below that) comes from aggregator reporting, so confirm current rates on the platform before you build a budget around them.

## Who should use it

Teams running agent harnesses (Claude Code-style CLIs, browser agents, long-horizon coding loops) who want an MIT-licensed brain they can eventually self-host, and researchers who want to study trillion-scale sparse attention with full weight access. If you need a model for one GPU, this is not it; look at [Qwen 3.6 27B](/models/qwen-3-6.html) instead. If you need the absolute best agentic scores regardless of license, Claude Opus 4.8 still holds that crown.

## Try it in about 10 minutes

The fastest path is the free chat at [longcat.ai](https://longcat.ai/). For API access, the platform exposes both OpenAI- and Anthropic-compatible endpoints, so existing harnesses connect with a base-URL swap; the [README](https://github.com/meituan-longcat/LongCat-2.0) documents the exact setup for Claude Code and friends. If you have serious hardware, the weights are one command away:

```

# Pull the FP8 checkpoint (bring a multi-node GPU server and patience)
hf download meituan-longcat/LongCat-2.0-FP8

# Deployment recipes for SGLang (GPU and NPU) live in the repo
git clone https://github.com/meituan-longcat/LongCat-2.0

```

Then point whatever agent harness you already run at the endpoint and give it a long multi-step task. A model tuned for agents deserves a real agent workload, not a haiku request.

## Sources and further reading

- [LongCat-2.0 model card (Hugging Face)](https://huggingface.co/meituan-longcat/LongCat-2.0)

- [Official GitHub repo, benchmark table and deployment guides](https://github.com/meituan-longcat/LongCat-2.0)

- [VentureBeat coverage](https://venturebeat.com/technology/meituan-open-sources-longcat-2-0-the-1-6t-near-frontier-agentic-coding-model-thats-been-leading-openrouter-trained-entirely-on-chinese-chips)

- [Decrypt: the Owl Alpha stealth run](https://decrypt.co/372579/longcat-2-0-meituan-ai-stealth-model-openrouter)

- [SCMP: China's biggest model trained on local chips](https://www.scmp.com/tech/tech-trends/article/3358854/china-debuts-biggest-ai-model-trained-local-chips-meituan-releases-longcat-20)

- [MarkTechPost: architecture breakdown](https://www.marktechpost.com/2026/07/05/meituan-releases-longcat-2-0-a-1-6t-parameter-open-moe-model-with-native-1m-context-and-longcat-sparse-attention/)

- [LongCat-Flash technical report (arXiv)](https://arxiv.org/pdf/2509.01322)

Tested on: not independently tested. LongCat-2.0 is a roughly 1.8T-parameter checkpoint that needs a multi-node GPU or NPU cluster, beyond our bench. All benchmark figures are Meituan-reported; OpenRouter traffic figures are community-reported and flagged as such. Sources linked above.

Date checked: 2026-07-09

---


## Tools

# PixelRAG: Visual RAG That Retrieves Screenshots Instead of Parsing Text
Source: https://singularitybyte.com/tools/pixelrag-visual-rag.html

Every RAG pipeline you have built starts by throwing away information. You parse a PDF to text, and the table that held the answer becomes a column of orphaned numbers. PixelRAG takes the other road: it renders documents to images and retrieves over the pixels, so tables, charts and layout survive intact. It hit 7,063 GitHub stars since late May, ships under Apache 2.0, and v0.4.0 landed on July 16. We installed it, rendered a real paper, and ran queries against its hosted 8.28-million-page Wikipedia index. Here is what holds up.

## TL;DR

- What it is: a visual RAG pipeline (render, embed, index, search) that screenshots documents and retrieves image tiles using a LoRA-tuned Qwen3-VL-Embedding-2B, instead of parsing text.

- Why it matters: it runs on a laptop CPU, it is Apache 2.0, and it fixes a failure mode every text pipeline has, where table structure and superscripts are silently corrupted before your model ever sees them.

- The catch: retrieval returns a picture of a page region, not an answer, so you still need a vision model to read it. Top-1 precision was noisy in our tests, and the full Wikipedia index is about 217 GB.

## The problem, shown rather than asserted

We rendered the original Transformer paper, "Attention Is All You Need", and pulled page 8. That page holds Table 2, a two-level header table: BLEU and Training Cost across the top, each split into EN-DE and EN-FR below.

PixelRAG keeps that page as a 1700x2200 pixel image. Every rule line, every column boundary, every empty cell stays where the typesetter put it. Now here is the same page run through PyMuPDF, which is what a conventional pipeline feeds your model:

```

import pymupdf
doc = pymupdf.open("attention.pdf")
print(doc[7].get_text())

```

The output flattens to this:

```

Model
BLEU
Training Cost (FLOPs)
EN-DE
EN-FR
EN-DE
EN-FR
ByteNet [18]
23.75
Deep-Att + PosUnk [39]
39.2
1.0 · 1020
GNMT + RL [38]
24.6
39.92
2.3 · 1019
1.4 · 1020

```

Three things just broke, and none of them raised an error.

Empty cells vanished. ByteNet has an EN-DE score and no EN-FR score, so the text gives you one number with no way to know which column it came from. Column alignment died with it. Deep-Att shows "39.2" then "1.0 · 1020", and the natural reading is EN-DE BLEU and EN-DE cost. Both are wrong: 39.2 is the EN-FR score, and that cost belongs to EN-FR too. And the superscripts silently corrupted. Look at "1020". In the paper that is 10 to the power of 20. The text extraction turned an exponent into a four-digit number, so the training cost is off by eighteen orders of magnitude, in a string your model will read as fact.

This is the entire argument for pixel-native retrieval, and you can reproduce it in two commands. Text parsing does not fail loudly. It fails into plausible, confidently wrong numbers.

## How it works

The pipeline is five stages, each a separate CLI verb, so you can stop after any of them: render, chunk, embed, build-index, serve. Rendering uses a browser engine for URLs and HTML, and Poppler for PDFs, cutting each page into fixed-height tiles. Embedding runs Qwen/Qwen3-VL-Embedding-2B, a 2B vision-language embedding model that the project fine-tuned with LoRA on screenshot data, with the adapters published separately. Indexing writes to FAISS by default, or Qdrant if you point it at a server.

Retrieval returns tile coordinates, not prose. A hit tells you the source document, which tile, and the vertical offset within the page. Handing that crop to a vision model is your job, which is why anthropic ships as a base dependency for the reader step. If you want the reader local instead, any vision model works, including the multimodal open-weights options we covered in the [Inkling](/models/thinking-machines-inkling.html) writeup.

## Hands-on: install and render

One gotcha before you start. PyPI marks the package requires_python >=3.12, while the README still says 3.10 or newer. On Python 3.9 the install simply refuses, so start with a modern interpreter:

```

python3.13 -m venv venv
./venv/bin/pip install pixelrag

# PDF support is NOT in the base install.
# You also need poppler on the system (pdftoppm, pdfinfo).
./venv/bin/pip install 'pixelrag[pdf]'

# Render any mix of URLs and PDFs to tiles
./venv/bin/pixelshot attention.pdf -o ./tiles

```

Skip that second line and rendering dies with "pdf2image is required for PDF rendering", which then tells you to install pixelrag-render[pdf]. That package name does not resolve. The one that works is pixelrag[pdf], as documented in the README. Minor, but it will cost you a few minutes.

With that sorted, the 15-page paper rendered in 7 seconds on a 6-core CPU with no GPU, producing 15 tiles at 1700x2200 pixels for 6.5 MB total. That is the honest headline for local use: the render stage is cheap and needs no accelerator. Embedding is where a GPU starts to matter, and the project quotes roughly 3 minutes on Apple Silicon or 1 minute on a GPU to index a paper.

## Hands-on: querying the hosted index

You do not have to index anything to try retrieval. The project hosts a prebuilt index of 8.28 million Wikipedia pages, open to unauthenticated POSTs:

```

curl -X POST https://api.pixelrag.ai/search \
-H "Content-Type: application/json" \
-d '{"queries": [{"text": "periodic table of elements"}], "n_docs": 3}'

```

Three queries, three responses, all between 2.18 and 2.36 seconds round trip from Europe. Results below, with the similarity score and the vertical offset of the winning tile:

Query | Top result | Score | Tile offset
periodic table of elements | Periodic_table | 0.665 | y=0
population of Germany by year table | Yablonovka,_Saratov_Oblast | 0.612 | y=1024
population of Germany by year table (2nd hit) | Demographics_of_Germany | 0.612 | y=5120
GPU architecture comparison chart | Nvidia_Drive | 0.608 | y=4096
Measured 2026-07-22 against the hosted index. Scores are cosine similarity as returned by the API.

The offsets are the interesting part. For the Germany query it returned Demographics_of_Germany at y=5120, which is deep into a long page, exactly where the population tables live rather than the intro paragraph. That is the pitch working: it located a region of a page because that region looks like the answer.

The same query is also the clearest miss. A Russian village, Yablonovka in Saratov Oblast, tied the correct article at 0.612 and sorted above it. Scores across all our queries clustered between 0.53 and 0.67, so there is no clean threshold separating a strong hit from noise. Retrieve several tiles and let the reader model discard the duds. Treat top-1 as a suggestion.

## Running it on your own documents

```

# Full local pipeline
./venv/bin/pip install 'pixelrag[index,serve]'

./venv/bin/pixelrag chunk --tiles-dir ./tiles
./venv/bin/pixelrag embed --shard-dir ./tiles --output-dir ./embeddings
./venv/bin/pixelrag build-index --embeddings-dir ./embeddings --output-dir ./index
./venv/bin/pixelrag serve --index-dir ./index --port 30001

curl -X POST http://localhost:30001/search \
-H "Content-Type: application/json" \
-d '{"queries": [{"text": "training cost table"}], "n_docs": 3}'

```

Or drive it from a config file, which is the cleaner route for a repeatable corpus:

```

# pixelrag.yaml
source:
type: local
path: ./my_docs
embed:
model: Qwen/Qwen3-VL-Embedding-2B
device: auto # cuda on Linux, mps on macOS, cpu fallback
output: ./my_index

```

## Limitations and gotchas

- Retrieval gives you an image crop, not an answer. Budget for a vision model on the read step, and for the tokens that image will cost you.

- Top-1 precision was unreliable in our sampling, and similarity scores cluster too tightly to threshold. Pull 3 to 5 tiles per query.

- The prebuilt Wikipedia index is roughly 217 GB in FAISS format. The hosted API is the sane way to try it.

- No published accuracy comparison against text-based RAG. The README argues the case by example, as we did above, but there are no benchmark tables yet.

- Python 3.12 or newer despite what the README says, plus Poppler for PDFs and a Chromium for web pages.

- Storage and bandwidth are real. Page images are heavy next to text chunks: our 2.2 MB PDF became 6.5 MB of tiles.

- Qdrant users run their own server; only FAISS works out of the box.

## Who should use it

Use PixelRAG if your corpus is documents that were designed to be looked at: financial filings, scientific papers, datasheets, slide decks, anything where a table or a chart carries the answer. The render stage costs nothing on CPU, so you can evaluate it on your own files this afternoon and see whether visual retrieval finds things your text pipeline misses.

Stay with text RAG if your corpus is already clean text like chat logs, code, or Markdown docs, where rendering to pixels adds cost and removes the exact-match precision you get from keyword search. And if you need audited accuracy numbers before adopting anything, wait: this project is under two months old, and the evidence so far is demonstrative rather than measured.

## Sources and further reading

- [GitHub: StarTrail-org/PixelRAG](https://github.com/StarTrail-org/PixelRAG)

- [PixelRAG project site and hosted API](https://pixelrag.ai)

- [PyPI: pixelrag 0.4.0](https://pypi.org/project/pixelrag/)

- [Hugging Face: the screenshot embedding LoRA adapters](https://huggingface.co/Chrisyichuan/wiki-screenshot-embedding-lora)

- [Attention Is All You Need (the PDF used in our test)](https://arxiv.org/abs/1706.03762)

Tested on: AMD 6-core x86_64 · 8 GB RAM · no GPU · Linux 6.1 · Python 3.13 · PixelRAG 0.4.0 from PyPI · Poppler 22.12. Render timing measured on that machine; search latency measured against the hosted api.pixelrag.ai index from Europe. Embedding and indexing were not benchmarked locally, since the 2B embedding model wants a GPU we do not have on this bench.

Date tested: 2026-07-22

---


# OpenCode
Source: https://singularitybyte.com/tools/opencode.html

If you write code with an AI agent in your terminal, you have probably already switched to OpenCode, or you are about to. The MIT-licensed coding agent crossed 179,000 GitHub stars and around 7.5 million monthly developers by late June 2026, ships a new release almost every day, and works with 75-plus model providers instead of locking you to one lab. Then on June 18, Google pulled the plug on Gemini CLI for free and consumer users, and the migration turned into a stampede. Here is what OpenCode is, why it won, and how to point it at the open models we have been covering.

## What OpenCode is

OpenCode is an open-source AI coding agent that runs in your terminal, plus a beta desktop app and a VS Code extension. It is not a chat box. It reads your files, edits code, runs shell commands, and tests its own work, the full agentic loop. It is built by Anomaly, the team formerly known as SST. (One note to avoid confusion: the original Go version split off and continues as "Crush"; the TypeScript OpenCode at github.com/anomalyco/opencode is the one everyone means now, and the old sst/opencode URL just redirects there.)

The defining choice is that OpenCode ships no model of its own. It is model-agnostic, talking to 75-plus providers through Models.dev: Claude, GPT, Gemini, DeepSeek, and any OpenAI-compatible endpoint, including local servers. It supports MCP (the Model Context Protocol, the now-standard way agents call external tools) and LSP (Language Server Protocol) for real code intelligence. It ships built-in agents (a read-write "build" agent, a read-only "plan" agent, a "general" subagent) and lets you define your own.

## Why it won: the Gemini CLI exodus

OpenCode was already the most-starred open coding harness. Then Google handed it the market. On June 18, 2026, Google shut down Gemini CLI for free, Pro, and Ultra consumer users with no grace period, and is ending consumer Code Assist for GitHub on July 17. The replacement, Antigravity CLI, is closed-source and cut the free tier from roughly 1,000 requests a day to about 20, a 98 percent reduction.

For a developer who built a workflow around a free, open terminal agent, that is a forced migration. OpenCode is the obvious landing spot, because it still talks to Gemini models through the API if you want them, alongside 74 other providers, with no vendor able to switch it off. Qwen Code, Alibaba's Apache-2.0 fork of the old Gemini CLI, picked up the overflow. The lesson writes itself: an open, model-agnostic tool cannot be discontinued out from under you.

## One honest caveat about benchmarks

You will see people quote SWE-Bench or Terminal-Bench numbers "for OpenCode." Ignore them. Because OpenCode has no built-in model, it has no intrinsic score; its ceiling is whatever model you feed it. Point it at Claude Opus 4.8 or GPT-5.5 and you get top-of-leaderboard coding (Opus 4.8 lands around 74 percent on Terminal-Bench 2.1 and 88 percent on SWE-Bench Verified). Point it at a weak local model and you get weak results. The tool is the harness, not the brain.

## Install it and point it at an open model in 10 minutes

Install is one line, by whatever package manager you already use.

```

# pick one
npm i -g &#x6f;&#x70;&#101;&#x6e;&#x63;&#x6f;d&#101;&#x2d;&#x61;&#105;&#64;&#x6c;&#x61;&#x74;&#101;&#115;&#116;
brew install anomalyco/tap/opencode
curl -fsSL https://opencode.ai/install | bash

# then just run it inside a project
opencode

```

The interesting part for this site is wiring it to an open model instead of a closed API. OpenCode reads an opencode.json in your project. Here it is pointed at a local server (Ollama, llama.cpp, or LM Studio) over the OpenAI-compatible API; the same pattern works for a rented GPU running GLM-5.2 or Qwen3.6:

```

{
"$schema": "https://opencode.ai/config.json",
"provider": {
"local": {
"npm": "@ai-sdk/openai-compatible",
"options": { "baseURL": "http://localhost:11434/v1" },
"models": { "qwen3.6:27b": { "name": "Qwen3.6 27B (local)" } }
}
},
"model": "local/qwen3.6:27b"
}

```

Switch models mid-session with the /models command. That is the whole pitch: one agent, every model, including the open ones you can actually own. Pair it with [GLM-5.2](/models/glm-5-2.html) for cheap agentic coding, or a local [Qwen](/models/alibaba-qwen-3-5.html) when you want it air-gapped.

## Who should use it

Anyone who wants a serious terminal coding agent without lock-in: Claude Code users who want model freedom, ex-Gemini-CLI users who just got evicted, and anyone running local or open-weight models who needs a harness that speaks their endpoint. If you are deep in a single closed ecosystem and happy there, you may not need it. Everyone else: the switching cost is one install command.

## Sources and further reading

- [OpenCode official site](https://opencode.ai/)

- [OpenCode on GitHub (MIT)](https://github.com/anomalyco/opencode)

- [OpenCode docs: providers and local models](https://opencode.ai/docs/providers/)

- [9to5Google: Gemini CLI shutting down](https://9to5google.com/2026/06/17/gemini-cli-code-assist-shutting-down/)

Tested on: not benchmarked. OpenCode is model-agnostic and has no intrinsic score; its quality equals the model you connect. The facts here (stars, version v1.17.11, 75-plus providers, the June 18 Gemini CLI shutdown) are verified against the GitHub repo and primary coverage.

Date checked: 2026-06-26

---


# StableDAW: The One-Click AI Music Studio (Now theDAW)
Source: https://singularitybyte.com/tools/stabledaw-ai-music-studio-pinokio.html

Update, 2026-07-25: StableDAW has been archived and continues as [theDAW](https://github.com/gantasmo/theDAW), now with ten workspaces instead of seven. The Pinokio launcher below has not been updated and still installs the archived June build. This article remains an accurate account of that build. For the current release and working install paths, read [part two: StableDAW is now theDAW](/tutorials/stabledaw-is-now-thedaw-whats-new-2026.html).



StableDAW just landed in the Pinokio catalog, and it is the first AI music tool that feels like an actual studio instead of a text box with a download button. It is a browser-based digital audio workstation built around Stability AI's open-weight Stable Audio 3 models, it installs in one click, and the small model runs on a plain CPU. No account, no API key, no cloud credits. You generate, edit, mix, and perform entirely on your own machine.




The catch up front, so nobody wastes a download: Stable Audio 3 makes instrumental music and sound effects. It does not sing. If you need vocals, this is not your tool. If you want a local, MIT-licensed beat factory that you fully control, read on.



StableDAW, branded "theDAW" in the app, was built by GANTASMO at a Music Hackspace hackathon.



## What StableDAW actually is


StableDAW (the app calls itself "theDAW") was built by [GANTASMO](https://github.com/gantasmo) during a Music Hackspace music-technology hackathon at Berklee College of Music. The code is [MIT-licensed](https://github.com/gantasmo/theDAW) on GitHub, and [cocktailpeanut](https://github.com/cocktailpeanut) wrote the Pinokio launcher that turns the whole stack into a one-click install.


It is not a one-trick prompt-to-clip generator. It is a full workstation with seven workspaces, laid out as a left-to-right workflow across the top tab bar:




- MAKE generates audio from text, from your own recordings, or both. Text-to-audio, audio-to-audio, inpainting, continuation, and a "Chimera" blend that fuses multiple clips.

- EDIT is a multitrack timeline with clip arrangement, trimming, fades, and a snap grid.

- MIX hosts 24 FFmpeg-powered effects, a mastering chain, a drum sequencer, and a piano roll, and exports to FLAC, MP3, AAC, or Opus.

- DJ is a two-deck console with sync and FX, and it maps to MIDI controllers.

- VJ drives a 3D audio-reactive visualizer for live shows.

- TRAIN fine-tunes LoRA adapters on your own audio (this one needs an NVIDIA GPU).

- LEARN draws an interactive genealogy graph that traces how every clip descended from its parents.


Those seven are the June build documented here. theDAW now runs ten: TRAIN was renamed Underfit, and Perform, Foundry and Audimate joined it. [Part two covers all four](/tutorials/stabledaw-is-now-thedaw-whats-new-2026.html).



That last one is the tell. StableDAW keeps a persistent, searchable library and remembers the full lineage of every render: which prompt made it, which clip it was inpainted from, which stems it was split into. For anyone who has lost a good take to a closed cloud tool, that local history is the selling point.



## The engine: Stable Audio 3, open weights, no vocals


Under the hood is [Stable Audio 3](https://huggingface.co/stabilityai/stable-audio-3-medium), a diffusion transformer paired with Stability AI's SAME autoencoder. It outputs 44.1 kHz stereo, which is CD quality. StableDAW ships two checkpoints, and the difference decides what hardware you need.





Model
Size
Hardware
Max length
Use it for
Small (433M)
~3.5 GB
CPU or any GPU
120 sec
Default. Loops, sketches, SFX, laptops without a GPU.
Medium (1.4B)
~10.4 GB
NVIDIA, ~8 GB VRAM
380 sec
Full-length tracks, higher musicality.
The launcher pulls both from ungated community mirrors (cocktailpeanut/stable-audio-3-* on Hugging Face), so there is no license click-through and no token. The Small model downloads automatically; you grab Medium from a button when you want it. A separate Medium-RF checkpoint is the base for LoRA training in the TRAIN tab.


On licensing, the app code is MIT. The model weights sit under the Stability AI Community License, which lets organizations under 1 million dollars in annual revenue use the generated audio commercially. Past that threshold you need an enterprise agreement. For an indie producer or a small studio, the practical answer is: the music you make is yours.



## Install it in one click with Pinokio



No longer current. The launcher below took its last commit on 2026-06-12 and still clones the archived gantasmo/stabledaw repo. The install succeeds, so there is no error to warn you, but you get the June build. See [part two](/tutorials/stabledaw-is-now-thedaw-whats-new-2026.html) for install paths that land on theDAW.


If you have not used [Pinokio](https://pinokio.computer) before, it is a desktop launcher that installs and runs AI apps inside their own isolated environments, no terminal required. Install Pinokio first, then add StableDAW from its repository page.



```
# 1. Install Pinokio from pinokio.computer (Windows, Linux, or macOS)
# 2. In Pinokio, open the StableDAW launcher repo:
# github.com/cocktailpeanut/stabledaw.pinokio
# 3. Click Install, then Start. The launcher handles the rest.
```


Behind that one click, the launcher clones the repo, builds a Python 3.10 environment with the right torch for your platform via uv sync, runs npm ci for the React frontend, and downloads the ~3.5 GB Small model. Then it boots a FastAPI backend on port 8600 and a Vite frontend on port 5173, and opens the web UI.


Two things to budget for. You need roughly 10 to 15 GB of free disk for the environment plus the Small model, and another ~10.4 GB if you add Medium. And both ports (8600 and 5173) must be free, because Vite runs with strictPort and will not shop around for another one.


Platform notes: Windows installs CUDA 12.8 and Flash Attention automatically, Linux x86_64 wants CUDA 12.6, and macOS runs the Small model on CPU. That includes Apple Silicon, which works fine without any NVIDIA hardware, just slower than a CUDA box.



## Hands-on: making your first track


The MAKE tab is where you live at the start. Drop a description into the prompt box, set the controls on the left (model, length, sampler steps, CFG, seed, batch), and hit CREATE in the bottom right. Be specific. Genre, tempo, instrumentation, and mood all move the output.



The MAKE workspace: prompt boxes up top, sampler controls on the left, the Chimera blend stage in the center, output format on the right.


A prompt that works well as a starting point:



```
Lo-fi boom bap with dusty vinyl crackle, warm Rhodes chords,
mellow upright bass, soft swung drums, 84 BPM
```


Generation speed depends entirely on the model and the silicon. On a recent NVIDIA card the Small model returns a two-minute clip in a few seconds. On CPU, the same clip takes longer, and the Medium model on CPU is measured in minutes, not seconds, so save it for the GPU. These are community-reported figures; we cover the disclosure at the end.


Once you have a clip you like, the generative tools stack. Paint a region and inpaint just that bar. Extend a loop with continuation. Or drag several clips into Chimera and let it beat-align them into one new generation. Every result drops into the library with its full prompt and parameters, so a good seed is never lost.



## Beyond generation: edit, mix, perform


This is where StableDAW earns the "DAW" in its name. The EDIT tab gives you a real multitrack timeline. You arrange clips, trim them, add fades, and commit edits, all in the browser.



The EDIT tab: a standard multitrack timeline. Drop clips from the library and arrange them.


The MIX tab is a surprise for a hackathon project. It carries 24 FFmpeg-powered effects grouped into a library (mastering chain, vocal processing, lo-fi and vinyl, stereo widener, reverb and delay, club EQ, compressor, loudness normalization, and more), a QUICK MASTER section with punch, air, drive, and ceiling knobs, plus a drum sequencer and a piano roll. You chain effects left to right and hit process.



The MIX tab: an FFmpeg effects library, a quick-master section, and an effects chain you build left to right.


Then it keeps going. The DJ tab is a two-deck console with sync, per-deck FX, and MIDI mapping. The VJ tab is a 3D audio-reactive visualizer for live performance. And the TRAIN tab lets you fine-tune LoRA adapters on a folder of your own audio, then stack them at generation time for a custom sound, provided you have the Medium-RF model and an NVIDIA GPU.



## Limitations and gotchas


It is a hackathon build, and a few rough edges show:




- No vocals, period. Stable Audio 3 is instrumental and SFX only. Pair it with a separate vocal tool if you need singing.

- CPU is slow. The Small model is usable on CPU; Medium really is not. Treat a GPU as the requirement for full-length work.

- First boot lags. The initial PyTorch import takes about a minute. That is normal, not a hang.

- The Reset button is a real reset. It wipes your generated-audio library. Export anything you care about first.

- The original repo is archived. Active development moved to [gantasmo/theDAW](https://github.com/gantasmo/theDAW). The Pinokio launcher tracks the working build, so the one-click path still works, but watch the new repo for updates.



## Who should use it, and what to do in the next 10 minutes


StableDAW is for producers, indie hackers, and anyone who wants AI music generation they actually own: local files, open weights, full edit history, and a clear commercial path under the Community License. It trades the polished vocals of closed tools for control, transparency, and a studio that never phones home.


The 10-minute version: install Pinokio, add the StableDAW launcher, click Start, and let the Small model download. While it pulls, free up ports 8600 and 5173. When the UI opens, paste the lo-fi prompt above into MAKE and hit CREATE. You will have a track playing before your coffee is cold, and it will be sitting in a library on your own disk, not someone else's server.


If you like running models locally, our [tools section](tools.html) has more open-source workstations worth your disk space.



## Sources and further reading




- [Pinokio announcement: StableDAW, a full AI music studio that runs on your own machine](https://beta.pinokio.co/posts/01kty98bwb20619q6274768kcm)

- [theDAW on GitHub, the active successor (gantasmo/theDAW)](https://github.com/gantasmo/theDAW)

- [StableDAW on GitHub, archived (gantasmo/StableDAW)](https://github.com/gantasmo/StableDAW)

- [Pinokio launcher repo, unmaintained (cocktailpeanut/stabledaw.pinokio)](https://github.com/cocktailpeanut/stabledaw.pinokio)

- [Part two: StableDAW is now theDAW, what changed and how to move](/tutorials/stabledaw-is-now-thedaw-whats-new-2026.html)

- [Stable Audio 3 Medium model card on Hugging Face](https://huggingface.co/stabilityai/stable-audio-3-medium)


Tested on: install walkthrough verified against the Pinokio launcher and the StableDAW GitHub README. Generation timings are community-reported, not benchmarked on our own hardware. Model version: Stable Audio 3 Small (433M) and Medium (1.4B) via the StableDAW launcher.
Date tested: 2026-06-15

---


# Google TurboQuant
Source: https://singularitybyte.com/tools/google-turboquant.html

Long context is a memory problem. The KV cache, the running store of key and value vectors that attention reads back on every token, balloons with sequence length and quietly eats your VRAM. Google Research's TurboQuant attacks exactly that: it squeezes the KV cache to 2.5 to 4 bits with no calibration data and near-optimal error. The community already wired it into llama.cpp and vLLM, so you can try it this afternoon. The headline numbers, though, need an asterisk or three.

## What TurboQuant actually is

First, the thing people keep getting wrong: TurboQuant quantizes the KV cache, not the model weights. It does not replace GPTQ, AWQ, or your GGUF Q4 weights. It stacks on top of them. Weight quantization shrinks the model on disk and in memory; TurboQuant shrinks the per-token cache that grows as you generate. Different problem, complementary fix.

It comes from a Google Research paper, "TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate" (Zandieh, Daliri, Hadian, Mirrokni), posted to arXiv in April 2025 and accepted at ICLR 2026. Google blogged about it in March 2026. Two properties make it interesting for builders: it is online (compresses vectors as they are written, no separate pass) and it is data-oblivious (no calibration set, no tuning run). You point it at a model and go.

## How it works

The trick is a random rotation. Quantization hates outliers, because a few large coordinates force a coarse scale on everything else. TurboQuant multiplies each key or value vector by a random orthogonal matrix first, which spreads outlier energy evenly across dimensions and pushes the per-coordinate distribution toward a predictable shape. After that rotation, each coordinate is close to independent, so a per-coordinate Lloyd-Max quantizer (the analytically optimal scalar quantizer for a known distribution) does the rest. Because the distribution is known ahead of time, there are no per-block scales or zero-points to store, which is where the bit savings come from.

That is the MSE variant. There is a second variant tuned for attention. Attention does not care about reconstructing vectors, it cares about dot products. So TurboQuant runs the MSE quantizer at one bit lower, then applies a 1-bit Quantized Johnson-Lindenstrauss transform to the residual error. At read time, a special estimator combines both parts to recover an unbiased inner product. The paper argues the whole scheme lands within a small constant factor of the information-theoretic distortion floor.

## The numbers, and the asterisks

Google's reported results are genuinely good. On Llama-3.1-8B-Instruct, long-context quality barely moves at 3.5 bits per channel.

Config | LongBench avg | Needle-in-a-Haystack | Source
FP16 baseline | 50.16 | 0.997 | paper
TurboQuant 3.5-bit | 50.06 | 0.997 | paper
TurboQuant 2.5-bit | 49.44 | ~baseline | paper
Pair that with the memory math (roughly 6x smaller KV cache at 3-bit, and up to 8x faster attention at 4-bit on an H100) and it sounds like a free lunch. It is not, and the people who run inference for a living said so.

The vLLM team published the first large independent study in May 2026, testing Llama-3.3-70B, Qwen3-30B-A3B, and others on H100s. Their verdict: plain FP8 KV cache is still the best default. Here is the shape of it.

Method | KV capacity | Throughput vs BF16 | Accuracy cost | Source
FP8 KV | 2.0x | ~100% | ~0 | vLLM (community)
TurboQuant k8v4 | 2.4x | ~75 to 80% | ~0 | vLLM (community)
TurboQuant 4bit-nc | up to 3.4x | ~75% | ~4 pts reasoning | vLLM (community)
TurboQuant 3bit-nc | highest | ~66 to 73% | ~20 pts reasoning, ~30% long-ctx retrieval at 256K | vLLM (community)
Read that bottom row twice. At aggressive 3-bit settings on a 256K context, long-context retrieval drops by about 30 percent and reasoning accuracy falls roughly 20 points. The compression is real, but so is the tax, and FP8 gets you 2x for free with no measurable loss. TurboQuant earns its keep when you are memory-bound and FP8's 2x is not enough, not when you are chasing raw throughput.

## How to run it today

There is no official Google package. Every usable build is a community fork. Two are worth your time.

### llama.cpp (local, Apple Silicon or a single GPU)

Community forks add tq3_0 (3-bit, about 4.9x KV compression) and tq4_0 (4-bit, about 3.8x) cache types. Build for your backend, then pass the cache type at runtime:

```
# Build (pick your backend)
cmake -B build -DGGML_METAL=ON && cmake --build build -j # Apple Silicon
# cmake -B build -DGGML_CUDA=ON && cmake --build build -j # NVIDIA

# Run with a 4-bit TurboQuant KV cache
./build/bin/llama-cli -m model.gguf \
--cache-type-k tq4_0 --cache-type-v tq4_0 \
-c 32768 -p "Summarize this long document..."
```

### vLLM (server, NVIDIA)

The community vLLM fork exposes TurboQuant as a kv-cache dtype. Build from source, then serve:

```
vllm serve /models/your-model \
--tensor-parallel-size 4 \
--attention-backend TRITON_ATTN \
--kv-cache-dtype turboquant35 \
--enable-turboquant
```

The under-10-minute move: take a model that currently runs out of memory at your target context, switch the KV cache to tq4_0 (or turboquant35 on vLLM), and see whether the longer context now fits. If it does and your eval holds, you win memory for almost nothing. If quality dips, step up the bit-width or fall back to FP8.

## The caveats nobody puts on the slide

Three things temper the hype.

No official code. Google published a paper and a blog post, not a library. TechCrunch called it a lab breakthrough, and that is accurate. Anything labeled "Google's TurboQuant library" is community code, maintained by individuals, not Google.

The RaBitQ dispute. The core move, random rotation before quantization, is the same idea behind RaBitQ (Gao and Long, 2024). Critics on OpenReview and elsewhere argue the paper mischaracterized RaBitQ's method and guarantees, and that the speed comparison pitted TurboQuant on an A100 GPU against RaBitQ on a single CPU core with multithreading switched off. One author acknowledged multiprocessing was disabled. Treat the head-to-head speed claims against RaBitQ as unsettled.

The famous numbers are narrow. The 8x is attention-logit computation only, not end-to-end inference. The "zero accuracy loss" applies to the 3.5-bit setting on long-context benchmarks; the paper never ran MMLU, GSM8K, or HumanEval, and the vLLM study found real drops once you push to 3-bit or stress reasoning.

## Who should use it, and who should wait

Try it if you run long contexts on memory-bound hardware: a consumer GPU, a Mac, an edge box that hits out-of-memory before it hits a speed wall. A 4-bit KV cache can be the difference between a 32K context fitting or not. Models like [Gemma 4](/models/google-gemma-4.html), [Qwen3.5](/models/alibaba-qwen-3-5.html), and [DeepSeek-V4](/models/deepseek-v4.html) are all popular community targets, and [MiniMax](/models/minimax-m3.html)'s long-context models showed up in the vLLM test set too.

Wait, or just use FP8, if you are on datacenter GPUs optimizing for throughput, or running aggressive low-bit configs on reasoning-heavy workloads. In those cases FP8 KV cache gives you 2x with no throughput penalty and no accuracy hit, and that is hard to beat.

## Sources and further reading

- [TurboQuant paper (arXiv 2504.19874)](https://arxiv.org/abs/2504.19874)

- [Google Research announcement](https://research.google/blog/turboquant-redefining-ai-efficiency-with-extreme-compression/)

- [vLLM: first comprehensive TurboQuant study](https://vllm.ai/blog/2026-05-11-turboquant)

- [llama.cpp community discussion and benchmarks](https://github.com/ggml-org/llama.cpp/discussions/20969)

- [The RaBitQ attribution dispute](https://dev.to/gaoj0017/turboquant-and-rabitq-what-the-public-story-gets-wrong-1i00)

- [ICLR 2026 OpenReview thread](https://openreview.net/forum?id=tO3ASKZlok)

Tested on: not independently tested. The figures here are paper- or community-reported via the sources above, with the vLLM study and the RaBitQ dispute flagged as such.

Date checked: 2026-06-09

---


# Unsloth
Source: https://singularitybyte.com/tools/unsloth-the-indie-fine-tuning-backbone.html

If you run fine-tuning on a single GPU, you have almost certainly used [Unsloth](https://github.com/unslothai/unsloth) and you probably did not realize it is a two-person operation. Daniel and Michael Han ship the library that quietly turned LLM fine-tuning from "rent eight H100s for a week" into "borrow an RTX 4090 for an afternoon," then went home and did it again the next day. 64,000+ GitHub stars, two thousand more every month, full Apache-2.0 core. If the local-LLM scene has unpaid infrastructure on the inference side (the GGUF quant-masters), Unsloth is the equivalent on the training side.

## What Unsloth actually does

Unsloth is a drop-in replacement for HuggingFace Transformers + PEFT specifically for the fine-tuning step. The headline claim from the [official benchmarks](https://unsloth.ai/docs/blog/3x-faster-training-packing) is roughly 2x faster training and 30 to 70 percent less VRAM, with the speedups jumping further on short-sequence datasets and Qwen3-class models. The savings come from hand-tuned Triton kernels, smarter memory layout for LoRA adapters, and a packing implementation that beats Flash Attention 3 on most realistic training mixes.

Concrete numbers from the docs: a 7B model needs about 5 GB of VRAM for QLoRA, 19 GB for full LoRA. A 70B model fits in 41 GB of VRAM with QLoRA, which means a single A6000 can fine-tune Llama 3.3 70B. None of that is achievable with a stock HuggingFace pipeline on the same hardware.

## Install and first run

### Install

```
pip install unsloth
```

Python 3.9 or newer, PyTorch 2.0 or newer, and a CUDA 7.0 or newer GPU. That covers everything from a V100 through Blackwell. The current release also lights up on RTX 3090, 4090, and 5090, plus the standard datacenter parts. Apple Silicon is in progress; MLX training is marked "coming soon" in the [requirements page](https://unsloth.ai/docs/get-started/fine-tuning-for-beginners/unsloth-requirements).

### Fine-tune Llama 3.1 8B in a few lines

```
from unsloth import FastLanguageModel
import torch

model, tokenizer = FastLanguageModel.from_pretrained(
model_name = "unsloth/Meta-Llama-3.1-8B",
max_seq_length = 2048,
dtype = None,
load_in_4bit = True,
)

model = FastLanguageModel.get_peft_model(
model,
r = 16,
target_modules = ["q_proj","k_proj","v_proj","o_proj",
"gate_proj","up_proj","down_proj"],
lora_alpha = 16,
use_gradient_checkpointing = "unsloth",
)
```

From there you point a SFTTrainer from TRL at your dataset and let it run. The library ships notebooks for every supported model on the [model catalog page](https://unsloth.ai/docs/get-started/unsloth-model-catalog), so the cold-start path is "click the notebook, swap your dataset path, run."

## Model support

The catalog is wide. The major families currently maintained: Llama 4, 3.3, 3.2, 3.1; Mistral 3.2, 3.1, Ministral; Gemma 4, 3n, 2; Qwen 3.6, 3.5, 3, 2.5; Phi 4; DeepSeek-V3 and R1; GLM. Vision-language models, audio models, and Whisper-class TTS fine-tuning are also supported via the [vision and multimodal collection](https://huggingface.co/collections/unsloth/vision-multimodal-models).

Output formats matter as much as input support. Unsloth ships exporters for 4-bit, 8-bit, and 16-bit safetensors, GGUF for llama.cpp and Ollama, and the vLLM format for production serving. The path from "I fine-tuned a model" to "I am serving it on my own hardware" is one CLI command, not an afternoon of conversion scripts.

## Beyond plain SFT

Reward modeling and preference optimization are first-class. The [RLHF docs](https://docs.unsloth.ai/basics/reward-modelling-dpo-and-orpo) cover DPO, ORPO, KTO, and GRPO with example notebooks. Recent releases added vision fine-tuning and audio model training, and the catalog now includes speech models like Sesame and Orpheus. The pace is unusual for a project this size: notable feature drops land monthly, and minor releases land weekly.

## The two-person team

Unsloth is built by [Daniel Han](https://x.com/danielhanchen) and his brother Michael, brothers from Sydney who went through Y Combinator's summer 2024 batch. Daniel previously worked on optimized t-SNE and SVD at NVIDIA, which is exactly the right background for the hand-written kernel work that drives Unsloth's speedups. The pair posts kernel-level deep dives on the [Unsloth blog](https://unsloth.ai/about) and Twitter, and most of the framework's heavyweight performance jumps have been written up there before they ship.

## Who uses Unsloth in the wild

The fingerprint shows up across the open-weights ecosystem. Hugging Face used Unsloth for [SmolLM3-3B](https://huggingface.co/unsloth/SmolLM3-3B), and the documentation pages call out direct collaborations with Qwen, Mistral, NVIDIA, and Microsoft. Beyond the named users, scanning HuggingFace model cards for "fine-tuned with Unsloth" returns thousands of community results across every base family the library supports.

## Language-specific fine-tuning is now trivial

One under-rated use case is language-specific instruction tuning on a small base. If you want a Polish-speaking Llama 3.2 3B that performs better on Polish dialogue than the multilingual base, Unsloth is the path of least resistance: the [Qwen3.5 documentation](https://unsloth.ai/docs/models/qwen3.5/fine-tune) covers 201 languages and Gemma 4 covers 140, so the underlying models already speak Polish (or Czech, or Slovenian, or any of the 200-language tail). Adding a few thousand Polish instruction-output pairs is a single notebook away. The catch most teams hit is dataset quality, not training infrastructure.

## Limitations and gotchas

Unsloth is still a single-GPU framework at heart. Multi-GPU support exists but is less battle-tested than the single-GPU path. The "2x speedup" headline depends on baseline; against a naive HuggingFace pipeline the speedup is sometimes much larger, and against a hand-tuned Flash Attention 3 pipeline it's smaller. Numbers in [the published methodology](https://unsloth.ai/docs/blog/3x-faster-training-packing) are honest about that. And the GGUF and vLLM export paths assume llama.cpp or vLLM in the latest minor; older runtime versions occasionally choke on tokenizer additions.

## Who should care

If you fine-tune on anything smaller than an 8-GPU H100 node, you should be using Unsloth. The library is free, the model coverage is comprehensive, the export path lands on every common inference runtime, and the maintainer cadence is healthy. The cost of trying it is one pip install and a sample notebook. Pick a base model, grab a thousand examples, and have your first fine-tune running in the time it takes to read this article.

## Sources and further reading

- [Unsloth on GitHub](https://github.com/unslothai/unsloth)

- [Unsloth model catalog](https://unsloth.ai/docs/get-started/unsloth-model-catalog)

- [Hardware and software requirements](https://unsloth.ai/docs/get-started/fine-tuning-for-beginners/unsloth-requirements)

- [Benchmark methodology: 3x faster training with packing](https://unsloth.ai/docs/blog/3x-faster-training-packing)

- [DPO / ORPO / KTO / GRPO documentation](https://docs.unsloth.ai/basics/reward-modelling-dpo-and-orpo)

- [Daniel Han on X](https://x.com/danielhanchen)

Numbers community-reported from Unsloth's published methodology and HuggingFace model card metadata. Not independently re-run. Compiled 2026-05-19.

---


# Heretic: One-Command Abliteration
Source: https://singularitybyte.com/tools/heretic-one-command-abliteration.html

Abliteration used to be a research move that took afternoon-long PyTorch sessions, hand-picked layers, and a stomach for spending weights you might never get back. Heretic compresses that into one command. pip install -U heretic-llm, then heretic Qwen/Qwen3-4B-Instruct-2507, and roughly 45 minutes later you have an uncensored variant whose damage to the base model is smaller than anything a human has hand-tuned for it. There are already more than a thousand community Heretic variants on HuggingFace, and the wait between a fresh open-weights release and its decensored twin is collapsing toward zero.

## Why abliteration suddenly matters to open-source builders

For anyone running local models, safety alignment is a tax. It pays for itself when you ship a consumer chatbot, but it gets in the way of agent loops, red-team work, fiction, security research, creative writing, and dozens of legitimate uses that hit a wall the moment the model decides to apologize. Until recently, the only way around it was a curated jailbreak prompt or a full fine-tune.

Heretic kills the wait. The day a new base model lands on HuggingFace, somebody runs heretic against it, pushes the result, and within hours [bartowski](https://huggingface.co/bartowski) and [mradermacher](https://huggingface.co/mradermacher) have GGUFs out for every common quantization. That is what the 1,000+ Heretic uploads on HuggingFace actually represent: an end-to-end pipeline where any aligned open model is one command and a few hours of community labor away from being usable for whatever you actually want to do with it.

## What abliteration actually is

Abliteration is a surgical edit to a model's weights, not a fine-tune. It rests on a 2024 result from [Arditi et al.](https://arxiv.org/abs/2406.11717) (NeurIPS 2024), which showed that refusal in transformer language models is mediated by a single direction in the residual stream. That direction is consistent across 13 open-weights chat models, all the way up to 72B parameters. Project it out of the activations, or subtract it from the relevant weight matrices, and the model stops refusing without losing the rest of its skills.

The technique reached a wider audience through [Maxime Labonne's HuggingFace blog post](https://huggingface.co/blog/mlabonne/abliteration), which turned the Arditi paper into a recipe anyone with a GPU could follow. The recipe worked, but it was finicky. You had to pick which layers to read activations from, decide how strongly to project, and eyeball whether the resulting model was still coherent. Get the layer wrong and the model talked freely but lost its math. Project too weakly and it still refused.

## How Heretic differs from the manual approach

Heretic treats abliteration as an optimization problem. It uses [Optuna's Tree-structured Parzen Estimator](https://optuna.readthedocs.io/en/stable/reference/samplers/generated/optuna.samplers.TPESampler.html) to search the joint space of layer ranges, ablation weights, and direction indices, co-minimizing two objectives at every trial: the count of refusals on a harmful-prompt probe set, and the KL divergence between the patched model and the original. The model that comes out the other end is the one that refuses least while drifting least.

The numbers from the repo are striking. A search that takes 30 to 90 minutes on a single RTX 3090 now outperforms expert tuning that took days.

Approach (Gemma-3-12B-IT) | KL divergence vs base | Refusals / 100 | Human effort
Heretic, default config | 0.16 | 3 | One CLI command
Best hand-tuned abliteration | 1.04 | ~3 | Days of expert tuning
Same refusal rate, roughly 6.5 times less damage to the base model's behavior, and the cost of producing the Heretic row is one terminal command on a consumer GPU.

Heretic builds on extensions to the original technique, including Lai 2025's [projected abliteration](https://huggingface.co/blog/grimjim/projected-abliteration) and [norm-preserving biprojected abliteration](https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration). The author, [p-e-w](https://github.com/p-e-w) (Philipp Emanuel Weidmann), keeps the codebase under AGPL-3.0 and ships sensible defaults so the typical run is a single argument.

## Hands-on: from pip install to your own variant

### Install

```
pip install -U heretic-llm
```

Heretic needs Python 3.10 or newer, PyTorch 2.2+ (2.6 recommended), and a CUDA GPU. An RTX 3090 with 24 GB of VRAM is the sweet spot for 8B to 12B models. Hybrid models like Qwen3.5 work, multimodal models work, and most MoE architectures work. Pure state-space models and a handful of research architectures are still on the to-do list.

### Run

```
heretic Qwen/Qwen3-4B-Instruct-2507
```

That is the whole interface. Heretic streams the model, benchmarks batch sizes on your GPU, runs the optimizer for a few dozen trials, prints the final KL and refusal numbers, and writes the patched weights to disk. If you want to bias the search toward retaining more or refusing less, the config.default.toml and config.noslop.toml presets in the repo are good starting points.

### Quantize and share

Patched weights are large. The community convention is to upload the full-precision Heretic variant to your HuggingFace account, then wait a day or two for bartowski or mradermacher to publish imatrix GGUFs in the usual Q4_K_M, Q5_K_M, and IQ4_XS flavors. If you cannot wait, the llama.cpp conversion scripts handle the same job in one pass.

## Names to know in the scene

[p-e-w](https://github.com/p-e-w) is the author of Heretic and the maintainer of the [heretic-org](https://huggingface.co/heretic-org) HuggingFace organization. The repo is the canonical reference; the org curates clean Heretic variants for popular base models.

[mlabonne](https://huggingface.co/mlabonne) popularized abliteration in the first place and continues to ship merges, fine-tunes, and educational material. His blog is the recommended entry point for anyone who wants to understand what the projection step is doing.

[huihui-ai](https://huggingface.co/huihui-ai) runs the most prolific uncensoring operation on HuggingFace, with 200+ variants spanning Qwen, DeepSeek, Llama, Gemma, and most major open families. If a base model dropped last week, huihui-ai almost certainly has an abliterated version of it already.

[DavidAU](https://huggingface.co/DavidAU) takes things further. His "Dark Champion" line is a series of MoE merges that combine abliterated experts with creative-writing fine-tunes, and the resulting models have a cult following among fiction writers and roleplayers running local stacks. The [Llama-3.2-8X3B Dark Champion](https://huggingface.co/DavidAU/Llama-3.2-8X3B-MOE-Dark-Champion-Instruct-uncensored-abliterated-18.4B) is a representative entry.

[bartowski](https://huggingface.co/bartowski) and [mradermacher](https://huggingface.co/mradermacher) are the GGUF quant-masters. Between them they keep current quantizations available for essentially every model the local-LLM crowd cares about. They are unpaid infrastructure for the entire ecosystem, and if you run Ollama, LM Studio, or llama.cpp you have almost certainly downloaded one of their files.

## Limitations, ethics, and the AGPL question

Abliteration removes the refusal behavior. It does not remove the training distribution. A Heretic variant of a model that learned bad chemistry from filtered web text still does not know good chemistry. It will answer; the answer can be wrong. The same goes for medical advice, legal advice, and any other domain where the base model was already weak. Treat the output the same way you would treat any local model: as a draft from a confident-sounding intern.

The legal frame matters too. Heretic itself is AGPL-3.0, which means any service that exposes the tool over a network has to make its source available. The model weights Heretic produces are governed by the upstream license of whatever base model you patched. A Heretic variant of Llama-3 still inherits the Llama license. A Heretic variant of Qwen is still Qwen-licensed. Read those before redistribution.

## Sources and further reading

- [Heretic on GitHub](https://github.com/p-e-w/heretic) (canonical repo, README, configs)

- [heretic-llm on PyPI](https://pypi.org/project/heretic-llm/)

- [heretic-org on HuggingFace](https://huggingface.co/heretic-org) (curated Heretic variants)

- [Arditi et al. 2024, "Refusal in Language Models Is Mediated by a Single Direction"](https://arxiv.org/abs/2406.11717) (NeurIPS 2024)

- [mlabonne, "Uncensor any LLM with abliteration"](https://huggingface.co/blog/mlabonne/abliteration)

- [Lai 2025, "Projected Abliteration"](https://huggingface.co/blog/grimjim/projected-abliteration)

- [Lai 2025, "Norm-Preserving Biprojected Abliteration"](https://huggingface.co/blog/grimjim/norm-preserving-biprojected-abliteration)

- [Optuna](https://optuna.org/) (the TPE optimizer backing Heretic)

Benchmarks community-reported from p-e-w's repo and HuggingFace variant cards. Not independently verified. Compiled 2026-05-19.

---


## News

# MCP 2026-07-28 Spec: Stateless Core, Auth Hardening, RufRoot Fallout
Source: https://singularitybyte.com/news/mcp-spec-2026-07-28-stateless-auth-hardening.html

Two MCP stories landed inside one 48-hour window, and they belong in the same article. On July 28, the Agentic AI Foundation published the 2026-07-28 revision of the [Model Context Protocol](/tools/opencode.html), the standard your agents use to call external tools. It is the largest change since remote MCP servers arrived: the handshake is gone, sessions are gone, and OAuth grows teeth. A day or two later, researchers went public with CVE-2026-59726, a CVSS 10.0 hole in Ruflo's MCP bridge that handed 233 tools, shell execution included, to anyone who could reach port 3001.

If you build or run MCP servers, both stories are about you. The protocol you target changed shape this week, and the failure mode the new auth rules exist to prevent just got a name, a nickname, and a patch you should already be running.

## The MCP 2026-07-28 spec: what shipped

- The stateless core removes the initialize handshake and the Mcp-Session-Id header. Every request describes itself, so any server instance can answer it (SEP-2575, SEP-2567).

- Authorization gets hardened: RFC 9207 issuer validation becomes mandatory, client credentials are bound to the server that issued them, and Dynamic Client Registration is deprecated in favor of Client ID Metadata Documents.

- Deprecations now run on a formal clock: a feature keeps working at least 12 months after deprecation, shortened to 90 days only for security emergencies.

The spec [went final on July 28](https://blog.modelcontextprotocol.io/posts/2026-07-28/) under the Agentic AI Foundation, the Linux Foundation directed fund that now stewards the protocol Anthropic open-sourced in late 2024. The release candidate was locked on May 21 and then sat through ten weeks of validation by SDK maintainers and client implementers before publication. Lead maintainer David Soria Parra is not underselling it: "The new release is MCP's most important since remote MCP first launched over a year ago."

The scale explains the caution. The Tier 1 SDKs (TypeScript, Python, Go and C#) pull close to half a billion downloads a month, with TypeScript and Python each past one billion total. All four speak 2026-07-28 as of publication day with migration notes included, Rust support is in beta, and FastMCP 4.0 ships stateless out of the box.

## Stateless core: the handshake is gone

Until now, every MCP connection opened with an initialize/initialized exchange that negotiated versions and capabilities, and remote transports pinned the result to an Mcp-Session-Id header. That made every server stateful by default. Load balancers needed sticky sessions, a restart dropped your clients, and scaling horizontally meant shared session storage.

2026-07-28 deletes the ceremony. The handshake and the session header are gone, and each request carries its protocol version, client identity and capabilities in the _meta field, so the server learns everything it needs from the request itself:

```

{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"_meta": {
"io.modelcontextprotocol/protocolVersion": "2026-07-28",
"io.modelcontextprotocol/clientInfo": { "name": "ExampleClient", "version": "1.0.0" },
"io.modelcontextprotocol/clientCapabilities": { "elicitation": {} }
},
"name": "confirmThenGreet",
"arguments": {}
}
}

```

That wire format is lifted straight from the Go SDK's 2026-07-28 conformance suite, not paraphrased. A version mismatch returns UnsupportedProtocolVersionError, and clients that want to probe first can call the new server/discover RPC, which advertises supported versions and capabilities. The payoff: any instance can answer any request, so a plain round-robin load balancer is enough. The servers you already wire into [Claude Code](/tutorials/claude-ai-tutorial-claude-code.html) get simpler to deploy, not harder to use.

## Multi round-trip requests replace server-initiated calls

MCP servers could previously call the client mid-request: elicitation/create to ask the user something, sampling/createMessage to request a completion, roots/list to inspect the workspace. Server-initiated calls are the natural enemy of statelessness, so SEP-2322 inverts the flow. A server that needs input finishes its response instead, returning resultType "input_required" plus a machine-readable list of what it needs:

```

{
"resultType": "input_required",
"requestState": "step=1",
"inputRequests": {
"who": {
"method": "elicitation/create",
"params": {
"mode": "form",
"message": "What is your name?",
"requestedSchema": {
"type": "object",
"properties": { "name": { "type": "string" } }
}
}
}
}
}

```

The client gathers answers and retries the original call, attaching "inputResponses": {"who": {"action": "accept", "content": {"name": "MCP Go"}}} and echoing the opaque requestState token back. The server then completes with resultType "complete". No channel held open to the client, no state parked on the server between the two calls, and the SDKs run the retry loop for you by default.

## Routing headers and cacheable lists

Two smaller changes matter to anyone running MCP behind real infrastructure. SEP-2243 requires clients to send Mcp-Method and Mcp-Name HTTP headers with each call, so gateways, WAFs and metering proxies can route on the operation without parsing JSON bodies. Your rate limiter can treat a shell-adjacent tools/call differently from a harmless resources/read without deep packet inspection.

SEP-2549 makes list results cacheable: tools/list, prompts/list, resources/list and resources/read responses now carry ttlMs and cacheScope fields. Caching a tool catalog used to be pointless because lists could vary per connection. Stateless servers return the same lists to everyone, so a CDN can finally hold them for the advertised lifetime.

## Auth hardening: issuer checks, bound credentials, CIMD

The authorization changes read like a direct response to a year of agent-security incidents. Clients must validate the iss parameter of an authorization response before redeeming the code (RFC 9207, SEP-2468). That blocks mix-up attacks, where a malicious server maneuvers a client into spending its credentials at the wrong authorization server. Client credentials are also bound to the server that issued them (SEP-2352): if a resource migrates, the client re-registers instead of reusing credentials across servers.

Token audience is enforced at protocol level through RFC 8707 resource indicators: a token minted for server A is dead on arrival at server B, closing the confused-deputy replay that [security write-ups have warned about](https://workos.com/blog/mcp-2026-spec-agent-authentication) since 2025. Operators must publish RFC 9728 metadata at /.well-known/oauth-protected-resource. Dynamic Client Registration is formally deprecated in favor of Client ID Metadata Documents (CIMD), with backward compatibility for authorization servers that have not caught up, and the new application_type parameter finally makes localhost redirects behave for desktop and CLI apps (SEP-837).

All of this assumes one thing: that your bridge has authentication to harden. Last week showed how low the ecosystem's floor still sits.

## RufRoot: the CVSS 10.0 argument for all of this

Ruflo is the multi-agent orchestration platform that started life as Claude Flow, with more than 66,500 GitHub stars. Its MCP bridge, the Express.js server that fronts all tool invocations, shipped a default docker-compose.yml that bound port 3001 to 0.0.0.0 and answered POST /mcp with no authentication of any kind. [Noma Labs researcher Eli Ainhorn found](https://noma.security/blog/rufroot-the-mcp-bridge-vulnerability-that-turns-agents-into-rogue-admins-cve-2026-59726/) that one unauthenticated HTTP POST reached 233 tools, including ruflo__terminal_execute.

Item | Detail
CVE | CVE-2026-59726 "RufRoot"
CVSS | 10.0
Affected | Ruflo (ex Claude Flow), all versions before 3.16.3
Exposure | MCP bridge on port 3001, bound to 0.0.0.0 by default docker-compose
Auth | None on POST /mcp and POST /mcp/:group
Surface | 233 tools: shell execution, database ops, agent management, memory store
Found by | Noma Labs (Eli Ainhorn), reported June 30
Fix | v3.16.3 within 24 hours: loopback bind, terminal_execute gated, MongoDB auth on
From that single request an attacker could run shell commands inside the container, read every LLM provider key from the environment, dump stored conversations from the bundled MongoDB, drop a payload into /app for persistence, and poison the AgentDB pattern store so the platform's agents learn attacker-chosen behavior. Spawning agent swarms billed to the victim's API keys was the closing insult. [The public write-ups landed this week](https://thehackernews.com/2026/07/ruflo-mcp-flaw-lets-unauthenticated.html).

Credit where due: maintainer Reuven Cohen shipped 3.16.3 within 24 hours of the June 30 report. The lesson is not that one project was uniquely careless. The lesson is that "an MCP bridge on 0.0.0.0 with no auth" was a shippable default in 2026 at all. A spec cannot patch a door that ships open, but audience-bound tokens and mandatory issuer checks are the same class of failure finally taken seriously at protocol level.

## Extensions, deprecations, and the 12-month clock

Tasks leave the experimental core and become the io.modelcontextprotocol/tasks extension. The blocking tasks/result call is removed, long-running work is polled through tasks/get, and notification streams collapse into a single subscriptions/listen per type (SEP-2663). Roots, sampling and logging are deprecated (SEP-2577), and the legacy HTTP+SSE transport starts a one-year goodbye in favor of stateless Streamable HTTP.

Deprecated does not mean dead. The new feature lifecycle guarantees at least twelve months between deprecation and the earliest possible removal, with a 90-day exception reserved for security emergencies. Nothing switches off on you today: servers speaking 2025-11-25 keep working, and nobody is forced to upgrade this quarter. The clock is real, though, and it started on Monday.

## What this means for your server

We run MCP servers in production ourselves, so here is the migration map we are actually working from, not a rewording of the changelog.

Change | SEP | If you run a server today | Do this
Handshake removed | 2575 | Requests carry version and capabilities in _meta; mismatches return UnsupportedProtocolVersionError | Read _meta per request; implement server/discover
Sessions removed | 2567 | List results no longer vary per connection; sticky routing is pointless | Mint explicit handles, pass them as ordinary tool arguments
Server-initiated calls gone | 2322 | elicitation/create, sampling/createMessage, roots/list no longer originate server-side | Return resultType "input_required", handle inputResponses retries
Blocking tasks/result removed | 2663 | Long-running calls no longer block | Adopt the tasks extension, poll tasks/get
OAuth tightened | 2468, 2352 | Clients validate iss; credentials stop working across auth servers | Serve RFC 9728 protected-resource metadata; publish a CIMD
HTTP+SSE deprecated | transport | Works for one more year | Move to stateless Streamable HTTP
The session removal is the one that changes your code. If your server stashes per-connection state behind Mcp-Session-Id, that pattern is dead: mint explicit handles and pass them as ordinary tool arguments, the way REST APIs have passed resource IDs forever. It is more honest anyway, since hidden session state is exactly how list results ended up varying per connection. Read _meta on every request, implement server/discover, and delete your handshake path with prejudice.

### Your 10-minute audit

RufRoot's root cause was not exotic. It was a listener on the wrong interface with nothing in front of it. Check your own machines tonight:

```

# Anything bound to 0.0.0.0 on these ports answers the internet, not just you
ss -tlnp | grep -E ':(3001|27017)'

# Probe your own bridge: anything but 401, 403 or connection refused
# means it talks to strangers
curl -s -o /dev/null -w '%{http_code}\n' -X POST http://localhost:3001/mcp \
-H 'Content-Type: application/json' -d '{}'

```

If you run Ruflo, upgrade to 3.16.3, close ports 3001 and 27017 at the firewall, rotate every LLM key the container could read, and audit the AgentDB store for patterns you did not put there. If you [exposed an n8n workflow as an MCP tool](/tutorials/n8n-mcp-server-expose-workflow-as-ai-tool-2026.html), ask the same question: who can reach that endpoint, and what tells them no? Run the two commands above before you close this tab. That is the whole ten minutes.

## Sources and further reading

- [The 2026-07-28 Specification, official announcement](https://blog.modelcontextprotocol.io/posts/2026-07-28/)

- [MCP specification 2026-07-28, full text](https://modelcontextprotocol.io/specification/2026-07-28)

- [modelcontextprotocol releases on GitHub](https://github.com/modelcontextprotocol/modelcontextprotocol/releases)

- [Noma Security: RufRoot advisory (CVE-2026-59726)](https://noma.security/blog/rufroot-the-mcp-bridge-vulnerability-that-turns-agents-into-rogue-admins-cve-2026-59726/)

- [The Hacker News: Ruflo MCP flaw report](https://thehackernews.com/2026/07/ruflo-mcp-flaw-lets-unauthenticated.html)

- [WorkOS: what the 2026 spec changes for agent authentication](https://workos.com/blog/mcp-2026-spec-agent-authentication)

- [SingularityByte: turn an n8n workflow into an MCP tool](/tutorials/n8n-mcp-server-expose-workflow-as-ai-tool-2026.html)

Tested on: not independently tested. No 2026-07-28 SDK ran on our bench long enough for a compliance pass; spec details are read from the official announcement and spec text, the JSON wire formats are copied verbatim from the Go SDK's conformance suite, and RufRoot details come from the Noma Labs advisory and The Hacker News reporting, not reproduced by us.

Date checked: 2026-07-31

---


# Run Kimi K3 Locally: Unsloth GGUFs Land at 594 GB
Source: https://singularitybyte.com/news/run-kimi-k3-locally-unsloth-gguf.html

When the [Kimi K3 weights landed on July 27](/news/kimi-k3-open-weights.html), our take was blunt: 1,561 GB, multi-node or nothing, and any day-one community quant deserves scrutiny. Three days later, the scrutiny has a serious target. [Unsloth published Dynamic GGUFs of Kimi K3](https://unsloth.ai/docs/models/kimi-k3) starting at 594 GB, plus a llama.cpp branch that runs the model with its vision tower intact. The largest open-weight model ever shipped is now a single-box problem. A very large box.

## Run Kimi K3 locally: what shipped

- [unsloth/Kimi-K3-GGUF](https://huggingface.co/unsloth/Kimi-K3-GGUF) holds six Dynamic quants of the 2.8T-parameter MoE, from a 594 GB 1-bit (UD-IQ1_S) to a near-lossless 1,560 GB Q8, shipped as split GGUF files that llama.cpp loads directly.

- Vision works. You build Unsloth's llama.cpp branch kimi-k3-fullsize-vision and pass the bundled mmproj-BF16.gguf projector file alongside the model.

- Moonshot's recommended settings carry over: temperature 1.0 with top_p 0.95, or top_p 1.0 for agent work. Thinking is always on, with low, high and max effort levels.

## The hardware bill

Unsloth's rule of thumb: combined RAM plus VRAM roughly equal to the quant size. Less still works through disk offloading, just much slower. Here is the honest table.

Quant | Disk size | RAM+VRAM for full speed
UD-IQ1_S (Dynamic 1-bit) | 594 GB | 610 GB
UD-IQ1_M | 648.9 GB | 665 GB
UD-IQ2_XXS | 711.1 GB | 726 GB
UD-Q2_K_XL | 861.3 GB | 880 GB
UD-Q4_K_XL | 1,510 GB | not listed
UD-Q8_K_XL (near-lossless) | 1,560 GB | 1.6 TB
So "locally" means an NVIDIA DGX Station, a fat EPYC box, or, per Unsloth, "a Mac Studio connected to a 128GB RAM device". That last one adds up: a 512 GB M3 Ultra plus one 128 GB machine totals 640 GB, which clears the 1-bit line. On B200-class hardware that fits the whole quant, Unsloth reports about 20 tokens per second generation and over 120 tokens per second throughput. Your 4090 is not in this story, and we respect you too much to pretend otherwise.

## Run it today

Build the branch, then pull the 1-bit quant and the vision projector. The 1-bit alone is 14 split files, so budget the disk and the weekend.

```

git clone https://github.com/unslothai/llama.cpp
cd llama.cpp
git fetch origin pull/48/head:kimi-k3-fullsize-vision
git checkout kimi-k3-fullsize-vision
cd ..
cmake llama.cpp -B llama.cpp/build -DBUILD_SHARED_LIBS=OFF -DGGML_CUDA=ON
cmake --build llama.cpp/build --config Release -j --clean-first

pip install -U "huggingface_hub[cli]"
hf download unsloth/Kimi-K3-GGUF --local-dir unsloth/Kimi-K3-GGUF \
--include "*mmproj-BF16*" --include "*UD-IQ1_S*"

```

Then point llama-cli at the first shard and the projector:

```

./llama.cpp/build/bin/llama-cli \
--model unsloth/Kimi-K3-GGUF/UD-IQ1_S/Kimi-K3-UD-IQ1_S-00001-of-00014.gguf \
--mmproj unsloth/Kimi-K3-GGUF/mmproj-BF16.gguf \
--temp 1.0 --top-p 0.95

```

One gotcha for agent builders, straight from the model card: K3 always thinks and returns reasoning_content, and multi-turn calls must feed the complete assistant message back into history, reasoning and tool calls included, not just the final text.

## Why this matters, and the caveat

Three days ago, self-hosting K3 meant roughly 19 H100s just to load the raw MXFP4 weights. A 610 GB RAM+VRAM target moves that to one serious workstation, and K3's linear-attention design keeps the KV cache tame if you push the 1M context. The caveat: a Dynamic 1-bit is not uniformly 1-bit. [Unsloth, the two-person outfit we profiled](/tools/unsloth-the-indie-fine-tuning-backbone.html), mixes bit-widths per tensor, and nobody has published quality numbers for these exact files yet. The Hugging Face counter already shows 12,178 downloads, which is a lot of terabytes pulled on faith.

Your under-10-minutes move: check whether any machine you own clears the line before you burn 594 GB of disk finding out.

```

# Your RAM + VRAM in GB, against the 610 GB the 1-bit wants.
echo "$(( $(free -g | awk '/Mem:/ {print $2}') + $(nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits 2>/dev/null | awk '{s+=$1} END {print int(s/1024)}') )) GB available"

```

If the number is a few hundred GB short, [the hosted API remains the sane path](/models/kimi-k3.html), and [Kimi K2.7 Code](/models/kimi-k2-7-code.html) still covers the single-node crowd.

## Sources and further reading

- [Unsloth: Kimi K3 local deployment guide (sizes, settings, commands)](https://unsloth.ai/docs/models/kimi-k3)

- [unsloth/Kimi-K3-GGUF on Hugging Face (all six quants + mmproj)](https://huggingface.co/unsloth/Kimi-K3-GGUF)

- [unslothai/llama.cpp, branch kimi-k3-fullsize-vision](https://github.com/unslothai/llama.cpp)

- [moonshotai/Kimi-K3 (original weights)](https://huggingface.co/moonshotai/Kimi-K3) and [the Kimi K3 License](https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE), which applies to these quants too

- [SingularityByte: what actually shipped in the K3 weights drop](/news/kimi-k3-open-weights.html)

- [SingularityByte: Kimi K3 model breakdown](/models/kimi-k3.html)

Tested on: not independently tested. The smallest Kimi K3 quant needs about 610 GB of combined RAM and VRAM, which is beyond our bench. Quant sizes, hardware guidance and throughput figures are Unsloth-reported; sampling settings and the multi-turn rule are from the model card; file sizes and download counts were read from the Hugging Face repo.

Date checked: 2026-07-30

---


# Kimi K3 Open Weights Are Live: What Actually Shipped
Source: https://singularitybyte.com/news/kimi-k3-open-weights.html

Moonshot AI made the Kimi K3 weights public on July 27, 2026, and the largest open-weight model anyone has shipped is now a download. The repo is moonshotai/Kimi-K3: 114 files, 1,561 GB, one custom license. That last part matters, because most of what got published about this release before it happened was wrong, including the repository name, the license, the file size, and the hardware you need. We read the actual repo. Here is what shipped.

## TL;DR

- What it is: a 2.8-trillion-parameter Mixture-of-Experts model with 104B active parameters, 93 layers, a 1,048,576-token context window, and native text, image and video input. Quantization-aware trained in MXFP4.

- Why it matters: the first open 3T-class model, and on Moonshot's own numbers it trades the lead with Claude Fable 5 and GPT-5.6 Sol across coding and agentic suites. You can audit it, fine-tune it, and serve it yourself.

- The catch: 1,561 GB of weights. That is roughly 19 H100s just to hold the model, before you allocate a single token of KV cache. And the license is not Apache 2.0.

## What actually landed

One repository, not the family of base, instruct and quantized variants that usually accompanies a release. Everything is in moonshotai/Kimi-K3, and the weights are already quantized: there is no separate FP16 checkpoint to download.

Property | Value (read from the repo)
Repository | moonshotai/Kimi-K3, public, not gated
Files / total size | 114 files, 1,561 GB (1,453.8 GiB)
License | Kimi K3 License (custom, license: other)
Total parameters | 2.8T
Activated parameters | 104B
Layers | 93 (1 dense), 69 KDA plus 24 Gated MLA
Experts | 896 routed, 16 selected per token, 2 shared
Context window | 1,048,576 tokens
Vocabulary | 163,840
Modalities | Text, image, video in; text out
Quantization | MXFP4 weights, MXFP8 activations, QAT from the SFT stage
### The license is custom, and clause 2 has teeth

It is not Apache 2.0, despite that being reported widely in the days before the drop. It is a bespoke "Kimi K3 License", MIT-shaped at the core with two commercial conditions bolted on. Read it yourself, it is short, but here is the shape of it.

Clause 2 targets Model-as-a-Service specifically. If you give third parties access to inference or fine-tuning in a way that lets them control inputs, parameters or training data, and you or your affiliates clear 20 million US dollars in aggregate revenue over any consecutive 12 months, you must sign a separate agreement with Moonshot before any commercial use. Note the trigger is total company revenue, not revenue from the model.

Clause 3 is an attribution rule. Ship a product with more than 100 million monthly active users or more than 20 million US dollars in monthly revenue, and "Kimi K3" has to appear prominently in your user interface.

Clause 4 is the escape hatch most readers will land in: neither condition applies to internal use, defined as use that does not expose the model, its outputs, or its capabilities to third parties. Run it inside your own company and you are clear. Resell inference at scale and you are negotiating.

## The numbers everyone published before the file listing existed

This is worth your attention as a builder, because it is now a pattern. In the week before the drop, a wave of posts appeared with hardware tables, vLLM commands, and download instructions for a file that did not exist yet. We checked the Hugging Face API repeatedly across that window: the moonshotai org held 18 models, newest Kimi K2.7 Code from June 15, and the K3 page was a countdown placeholder with a notify button.

Now that the real repo is up, we can score the claims.

Widely published claim | Reality
Download from moonshotai/Kimi-K3-MXFP4 | Never existed. The repo is moonshotai/Kimi-K3
About 594 GB | 1,561 GB. Off by 2.6x
About 1.4 TB | Closer. Actual is 1.45 TiB, so right if you meant tebibytes
Apache 2.0 license | Custom Kimi K3 License with revenue-triggered clauses
Roughly 50B active parameters | 104B
8x H100 minimum to load it | 640 GiB against 1,453.8 GiB of weights. Short by 2.3x
None of these were malicious. They were written ahead of a traffic event and nobody checked, because there was nothing to check against. The practical lesson for anyone about to pull a terabyte and a half: ask the API, not a blog. It takes one command. We went through the same exercise when [Wan 2.7 was widely reported as open-weights and was not](/tutorials/open-source-video-models-mid-2026-what-changed.html), and the method there works for any release claim.

```

# What has a lab actually published? No repo, no download,
# regardless of what any article says.
curl -s "https://huggingface.co/api/models?author=moonshotai" \
| python3 -c "import sys,json;[print(m['modelId'], m.get('lastModified')) for m in json.load(sys.stdin)]"

# Real file list, real byte total, real license field:
curl -s "https://huggingface.co/api/models/moonshotai/Kimi-K3?blobs=true" \
| python3 -c "import sys,json;d=json.load(sys.stdin);s=d.get('siblings',[]);\
print('license:', d.get('cardData',{}).get('license_name'));\
print('files:', len(s));\
print('total GB:', round(sum(f.get('size') or 0 for f in s)/1e9, 1))"

```

## The architecture, and why 1M context is affordable here

K3 runs 93 layers, and the attention is split: 69 layers use Kimi Delta Attention (KDA), Moonshot's linear attention variant, and only 24 use Gated MLA, the full-attention path. That ratio is the interesting part. Linear attention carries a fixed-size recurrent state instead of a KV cache that grows with sequence length, so at a 1M-token context only those 24 layers accumulate a conventional cache. That is roughly a quarter of the model paying full freight for long context, which is how a 1M window becomes tractable at all.

The MoE side is unusually sparse: 896 routed experts with 16 selected per token, plus 2 shared experts that run every time, routed by a sigmoid gate. Moonshot claims about 2.5x better overall scaling efficiency than K2 from this combination. Activated parameters land at 104B, so you store 2.8T and compute roughly 104B per token.

The tech report published alongside the weights adds a detail worth flagging: K3 uses no explicit positional embedding at all. There is no RoPE, and therefore no RoPE rescaling or interpolation to reach long context. Position is encoded implicitly through KDA's recurrent gating and decay, which is why the model extrapolates to 1M tokens without the usual positional surgery. Training grew the window in four stages, 8K to 64K during pre-training and 256K to 1M during cooldown, which keeps the expensive long-sequence compute inside a small slice of the budget.

Here is the part almost nobody covering the drop mentioned: both headline architecture pieces have been open on GitHub for months. [FlashKDA](https://github.com/MoonshotAI/FlashKDA), the KDA kernels, went up April 20, 2026 under MIT and sits at 473 stars. [Attention-Residuals](https://github.com/MoonshotAI/Attention-Residuals) has been public since March 15, 2026, now past 3,390 stars. You could read and benchmark K3's attention design roughly four months before you could download K3. If you build your own long-context stack, those repos are worth more to you than the checkpoint.

### What MXFP4 actually covers

Moonshot applied quantization-aware training from the supervised fine-tuning stage onward, using MXFP4 weights with MXFP8 activations. This is not a post-training squeeze, so the usual "the quant broke it" caveat does not apply the same way.

But the 4-bit format does not cover the whole model. The config's ignore list keeps self-attention, the shared experts, the dense MLP projections, the LM head, the vision tower and the multimodal projector out of MXFP4. Only the routed expert Linear layers are packed to 4 bits, at group size 32. That is exactly why 2.8T parameters still weigh 1,561 GB rather than the ~1.4 TB a naive 4-bit calculation gives you.

## What it takes to run

Blunt version: this is a multi-node deployment or nothing. The weights alone need 1,453.8 GiB resident before you allocate KV cache, activations, or any headroom for fragmentation.

Accelerator | GPUs for weights only | Realistic with headroom | Nodes of 8
H100 80GB | 19 | 23 | 3
H200 141GB | 11 | 13 | 2
B200 192GB | 8 | 10 | 2
MI300X 192GB | 8 | 10 | 2
So a single 8x H100 node does not load this model, and the widely repeated "8x H100 minimum" figure is short by more than a factor of two. On B200s you can technically fit the weights across eight cards, but you are leaving almost nothing for a 1M-token KV cache, so plan for two nodes.

Moonshot lists three supported inference engines in the model card: [vLLM](https://github.com/vllm-project/vllm), [SGLang](https://github.com/sgl-project/sglang), and TokenSpeed, each with a published K3 recipe. Day one there are no community GGUF, AWQ or Unsloth conversions, which is unsurprising: requantizing a QAT MXFP4 MoE of this size is not a weekend job, and anything that appears in the next few days deserves scrutiny before you trust it.

For scale, here is how much the previous open Kimi models actually get pulled, straight from the Hugging Face API. These are 30-day counts, and most of them are quantization pipelines, mirrors and eval harnesses rather than production deployments, which is worth remembering whenever a release gets called widely deployed. K3 is too fresh to compare: its counter had not meaningfully moved at the time of writing, and Hugging Face updates these on a lag.

Model | Total params | HF downloads (30 day)
Kimi K2.5 | 1T | 973,001
Kimi K2.6 | 1T | 907,081
Kimi K2.7 Code | 1T | 695,744
Kimi K2 Thinking | 1T | 73,772
Kimi K3 | 2.8T | just published
## Benchmarks, with the usual asterisk

Every number below comes from Moonshot's own model card, run on their harness. We have not reproduced any of it, and K3 is far beyond what we can bench locally. Treat this as a vendor claim until third parties replicate it. The pattern across all 46 reported benchmarks is that K3 and Fable 5 trade wins, with GPT-5.6 Sol close behind.

Credit where it is due: Moonshot says so itself. The tech report's abstract states that K3's overall performance "still trails the most powerful proprietary models, namely Claude Fable 5 and GPT-5.6 Sol", while beating every other model in their suite. A lab publishing that sentence in its own launch paper is worth more than any leaderboard screenshot, and it is a useful corrective to the week's louder claims that K3 simply beats everything.

Benchmark (Moonshot-reported) | Kimi K3 | Claude Fable 5 | GPT-5.6 Sol | GLM-5.2
GPQA Diamond | 93.5 | 92.6 | 94.1 | 91.2
Terminal-Bench 2.1 | 88.3 | 88.0 | 88.8 | 82.7
FrontierSWE | 81.2 | 86.6 | 71.3 | 67.3
SWE-Marathon | 42.0 | 35.0 | 39.0 | 13.0
BrowseComp | 91.2 | 88.0 | 90.4 | not reported
MCPMark-Verified | 94.5 | 87.4 | 92.9 | not reported
GDPval-AA v2 (Elo) | 1686 | 1747 | 1736 | 1510
AA-Briefcase (Elo) | 1548 | 1583 | 1495 | 1260
OSWorld-Verified | 84.8 | 85.0 | 83.0 | not reported
Video-MME (with subtitles) | 90.0 | not reported | 89.5 | not reported
K3 leads on SWE-Marathon by 7 points, on MCPMark-Verified by 7.1, and on BrowseComp by 3.2. It trails on FrontierSWE, GDPval and AA-Briefcase. Long-horizon agentic work is where it looks strongest, which lines up with what it was built for. For the launch-week pricing and arena context, see our [Kimi K3 model breakdown](/models/kimi-k3.html).

## What it actually built

The most concrete evidence in the report is not a benchmark, it is two artifacts you can go read. Both are Apache 2.0 on GitHub, published July 23.

[nano-kpu](https://github.com/MoonshotAI/nano-kpu) is an inference-chip prototype that K3 designed, optimized and verified in a single 48-hour autonomous run driving open-source EDA tools. Inside a 4 mm2 area budget it closes timing at 100 MHz and simulates decode throughput above 8,700 tokens per second, with 1.46M standard cells and an INT4 MAC array. [MiniTriton](https://github.com/MoonshotAI/minitriton) is a compact Triton-like GPU compiler K3 wrote end to end, from a tile-level Python frontend through MLIR passes to PTX code generation, with autograd and distributed primitives on top.

On kernel optimization, Moonshot reports K3 cutting AttnRes latency from 283.6 ms to 114.4 ms and reducing KDA runtime by 73.6 percent, and says an early K3 checkpoint was already doing most of their kernel work during late development. These are still first-party claims, but unlike a benchmark score the output is sitting in a public repo where you can judge it yourself.

## Limitations and gotchas

- Read the LICENSE before you build a business on it. The 20 million dollar Model-as-a-Service threshold is aggregate company revenue, not model revenue, and it is easy to misread as the latter.

- Benchmarks are entirely first-party at this point, produced on Moonshot's own agent harness. Nothing has been independently replicated yet.

- Size is the wall, not a temporary inconvenience. There is no path where this runs on one GPU, and MXFP4 is already applied, so there is no easy 2x saving left on the table.

- No community quants on day one. Early conversions of a QAT MXFP4 MoE this large are likely to be broken. Check the repo discussions before trusting one.

- The vision tower and multimodal projector are unquantized, so multimodal serving carries more memory than the headline 4-bit number suggests.

- Using the hosted API instead means sending your code to a China-hosted endpoint. Self-hosting is the entire point of this release for anyone with a data residency constraint.

## Who should use it

Pull the weights if you already run multi-node inference, you need frontier-class agentic coding you fully control, and license terms or data residency matter more than convenience. Pull them too if you research architectures, because a 69/24 linear-to-full attention split at 2.8T scale is the most interesting thing in this release regardless of the benchmark table.

Skip it if you have one GPU, or one node. That is arithmetic, not a judgment. For open coding models you can actually run this week, [Kimi K2.7 Code](/models/kimi-k2-7-code.html) and [GLM-5.2](/models/glm-5-2.html) remain the sane picks, and our [June 2026 open-weight roundup](/news/open-weight-wave-june-2026.html) covers the smaller end of the field.

## Get started in under 10 minutes

If you have the hardware, start the download and read the model card while it runs. Check the license and the layer split first, both are small files.

```

pip install -U "huggingface_hub[cli]"

# Read the license and architecture before committing 1.5 TB of disk.
hf download moonshotai/Kimi-K3 LICENSE config.json --local-dir ./kimi-k3
python3 -c "import json;c=json.load(open('./kimi-k3/config.json'))['text_config'];\
la=c['linear_attn_config'];\
print('layers:', c['num_hidden_layers']);\
print('KDA:', len(la['kda_layers']), 'full attention:', len(la['full_attn_layers']));\
print('experts:', c['num_experts'], 'active:', c['num_experts_per_token'])"

# Then the weights. Budget the disk and the time.
hf download moonshotai/Kimi-K3 --local-dir ./kimi-k3

```

No cluster? The API is OpenAI-compatible, so you can point an agent you already run at K3 by changing two lines.

```

# Any OpenAI-style SDK works; change base_url and model, keep the rest.
from openai import OpenAI

client = OpenAI(
api_key="sk-...", # key from platform.kimi.ai
base_url="https://api.moonshot.ai/v1",
)

resp = client.chat.completions.create(
model="kimi-k3",
messages=[{"role": "user",
"content": "Refactor this repo's auth module and add tests."}],
)
print(resp.choices[0].message.content)

```

Long single-shot builds are where a 1M-token context earns its keep. Hand it an entire spec, let it run, and compare the result against whatever you pay for now.

## Sources and further reading

- [moonshotai/Kimi-K3 on Hugging Face (weights, model card, config)](https://huggingface.co/moonshotai/Kimi-K3)

- [The Kimi K3 License, full text](https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE)

- [Kimi K3 technical report (PDF): architecture, training recipe, infrastructure, case studies](https://github.com/MoonshotAI/Kimi-K3/blob/main/k3_tech_report.pdf)

- [Moonshot: official Kimi K3 tech blog](https://www.kimi.com/blog/kimi-k3)

- [MoonshotAI/nano-kpu: the inference chip K3 designed autonomously](https://github.com/MoonshotAI/nano-kpu) and [MoonshotAI/minitriton: the GPU compiler it wrote](https://github.com/MoonshotAI/minitriton)

- [MoonshotAI/FlashKDA: Kimi Delta Attention kernels (MIT)](https://github.com/MoonshotAI/FlashKDA)

- [MoonshotAI/Attention-Residuals](https://github.com/MoonshotAI/Attention-Residuals)

- [vLLM](https://github.com/vllm-project/vllm) and [SGLang](https://github.com/sgl-project/sglang), the two supported open inference engines

- [SingularityByte: Kimi K3 model breakdown (launch week)](/models/kimi-k3.html)

Tested on: not independently benchmarked. Kimi K3 is a 2.8T-parameter MoE requiring multi-node inference, which is beyond our local bench, so every benchmark figure here is Moonshot-reported. Repository contents, file count, byte total, license text, architecture parameters, quantization config and download counts were read directly from the Hugging Face and GitHub APIs, not from secondary reporting. Hardware figures are computed from the measured 1,453.8 GiB weight total.

Date checked: 2026-07-27

---


# Free AI Showdown 2026: Claude vs GPT vs Gemini Developer Tiers
Source: https://singularitybyte.com/news/free-ai-showdown-claude-gpt-gemini-developer-tiers-2026.html

Here is the uncomfortable truth about building open-source AI agents in 2026: you almost always start by burning a closed-model free tier. You sketch the agent loop against Claude, you generate the synthetic training set with Gemini, you benchmark your Qwen fine-tune against GPT. Then you move the hot path to Ollama and call it a day. The question this deep dive answers is practical, not philosophical: which free AI tier from Anthropic, OpenAI, and Google actually lets an open-source builder ship without burning prepaid credits?

This is not a closed-model review. SingularityByte's editorial line is that we cover closed models only when they directly impact how the open-source ecosystem ships. Claude, GPT, and Gemini pass that test three ways: they subsidize prototype work that later runs on open weights, they generate the synthetic data that trains the next Qwen or Llama fine-tune, and they provide the evaluation baseline that every open-weights release measures itself against. If any of those pipelines breaks, the open-source flywheel slows down.

## The Framing Question

A working definition for this piece: a "useful" free AI tier for open-source work is one that lets you run a weekend-scale prototype (roughly ten to twenty thousand tokens a day of real agent traffic) without funding a card, without losing the right to ship the code under an open-source license, and without getting your prompts ingested into the provider's next training run. Measured on those three axes, none of the big three is perfect, and the rankings change depending on what you are actually building.

We are not asking "which model is smartest." We are asking "which one lets you ship open-source code in 24 hours for zero dollars without a compliance headache."

## Anthropic Claude Free Tier: Starter Credits, Not a Tier

Claude's free offering is the thinnest of the three, and Anthropic is upfront about it. A new account at platform.claude.com gets a small starter credit (community reports put it around five dollars at current Haiku 4.5 pricing) and no recurring monthly free allocation. Once the credit burns, the API returns HTTP 402 until you prepay. There is no "always free" Claude endpoint.

Where this stops mattering is the Claude for Open Source program, launched in late February 2026 and expanded on 2026-07-08. Qualifying maintainers get six months of Claude Max 20x free, roughly 1,200 dollars of value, across up to 10,000 spots. That is effectively unlimited Opus 5, Sonnet 5, and Haiku 4.5 traffic for anyone maintaining a reasonably active public project.

The July expansion widened eligibility past library authors to core contributors and critical packages. There are now four qualification tracks, and you only need one:

- Maintainers and library authors: 500+ dependent repos, 100+ dependent packages, or 200,000+ combined monthly downloads across npm, PyPI, crates.io, RubyGems or similar.

- Core contributors: a listed committer or maintainer on a recognized foundation or language project (CPython, the Rust team, Node.js TSC, Apache PMC, CNCF, Kubernetes, the Linux kernel, Django, Rails).

- Active contributors: 100+ pull requests merged into repos you do not own in the last 12 months.

- Community builders: one of your repos has had 20+ unique external contributors with merged PRs in the last 12 months.

Applications are reviewed on a rolling basis against that 10,000 cap, so check the official page for current status before you plan around it. None of these flows are self-serve: you fill in a form and wait.

For an open-source builder, the practical read is this. Use the starter credit to prove the agent loop works. If you clear any of the four tracks above, apply the same week and budget zero dollars after that. If you are a solo dev on a fresh GitHub account, your Claude free tier will last two days.

For sizing the starter credit, current API pricing per million tokens is Opus 5 at 5 dollars in and 25 out, Sonnet 5 at 3 and 15 (with an introductory 2 and 10 running through 2026-08-31), and Haiku 4.5 at 1 and 5. Haiku is what makes a small credit stretch.

### What You Can Actually Build In 24 Hours (Claude)

- Full agent loop with Sonnet 5 and tool calling, enough tokens to run roughly 200 to 400 real agent turns.

- A synthetic instruction-tuning dataset of a few thousand examples for a Qwen or Llama fine-tune, if you use Haiku 4.5 as the generator.

- Evaluation harness comparing your open-weights model to Sonnet 5 on a benchmark of a couple of hundred prompts.

## OpenAI GPT Free Tier: The Smallest Gate

OpenAI's free API story is the weakest of the three for open-source developers. There is no standing free tier on the API. New accounts get a small trial credit (historically three to eighteen dollars, now usually nothing unless you qualify for a promotional code) and the rest is prepaid. ChatGPT Free is a product, not an API: it does not give you an OpenAI API key, and you cannot call it from code.

The one thing that has moved the needle for open-source builders in 2026 is GPT-OSS, OpenAI's first open-weights release in years. The 120B and 20B variants are now hosted free on Groq, Cerebras, and OpenRouter. That is genuinely useful: you can evaluate against a GPT-adjacent model without touching OpenAI's API at all. GPT-OSS 120B is MIT licensed and runs on two H100s or the Cerebras free tier. If your angle is "I want to benchmark open-weights against the closest thing to a GPT we are allowed to see," go straight to Cerebras and skip OpenAI entirely.

For closed-source GPT-5 and o3-mini access, the answer for free is: use it through Microsoft's Azure AI Foundry free credits (one hundred dollars for students, one fifty for new Azure accounts), or wait for a marketing promo. The OpenAI free API tier that builders actually want does not exist.

### What You Can Actually Build In 24 Hours (GPT)

- On OpenAI direct: nothing meaningful for free. Assume you need credits.

- On Groq or Cerebras via GPT-OSS 120B: a full agent loop, synthetic data generation, and benchmark harness with a million tokens a day headroom on Cerebras alone.

- Via Azure AI Foundry credits: GPT-5 access for the duration of the promo, with a clock attached.

## Google Gemini Free Tier: The Volume Leader

Gemini is the answer if you want volume without a card. As of 2026-07-16 Google's pricing page lists free-tier rows for Gemini 3.5 Flash and Gemini 3.1 Flash-Lite on the standard tier. The practical quota for Flash lands around 15 requests per minute, 1 million tokens per minute, and 1,500 requests per day. Google Search grounding is included at 5,000 prompts per month.

One caveat that trips people up: Google no longer publishes a universal rate-limit table. Quotas are assigned per project and vary by region, account age, and whether billing is attached, so the numbers above are the commonly reported shape rather than a contractual floor. Your actual quota is visible in the AI Studio console, and that is the only figure worth planning against.

There is a price tag that is not measured in dollars. Google's pricing page is explicit: free-tier content is used to improve their products, paid-tier content is not. For open-source developers this is a two-way trade. If your prompts contain customer data, proprietary code, or anything under NDA, Gemini free tier is disqualified. If your prompts are public test harnesses, synthetic data generation on open datasets, or evaluation runs against open-weights models, the trade is fine, and you get the most generous free tier on the market.

Gemini Pro is not free on the standard tier. If you want Pro, you pay. Flash is the free workhorse, and it is strong enough to prototype against Qwen3 235B, GPT-OSS 120B, or Llama 3.3 70B with results you can learn from.

### What You Can Actually Build In 24 Hours (Gemini)

- A high-throughput synthetic dataset generator running at the RPM cap for several hours, producing tens of thousands of training examples for an open-weights fine-tune.

- A multimodal RAG prototype using Gemini 3.5 Flash for text and image inputs, no credit card.

- An evaluation harness that benchmarks your Qwen or Llama output against Flash on thousands of prompts per day.

## Head-to-Head: Open-Source-Friendly Ranking

Ranking for the specific question "which free AI lets me ship open-source in 24 hours for zero dollars?":

Use case | Winner | Runner-up | Why
Prototype an OSS agent you will later run on Ollama | Gemini 3.5 Flash | Claude Haiku 4.5 (starter credit) | Gemini's free tier actually recurs; Claude burns out by Monday
Generate synthetic training data for a Llama/Qwen fine-tune | Gemini 3.5 Flash | Mistral free tier (1B tokens/month) | Highest volume, multimodal input, no per-request cap headaches
Evaluate open-weights output against a closed baseline | Claude Haiku 4.5 via starter credit | Gemini 3.5 Flash | Claude Haiku 4.5 is still the strongest small-model judge for eval harnesses
Benchmark your fine-tune against a "GPT-class" model | GPT-OSS 120B on Cerebras | Gemini 3.5 Flash | No OpenAI key required, and it is actually open-weights
Production traffic without surprise bills | None | None | All three reserve the right to yank free access. Move to open weights before launch.
## How These Free AI Tiers Subsidize Open-Source Builders

There is a real, measurable economic story hidden in the free-tier data. Every open-weights fine-tune shipped in the last eighteen months (Qwen, DeepSeek, Hermes, OpenHermes, Dolphin, Nous) used synthetic data generated from a closed model at some step of the pipeline. The free AI tiers from Anthropic, OpenAI, and Google are, indirectly, the R&D budget of the open-source fine-tuning community. When Google gives away Gemini 3.5 Flash for free, a dozen hobbyist fine-tunes that would never get built at three dollars per million tokens get built at zero. When Anthropic gives Claude Max away to open-source maintainers, the agentic framework ecosystem (CrewAI, Letta, LangGraph) gets its prototyping subsidy.

This is not charity. It is a distribution play. Every free-tier token is an impression: "our model is the one you reach for first." The builders who stay in the ecosystem eventually pay. The open-source community benefits from the overflow.

What this means operationally: do not feel guilty about using a free tier to prototype open-source code. Do feel responsible about citing the tier you used, and about publishing your evaluation harness so other builders can replicate it on the same free quotas.

## The Rate-Limit Reduction Warning

One prediction for the next ninety days. Free AI tiers always shrink. Groq cut its Llama 70B daily token budget twice in 2025. Google silently reduced Flash free-tier RPM once already in 2026. Anthropic may narrow the open-source program eligibility when the first cohort reviews. OpenAI has trended toward less free access, not more. Build your agent with a fallback chain from day one: if Gemini free throws a 429, fall back to Groq, then Cerebras, then a local Ollama on Qwen3. Tools like LiteLLM, OpenRouter, and custom routers make this a three-line change in your client code.

## Actionable Framework: The Free-Tier OSS Workflow

A checklist for using closed-model free tiers to ship open-source code without regret:

- Pick one closed tier per pipeline stage. Prototype on Gemini, eval on Claude Haiku 4.5, benchmark against GPT-OSS on Cerebras. Do not blend them.

- Log every prompt and response. Use Langfuse or Phoenix. When the free tier shrinks, your logs become a replay harness on open weights.

- Never ship a closed dependency into production. Closed-model calls belong in the notebook, not in the Docker image.

- Publish your evals on open weights. Your closed-model baseline is private infrastructure. Your Qwen or Llama numbers are the public artifact.

- Apply to open-source programs the same week you start a project. Claude for Open Source, GitHub Models, Azure AI Foundry credits. A ten-minute form can save six months of compute spend.

## Risks and Open Questions

Three things to watch in the next ninety days. First, Gemini's data-for-compute trade gets tightened or loosened (it has moved twice in 2025). Second, Anthropic expands or contracts the open-source maintainer program based on cost. Third, OpenAI ships (or does not ship) a recurring free API tier to compete with Gemini. Any one of those three moves reshuffles this ranking.

The deeper question: as open-weights models close the gap with Claude and Gemini on instruction following and coding, does the free AI tier lose its value for open-source builders? Probably not for another 18 months. Gemini 3.5 Flash still catches failure modes that Qwen3 235B Instruct misses, and Claude Opus 5 still writes cleaner reasoning chains than any open model we have tested. Free closed-model tiers will stay useful as long as the frontier sits inside those APIs.

## Sources and Further Reading

- [Claude API pricing and free credits](https://platform.claude.com/docs/en/about-claude/pricing)

- [Claude for Open Source program](https://www.anthropic.com/news/claude-for-open-source)

- [Google Gemini API pricing](https://ai.google.dev/pricing)

- [OpenAI API pricing](https://openai.com/api/pricing/)

- [GPT-OSS open-weights release on Hugging Face](https://huggingface.co/openai)

Open Google AI Studio, grab a free Gemini 3.5 Flash key, set it as GEMINI_API_KEY, run the starter example at ai.google.dev, and you have a multimodal free AI endpoint in the terminal in under ten minutes.

Free tiers and model lineups move fast. Every figure above was verified against the providers' own pricing and program pages on the date below; check the linked sources before planning around any single number.

Date checked: 2026-07-25

---


# Ollama Is an Agent Now: v0.32 Turns the Bare Command Into Chat, Code, and Web Search
Source: https://singularitybyte.com/news/ollama-v0-32-interactive-agent-2026.html

Type ollama with no arguments after updating to v0.32 and you no longer get a help page: you get an interactive agent that chats, writes code, searches the web, and, in Ollama's words, delegates real work. The tool most of us installed to run models locally is now an agent harness. Three releases shipped in ten days, and one default deserves your attention before you press enter.



## What's new in v0.32




- The bare command is an agent (v0.32.0, July 11): running ollama launches "Chat, Code, & Work", a session that can chat with models, edit code, search the web, and run multi-step tasks. In v0.32.1 it picks up your current working directory, so it operates on the project you are standing in.

- Skills and longer tool runs (v0.32.2, July 20): the agent gained a skills system and unlimited tool rounds for cloud models, plus a Claude Code launch integration alongside the existing ChatGPT one. On Linux, CUDA compute capability 10.0 support arrived.

- Housekeeping that matters (v0.32.1, July 16): an MLX engine cache leak that grew memory across requests on Apple Silicon is fixed, and older agent models (CodeLlama, Qwen2.5-coder, Llama 3.x, Mistral, StarCoder, base DeepSeek-R1) now carry deprecation warnings.



## Why it matters for open-source builders


Every terminal agent so far has asked you to install something new. Ollama just gave one to everybody who already has it installed, which is a very large number of machines. But read the model name in the banner: the default agent runs on glm-5.2:cloud, Ollama's hosted deployment of [GLM-5.2](models/glm-5-2.html), and web search requires an ollama signin. That means the out-of-the-box experience of the local-AI tool sends your prompts, and potentially your code, through Ollama's cloud. The launcher has a model picker, so you can point the agent at a local model instead; just know that the default is no longer the thing Ollama's name stands for. We flagged the same drift when [the web search API landed in v0.31](news/ollama-update-gemma-4-speedup-new-scheduler-web-search-2026.html). The pattern is now unmistakable.



## Try it in two minutes



```
# Update, then launch the agent
curl -fsSL https://ollama.com/install.sh | sh # macOS/Windows: update the app
ollama

# Prefer a local model? Pick one in the launcher,
# or keep using the classic commands, unchanged:
ollama run gemma3

```


One-click links: [release notes on GitHub](https://github.com/ollama/ollama/releases) · [download Ollama](https://ollama.com/download) · [docs](https://docs.ollama.com)


Update, type ollama, and ask it to summarize the repo you are sitting in. Then open the model picker and see how far a local model gets on the same task. That comparison, on your own hardware, is the most honest benchmark you will read this week.


All changes are as documented in Ollama's official release notes for v0.32.0 through v0.32.2; we have not yet load-tested the agent mode ourselves.
Date checked: 2026-07-22

---


# Ollama Update: Gemma 4 Nearly 90% Faster on Apple Silicon, a New Scheduler, and a Web Search API
Source: https://singularitybyte.com/news/ollama-update-gemma-4-speedup-new-scheduler-web-search-2026.html

Ollama, the default way most of us run open models locally, shipped a dense two weeks: Gemma 4 now generates tokens nearly 90 percent faster on Apple Silicon, a rebuilt scheduler squeezes real speed out of GPUs you already own, and a new web search API gives local models live internet access. Two of the three make your current hardware faster for free. The third deserves one honest caveat. Here is the rundown.

## What's new, with numbers

- Gemma 4 on Apple Silicon (v0.31.1, June 30): nearly 90 percent faster token generation on average, measured on the Aider polyglot coding-agent benchmark with Gemma 4 12B (nvfp4) on an M5 Max, per the [official post](https://ollama.com/blog/faster-gemma-4-mlx-mtp). Gemma 4 ships a small draft model that proposes several tokens ahead; the main model verifies them in one pass, and Ollama now auto-tunes the draft length at runtime. On by default, and it does not change model output. A new MLX matmul kernel (2x to 2.5x faster on M5 Max) helps other models too.

- New model scheduler: Ollama now measures exactly how much memory a model needs instead of estimating. Official numbers: gemma3:12b at 128K context on an RTX 4090 jumps from 52.02 to 85.54 tokens per second (64 percent faster) because all 49 layers now fit on the GPU, and image-input prompt processing on a dual-4090 rig goes roughly 10x, from 127.84 to 1,380.24 tokens per second ([details and benchmarks](https://ollama.com/blog/new-model-scheduling)). Fewer out-of-memory crashes and honest memory reporting come along for the ride.

- Web search API: a hosted endpoint at ollama.com with a free tier for individuals, callable from curl, Python (ollama.web_search()), or JavaScript, built for feeding live results to local models as an agent tool ([announcement](https://ollama.com/blog/web-search)). Ollama has not published exact free-tier limits, just "generous."

Smaller but welcome, in [v0.31.2 (July 6)](https://github.com/ollama/ollama/releases): flash attention enabled on Pascal-era NVIDIA GPUs (compute capability 6.x), and a fix for model loading on paths with non-UTF-8 characters.

## Why it matters for local builders

The scheduler is the sleeper here. A 64 percent generation speedup at long context on a 4090, from a software update, is the kind of gain we usually pay for in silicon. If you run agent loops against a local [Qwen 3.6](/models/qwen-3-6.html) or point [OpenCode](/tools/opencode.html) at an Ollama endpoint, you get that for free by upgrading. And multi-token prediction landing in the mainstream local runtime matters beyond Gemma: it is the same speculative-decoding idea the big MoE models like [GLM-5.2](/models/glm-5-2.html) ship with, now working out of the box on a Mac.

## One honest caveat

Web search is not local. Your queries go to Ollama's servers, authenticated with an API key, and the feature sits next to a paid cloud tier (Free, $20 Pro, $100 Max). Parts of r/LocalLLaMA and Hacker News read this as cloud creep in a tool people chose precisely because it kept everything on-device; that is community sentiment, not an accusation, but the underlying point stands. Know which of your calls stay local and which do not, and skip the API key entirely if air-gapped is the point of your setup.

## Try it in 10 minutes

```

# Update Ollama (or grab the installer from ollama.com/download)
curl -fsSL https://ollama.com/install.sh | sh

# Feel the MTP speedup on an Apple Silicon Mac
ollama run gemma4:12b

# Optional: web search (key from ollama.com/settings/keys; this call leaves your machine)
curl https://ollama.com/api/web_search \
--header "Authorization: Bearer $OLLAMA_API_KEY" \
-d '{"query": "LongCat-2.0 benchmarks"}'

```

## Sources and further reading

- [Ollama blog: faster Gemma 4 with multi-token prediction](https://ollama.com/blog/faster-gemma-4-mlx-mtp)

- [Ollama blog: new model scheduling](https://ollama.com/blog/new-model-scheduling)

- [Ollama blog: web search API](https://ollama.com/blog/web-search)

- [Release notes on GitHub](https://github.com/ollama/ollama/releases)

- [Ollama cloud pricing tiers](https://ollama.com/pricing)

Tested on: not independently tested; no Apple Silicon or RTX 4090 bench available to us this week. All performance figures are Ollama's own published benchmarks (Aider polyglot on M5 Max; RTX 4090 and dual-4090 scheduler runs), linked above.

Date checked: 2026-07-09

---


# The June 2026 Open-Weight Wave: 16 Models in One Week
Source: https://singularitybyte.com/news/open-weight-wave-june-2026.html

The first week of June 2026 was the densest stretch of open-weight releases anyone has tracked. In roughly seven days, frontier-grade open models landed across every modality: language, vision, image, audio, speech, music, video, 3D, and world models. The roundups called it "25+ models in a week." The honest count of models you can actually name and verify is closer to sixteen, but the point stands. Open weights are no longer a trickle, they are a flood, and the license terms are getting better, not worse. Here is what shipped, what to believe, and what it means if you build on open models.

## What actually happened

Between roughly June 1 and June 8, a cluster of labs shipped at once. [MiniMax M3](/models/minimax-m3.html) and ByteDance's Bernini-R video model opened the week on June 1. [Gemma 4 12B Unified](/models/google-gemma-4.html) and Ideogram 4 landed June 3. NVIDIA's Nemotron 3 Ultra dropped June 4. The rest filled in around them. No single announcement defined the week; the volume did.

Two caveats before the table, because the roundups documenting this week are thin SEO posts and the numbers wander. First, "25+" is marketing math: about sixteen to eighteen models are actually named and checkable. Second, nobody has published aggregate download or star totals for the week, so anyone quoting a wave-level adoption number is inventing it. The individual releases below are real and verifiable; the framing around them is hype until proven otherwise.

## The standout releases

Model | Org | Modality | License | Why it matters
[Nemotron 3 Ultra](/models/nvidia-nemotron-3-ultra.html) | NVIDIA (US) | LLM / agents | OpenMDW-1.1 | 550B hybrid Mamba-MoE, 1M context, weights plus data and recipes
Gemma 4 12B Unified | Google (US) | Multimodal LLM | Apache 2.0 | Text, image, audio, video in one 12B model that runs in ~16GB
MiniMax M3 | MiniMax (CN) | Multimodal LLM | open | 428B / 23B active, 1M context, frontier coding and computer use
[Ideogram 4](/models/ideogram-4.html) | Ideogram (US) | Image | code Apache 2.0, weights non-commercial | First open weights from a closed image lab, best-in-class text rendering
Step-3.7-Flash | StepFun (CN) | Vision-LLM | Apache 2.0 | 198B / 11B active sparse MoE, 256K context
Mellum2-12B | JetBrains (EU) | Coding LLM | Apache 2.0 | JetBrains' first open MoE, tuned for code completion
PaddleOCR-VL-1.6 | Baidu (CN) | Document VLM | Apache 2.0 | ~1B model that beats document parsers 10x its size
dots.tts | RedNote (CN) | Speech / TTS | Apache 2.0 | Fully continuous autoregressive TTS, no codec tokens
Magenta RealTime 2 | Google (US) | Music | Apache 2.0 / CC-BY | Real-time music generation under 200ms, DAW plugins
Holo-3.1-4B | H Company (EU) | Computer-use VLM | Apache 2.0 | 4B agent for web, desktop, and mobile automation
Modality and license for the larger models (Nemotron, Gemma 4, Ideogram, MiniMax) are confirmed from primary sources. The smaller drops (Step-3.7, Mellum2, dots.tts, Holo-3.1) are reported by the week's roundups and should be treated as community-reported until you check the model card. Also shipped that week: Cosmos3-Super (NVIDIA world model), TripoSplat (VAST, image-to-3D, MIT), Higgs Audio v3 (Boson AI TTS).

## The license picture is the good news

Count the licenses in that table and the trend is obvious: permissive wins. Apache 2.0 dominates the LLM, audio, document, and agent releases. NVIDIA went further than most, shipping Nemotron 3 Ultra's weights, training data, and recipes under the permissive OpenMDW-1.1. A year ago the open frontier was full of "open but non-commercial" and "research-only" asterisks. This week, most of the asterisks were gone.

The holdout is image generation. Ideogram 4 made its code Apache 2.0 but kept its weights non-commercial, with a $300-a-month tier to actually ship product. That is the friction point to watch: text and agent models are racing toward genuine open-source, while image and video weights still come with a business-model string attached. If you are picking a model to build on, read the weights license, not the code license, and not the headline.

## China leads intelligence and volume, the US counters on efficiency

Look at who shipped. Chinese labs (MiniMax, StepFun, Baidu, RedNote, ByteDance) accounted for the bulk of the volume, and they lead the open intelligence rankings too. The mid-June coda made it stark: Z.ai's [GLM-5.2](/models/glm-5-2.html), released June 17 under MIT, took the top of Artificial Analysis's open-weights Intelligence Index, above every US open model. NVIDIA's Nemotron 3 Ultra is the strongest US open model, and it sits a notch below the Chinese frontier.

The US labs are competing on a different axis: efficiency and infrastructure. Google's Gemma 4 12B runs four modalities in 16GB of memory. NVIDIA's Mamba-MoE hybrid claims 5 to 6 times the throughput of its Chinese rivals on long agentic runs. The structural backdrop, from Hugging Face's Spring 2026 report, is that Chinese models already make up about 41% of all Hugging Face downloads, the largest single share. The open-weight center of gravity has moved, and this week did not reverse it.

## What this means if you build on open models

The practical takeaway is that "wait for the open version" is now a viable strategy in almost every modality. Need a coding agent, a document parser, a TTS engine, a music generator, or a computer-use agent? There is a fresh, permissively licensed open option as of this month. The release cadence has compressed from a steady drip to a weekly flood, which means the cost of betting on a closed API just went up, because the open alternative is rarely more than a few weeks behind now.

## Try the most accessible one in 10 minutes

Most of this week's headliners need a data-center GPU. The exception worth your ten minutes is Gemma 4 12B Unified, which runs on a single 16GB card or a recent Mac.

```

# The one wave model you can actually run on a laptop-class GPU
ollama run gemma-4:12b

# Then hand it text, an image, and a short audio clip in one prompt
# and see four modalities answered by a model small enough to self-host.

```

That is the real story of the week in a single command: a model that would have been a frontier lab's crown jewel two years ago, now an "ollama run" away, under a license that lets you ship it. Watch the next 90 days for the same thing to happen to video and image weights, the last holdouts still gated behind a paywall.

## Sources and further reading

- [Hugging Face: State of Open Source, Spring 2026](https://huggingface.co/blog/huggingface/state-of-os-hf-spring-2026)

- [Google: Gemma 4 announcement](https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/)

- [MiniMax M3 announcement](https://www.minimax.io/blog/minimax-m3)

- [Artificial Analysis: GLM-5.2 leads the open-weights index](https://artificialanalysis.ai/articles/glm-5-2-is-the-new-leading-open-weights-model-on-the-artificial-analysis-intelligence-index)

- [Release-week roundup (Mervin Praison)](https://mer.vin/2026/06/open-weight-ai-release-week-25-models-across-llms-image-audio-video-and-3d-june-2026/)

Tested on: not independently benchmarked. This is an ecosystem roundup; model counts and the "25+" figure are roundup-reported and flagged as approximate, individual release facts (params, licenses, dates) are drawn from primary model cards and announcements where available and labeled community-reported otherwise. No aggregate download or star totals are cited because none were published.

Date checked: 2026-06-26

---


# When Washington Pulled the Plug: The Anthropic Export Ban and the Case for Open-Weight AI
Source: https://singularitybyte.com/news/anthropic-export-ban-case-for-open-weight-ai-2026.html

On June 12, 2026, at 5:21pm Eastern, the US government sent Anthropic a letter. Ninety minutes of scrambling later, two of the most capable AI models on the planet went dark for everyone. Not throttled. Not geo-fenced. Off. If you had shipped a product on Claude Fable 5 or the model underneath it, Claude Mythos 5, your provider had just been ordered to pull the plug, and it did.

This is the part of the closed-model bargain nobody puts in the architecture diagram. When you build on an API, the off switch lives in someone else's building, and now we know a government can reach in and flip it with no notice and no appeal. The market understood the lesson within hours: capital and traffic rotated toward models you can download and run yourself. Here is the bold version of where this goes. By the end of 2026, "what is our open-weight fallback" stops being a hobbyist question and becomes a standard line in every serious AI architecture review.

## Ninety minutes: what actually happened

The directive did not, on paper, ban the models outright. It ordered Anthropic to suspend access to Fable 5 and Mythos 5 "by any foreign national, whether inside or outside the United States," including Anthropic's own non-citizen employees. The catch is operational. A frontier lab cannot reliably sort hundreds of millions of users by nationality in real time, so the only compliant move was the blunt one. In its own [statement](https://www.anthropic.com/news/fable-mythos-access), Anthropic said it had to "abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance," and it asked AWS to revoke Bedrock access too. Every other Claude model stayed online.

Fable 5 was three days old. It [launched on June 9](https://techcrunch.com/2026/06/09/anthropics-claude-fable-5-is-a-version-of-mythos-the-public-can-access-today/) as the public-facing version of Mythos 5, the more powerful base model whose [cybersecurity abilities](/news/ai-cybersecurity-mythos-glasswing-gpt-cyber-agentic-2026.html) Anthropic had deliberately fenced off. The government's cited concern, relayed to Anthropic, was a method of "jailbreaking" Fable 5 that could reach past those safeguards. The directive came from Commerce Secretary Howard Lutnick, addressed to CEO Dario Amodei.

Anthropic pushed back hard, and the pushback is worth quoting because it doubles as the open-source argument. The company said it "disagrees that the finding of a narrow potential jailbreak should be cause for recalling a commercial model deployed to hundreds of millions of people," called the whole thing a "misunderstanding," and noted that comparable capability already ships in other public models. The rest of the industry took the broader point: if a single reported jailbreak can recall a deployed model overnight, no closed frontier model is safe from the same fate.

As of late June, the models are still off. Anthropic's Managing Director for International, speaking in Seoul on June 18, said the company was "very confident that in the coming days, the models will become available again." That was several days ago. Fable 5 and Mythos 5 remain dark, with no announced return date. "Coming days" is doing a lot of work.

## How we got here: two years of open weights closing the gap

None of this would matter if open models were toys. They are not anymore. The turn started in January 2025, when DeepSeek R1 showed a permissively licensed model trading blows with the closed frontier at a fraction of the cost. Since then the Chinese labs in particular have shipped real weights under real open licenses, on a release cadence the closed shops cannot match.

The numbers moved with the models. Through 2025, Chinese-origin models passed US-origin models in share of Hugging Face downloads, and through 2026 the open-versus-closed capability gap on coding and agentic benchmarks narrowed from tens of points to single digits. We covered the shape of that shift in our [State of Open-Source AI](/news/hf-state-of-open-source-spring-2026.html) report, and the regulatory cross-currents in our breakdown of the [EU AI Act open-source exemption](/news/eu-ai-act-open-source-exemption-circus-2026.html). The short version: open weights stopped being the budget option and started being the default for anyone who wants control.

## The market read the memo

You can argue about sovereignty in the abstract. The stock market settled it in an afternoon. When trading opened after the shutdown, the two newly listed Chinese open-weight labs jumped, and the reason was not patriotism. It was supply risk repricing in real time.

Lab (ticker) | Move after the ban | What they were shipping that week
Zhipu / Z.ai (2513.HK) | Up ~33% (intraday as much as ~47%) | GLM-5.2 open weights, MIT license, no usage restrictions
MiniMax (0100.HK) | Up ~7.4% | M3, an open-weight sparse-attention coder, shipped June 1
Bank of America initiated coverage on both with "buy" ratings the same week, setting price targets of HK$1,250 for Zhipu and HK$500 for MiniMax, per [CNBC](https://www.cnbc.com/2026/06/15/china-ai-zhipu-minimax-artificial-intelligence-race-washington-trump-anthropic.html). The timing was almost theatrical: on roughly the same day Washington was pulling a closed model, Zhipu was putting GLM-5.2 weights on Hugging Face for anyone to download and keep. The New Stack [documented](https://thenewstack.io/fable-ban-open-weights/) four open models that teams swapped in before Anthropic could even restore access. The plug got pulled, and the network routed around it.

That routing shows up in usage, not just share price. On [OpenRouter](https://openrouter.ai/rankings), the most-used models by token volume this month are led by DeepSeek V4 Flash, with Tencent's Hy3 going from zero to near-parity in a single month. Chinese open weights now hold roughly half of the platform's top-tier traffic. The reader takeaway is simple: the alternatives are not theoretical, they are already what a large slice of the market is running.

## Who is ready to catch the runoff

If you need to de-risk a closed dependency this quarter, here is the open-weight bench. The column that matters for sovereignty is not the benchmark score, it is the license. MIT and Apache 2.0 mean you can download the weights, run them on your own metal, and nobody can revoke that later. A vendor "community" license means you need to read the fine print before you bet a product on it.

Model | Lab | License | Size (MoE) | Why it matters
DeepSeek V4 (Pro / Flash) | DeepSeek | MIT | 1.6T total / 49B active; 284B / 13B | 80.6% SWE-bench Verified on Pro-Max, the top open-weight score, with a 1M-token context
GLM-5.2 | Z.ai (Zhipu) | MIT | ~744B total / ~40B active | Leads the Artificial Analysis Intelligence Index v4.1 at 51, ahead of every other open model; built for agentic coding
MiniMax M3 | MiniMax | Community (check terms) | ~230B total / ~10B active | Sparse-attention coder tuned for cheap, long-horizon agent runs
Tencent Hy3 | Tencent | Hunyuan Community (not OSI) | 295B total / 21B active | The fastest climber on OpenRouter token volume; strong agentic workflows
MiMo V2.5 | Xiaomi | MIT | Omnimodal | Text, image, video, and audio in one model, tuned for efficient agent tasks
Qwen3 series | Alibaba | Apache 2.0 | 235B / 22B flagship + dense 27B | The permissive workhorse, 201 languages, an open option at almost every size
Two names on that list, DeepSeek and Z.ai, are MIT, which is about as close to "yours forever" as software gets. If you have written about these before on your own stack, you already have a head start: see our notes on [Z.ai GLM](/models/zai-glm-5-1.html) and [DeepSeek](/models/deepseek-v3-0324.html) for the earlier generations. The jump from API consumer to weight holder is smaller than it looks.

## The sovereignty checklist: de-risking a single-vendor dependency

You do not need to rip out your closed provider. You need an exit that you can pull as fast as the government pulled Anthropic's. Five concrete moves, in order of payoff:

- Inventory the chokepoints. List every feature that breaks if one closed API disappears tomorrow. That list is your actual risk, and it is usually shorter and scarier than people expect.

- Pick one open-weight fallback per chokepoint. Match capability to need: DeepSeek V4 or GLM-5.2 for heavy coding and agents, Qwen3 for general work and many languages, a smaller MiMo or Qwen for on-device.

- Abstract the provider. Put a thin interface between your app and the model so the provider is a config value, not a hard dependency wired through your codebase.

- Keep a model warm. Run the fallback somewhere you control, even at low scale, so failover is a switch and not a weekend project. Our [local AI stack guide](/tutorials/run-full-local-ai-stack-with-local-ai-packaged.html) and [self-hosted tools roundup](/tutorials/top-free-self-hosted-ai-tools-for-builders-2026.html) cover the plumbing.

- Audit licenses and data flow. Confirm the fallback's license actually permits your use, and confirm where prompts and data go. MIT and Apache pass cleanly. Community licenses need a read.

Here is the ten-minute version. Most SDKs already speak the OpenAI-compatible protocol, so the entire failover can be one environment variable.

```

# 1) Keep an open-weight model warm locally (MIT-licensed, yours to run)
ollama pull qwen3 # or any open model from ollama.com/library

# 2) Abstract the provider: same OpenAI-compatible SDK, different base_url.
# Swap a closed endpoint for one you control with one variable.
export OPENAI_BASE_URL="http://localhost:11434/v1" # local Ollama
# export OPENAI_BASE_URL="https://openrouter.ai/api/v1" # hosted fallback
export OPENAI_API_KEY="ollama" # any string for local; a real key for OpenRouter

# Your application code does not change. The provider does.

```

Do that once and you have moved the off switch back into your building. That is the whole point.

## The catch: this is not a free lunch

Open weights solve the kill-switch problem. They do not solve every problem, and pretending otherwise would be its own kind of hype. Three honest caveats.

First, self-hosting a frontier model is not cheap. GLM-5.2 wants roughly eight H100 GPUs to run well locally, which is a real bill, not a Raspberry Pi. For most teams the practical answer is a hosted open-weight endpoint plus a small local model kept warm, not a private datacenter. Second, "open weight" is not the same as "no questions asked." Some of the strongest models ship under vendor community licenses rather than OSI-approved ones, and several of the leaders are Chinese-origin, which carries its own data-governance and procurement scrutiny depending on your sector. Run the audit. Third, export policy is a moving target. The same machinery that reached a closed API could, in principle, reach open-weight distribution next. The difference is that weights you have already downloaded cannot be recalled.

## What to watch in the next 90 days

- Restoration and precedent. Whether Fable 5 and Mythos 5 come back, and on what terms, sets the template for the next time a model is deemed a national-security concern.

- More directives. One letter is an incident. A second one is a policy. Watch Commerce for follow-on action against other labs or capabilities.

- Architecture shifts. Expect "open-weight fallback" to show up in enterprise reference architectures and vendor risk reviews as a default requirement.

- The next open drops. DeepSeek, Z.ai, Alibaba, Tencent, and Xiaomi are shipping monthly. The bench gets deeper while the closed off switch stays exposed.

- Scope creep. Whether the "foreign national" standard stays narrow or widens to other models and other vendors.

The takeaway is not "closed models are bad." It is that access you do not control is a dependency you can lose without warning, and hundreds of millions of users just watched it happen in ninety minutes. Spend ten of your own minutes this week standing up a fallback you own. That is the cheapest insurance in AI right now.

## Sources and further reading

- [Anthropic: Statement on the US government directive to suspend access to Fable 5 and Mythos 5](https://www.anthropic.com/news/fable-mythos-access)

- [CNBC: Anthropic disables access to Fable 5 and Mythos 5 to comply with government directive](https://www.cnbc.com/2026/06/12/anthropic-disables-access-to-fable-5-and-mythos-5-to-comply-with-government-directive.html)

- [Al Jazeera: US orders Anthropic to disable AI models for all foreign nationals](https://www.aljazeera.com/news/2026/6/13/us-orders-anthropic-to-disable-ai-models-for-all-foreign-nationals)

- [TIME: Anthropic pulls its most powerful AI models after US bars foreign access](https://time.com/article/2026/06/13/anthropic-fable-mythos-ban-US-security/)

- [CSIS: The Department of Commerce restricted access to Anthropic's latest models. What comes next?](https://www.csis.org/analysis/department-commerce-restricted-access-anthropics-latest-models-what-comes-next)

- [CNBC: Zhipu surges as Wall Street raises bets on China AI after Anthropic curbs](https://www.cnbc.com/2026/06/15/china-ai-zhipu-minimax-artificial-intelligence-race-washington-trump-anthropic.html)

- [The New Stack: Fable 5 ban, four open models responded before Anthropic could restore access](https://thenewstack.io/fable-ban-open-weights/)

- [Hugging Face: GLM-5.2 open weights (zai-org/GLM-5.2)](https://huggingface.co/zai-org/GLM-5.2)

- [Hugging Face: DeepSeek V4 Pro open weights](https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro)

- [OpenRouter: LLM usage rankings](https://openrouter.ai/rankings)

---