Ternary Bonsai 2 27B

A 27B model that fits in 5.95 GB and still does math at 96.57 against the full-precision 97.06 sounds like a rounding error in somebody's benchmark script. Ternary Bonsai 2 has been downloaded 3.25 million times in eleven days, so a lot of people have already checked. The catch is not the quality. It is that stock llama.cpp will not run it, and one of the file formats it will load produces confident gibberish instead of an error.

Ternary means three values, and it was trained that way

Most small quantizations are something done to a model after training. You take FP16 weights and squeeze them down, and below about 4 bits the reasoning falls apart. Ternary Bonsai 2 is not that. Prism ML trained it with ternary weights from the start, meaning every weight is one of three values, minus one, zero or plus one, with FP16 group-wise scaling applied during training rather than bolted on afterwards.

The packing uses a blockwise Hadamard rotation and lands at a claimed 1.72 bits per weight, about a 9.3x reduction against FP16. The base architecture is Qwen3.8-27B, the same hybrid-attention 27B we covered when Qwen shipped its open line in August. Two GGUF packings ship: PTQ1_0 at 5.95 GB with dense trits, and PQ2_0 at 7.21 GB using 2-bit slots. An optional vision tower adds about 0.63 GB.

This is also not Prism ML's first attempt. The Bonsai family goes back to March 2026 with binary 1-bit models at 1.7B, 4B and 8B, picked up ternary variants in April, added image models in May, and reached 27B in July. Bonsai 2 is the second generation at that size. Cumulative downloads across the family run into the millions, which is worth knowing before you write them off as a lab nobody has heard of.

What the compression actually costs

Prism ML reports 98.2% intelligence retention across fourteen thinking-mode benchmarks: 84.78 average against the FP16 baseline's 86.32. All of these are author-reported and nobody has replicated them independently yet.

Read the third row, because it is the honest one. Math and coding survive the squeeze almost intact, and coding even edges ahead, which is within noise. Knowledge and vision lose five points, and that is the shape you should expect: compression takes memorised facts first and leaves procedure mostly alone. If you want a local model to reason and write code, this trade is excellent. If you want it to recall obscure facts without a retrieval layer, it is not.

The comparison that makes the case is against conventional low-bit quantization rather than against FP16. Prism ML puts IQ2_XXS at 72.59 on the same suite, against their 84.78. That is the gap between a 2-bit model that has lost its reasoning and one that has not, and it is the entire argument for training ternary instead of quantizing to it.

The gotcha that will waste your afternoon

Ternary Bonsai 2 does not run on stock llama.cpp. It needs Prism ML's fork, on the prism branch. The documentation is blunt about what happens otherwise: the PTQ1_0 and PQ2_0 files "are refused, and a Q2_0 file loads silently and outputs gibberish".

Sit with that second half. One of the two failure modes is not a failure at all from the software's point of view. The model loads, the server starts, tokens come out, and they are garbage. If you pull a Bonsai file into an unpatched llama.cpp or an LM Studio that wraps one, you will get output and you will have no error to search for. Check which binary you are running before you conclude the model is bad.

The fork is MIT and tracks current mainline, sitting at 802 stars with 82 open issues. Prism ML also publishes prebuilt binaries from release prism-b10658 onward covering macOS on Apple Silicon and Intel, Linux on CPU, CUDA 12.4 to 12.8, Vulkan and ROCm, Windows on CPU, CUDA, Vulkan and HIP, and iOS. Use those unless you have a reason not to.

Run it today

The quickest route is the demo repository, which picks the right model files and the right prebuilt binary for your hardware automatically. It is Apache 2.0 and sits at 3,149 stars.

If you would rather build the fork yourself:

git clone -b prism https://github.com/PrismML-Eng/llama.cpp
cd llama.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j

Add your backend flag to the configure step, so -DGGML_CUDA=ON for CUDA, and note that building CUDA on a machine with under 16 GB of VRAM needs the parallelism dropped to -j 2 or the compile will run you out of memory.

Prefer PQ2_0 on Metal, CUDA, HIP and CPU. PTQ1_0 is the smaller file, but its arithmetic unpacking cost varies by hardware generation, so the denser packing is not automatically the faster one on your machine. Throughput that Prism ML reports runs from about 28 tok/s on an Apple M5 Pro to about 130 tok/s on an RTX 5090.

There is already a small ecosystem around it: an LM Studio integration that wires the fork in with a reasoning-effort selector, a CUDA Docker image with CI for Hugging Face Inference Endpoints, and an independent attempt to rebase the ternary formats onto upstream mainline. That last one matters, because the day these formats land in stock llama.cpp is the day the gibberish problem stops existing.

Who should run this

If you have a 16 GB machine and have been choosing between a 27B you cannot fit and a 8B that disappoints you, this is the interesting release of the month. A 5.95 GB file that codes at parity with its FP16 parent changes what a laptop can host.

If you are on a 24 GB or larger card, the calculus is different. You could run the FP16 parent, or a 4-bit quant of something bigger, and get the five points back on knowledge tasks. Ternary buys you headroom rather than capability, and headroom is only worth it if you are actually short of it.

Either way, the speed question is separate from the size question, and it is the one we take up in part two of this series, where three different projects are racing to make Apple Silicon decode faster without changing a single output token.

Sources and further reading

• Ternary Bonsai 2 27B model card and GGUF weights
• The full Bonsai family on Hugging Face
• The Prism llama.cpp fork, prism branch
• Bonsai demo, the one-step path to a running model
• Prism ML build and format documentation
• The MLX 2-bit build for Apple Silicon
• Community rebase of the ternary formats onto upstream mainline

Ten minutes: clone the demo repository, let it pull the PQ2_0 file and the right prebuilt binary, and ask it a coding question you already know the answer to. You will know inside one reply whether 5.95 GB is enough for what you need it to do.

Tested on: not independently tested. Architecture, bit-width, file sizes, benchmark scores and throughput figures are all author-reported by Prism ML. The stock-llama.cpp failure behaviour is quoted from Prism ML's own documentation. Repository statistics and download counts were read from the GitHub and Hugging Face APIs on the date below.
Date checked: 2026-09-27