Qwen3.8-27B GSQ-RCO

This is part three of our local inference performance series, and it is the rebuttal to part one. In part one Prism ML argued that if you want a 27B model to survive extreme compression you have to train it that way from scratch, and you have to run it on their fork of llama.cpp. IST Austria's DASLab has taken the same base model and reached task-lossless at 11.8 GB using nothing but post-training quantization, in standard GGUF files that run unmodified in llama.cpp, Ollama and LM Studio. Same problem, opposite answer, and this one needs no fork.

Two acronyms, two separate ideas

GSQ is Gumbel-Softmax Quantization. It is post-training scalar quantization that jointly learns the per-coordinate grid assignments and the per-group scales through a Gumbel-Softmax relaxation. In plain terms, instead of picking a fixed grid and rounding weights onto it, the grid itself is learned. DASLab's claim is that this "closes most of the gap between scalar and vector quantization at 2 to 3 bits while remaining deployable in standard scalar formats", which is the important half: vector quantization has been more accurate at low bits for years, but it does not fit the formats your runtime already reads.

RCO is Riemannian Constrained Optimization, and it solves a different problem: which tensors deserve the bits. It assigns one of K quantization types to each of N tensors under a total size budget, reformulating that budget constraint as a smooth Riemannian manifold in logit space so the allocation can be optimized by gradient descent directly on the task loss while hitting the budget exactly.

Put together, GSQ quantizes each tensor well and RCO decides how much precision each tensor gets. That is the departure from a normal Q4_K_M, which applies broadly uniform bit-widths across the model. Here the allocation is non-uniform and derived from measured sensitivity, and each release ships the per-tensor allocation logs and importance matrices so you can check the work.

Both techniques have papers behind them: GSQ at arXiv:2604.18556 from April 2026, RCO at arXiv:2605.00649 from May. That is worth noting, because most quantization releases arrive as a folder of files and a claim.

Task-lossless at 4.6x smaller

All figures below are author-reported by DASLab. The BF16 base is Qwen3.8-27B at 53.8 GB.

IQ3_S is the one to look at. It matches the BF16 base exactly on AIME25 and LiveCodeBench and lands within 0.51 points on GPQA-Diamond, at 11.8 GB against 53.8 GB. DASLab calls that task-lossless and on these three benchmarks the description holds. Perplexity tells the same story more gradually: Wiki2 goes 7.05 at BF16 to 7.07 at IQ3_S, which is close to measurement noise.

Watch what happens at the bottom of the table instead. Between IQ2_S and IQ2_XS, less than a third of a bit apart, LiveCodeBench drops almost six points and AIME25 finally breaks. That is the cliff every low-bit quantization has, and the useful thing here is knowing where it sits: just under 2.75 bits per weight on this model.

It beats the quants you are probably already running

The comparison that matters for most readers is not against BF16, it is against the dynamic quants people actually download. DASLab puts GSQ-RCO against Unsloth Dynamic at matched sizes.

At 2-bit the gap is large enough to change whether the model is usable at all: ten to thirteen points across three different benchmarks. At 3.5-bit it mostly closes, and Unsloth actually takes GPQA-Diamond by half a point. The honest summary is that careful allocation matters most exactly where the bits are scarcest, and if you are running 3.5-bit or above you are already in good shape with what you have. Our Unsloth writeup covers the other side of that comparison.

The part that makes it easy

These are standard GGUF files. DASLab is explicit: "The resulting files are standard GGUF and run unmodified in llama.cpp, Ollama, and LM Studio." No fork, no patched binary, no silent gibberish if you get it wrong.

llama-cli -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS.gguf \
  -p "Explain mixed-precision quantization." -ngl 99

The base model is multimodal, and the vision path needs the separate projector file, 0.9 GB at BF16:

llama-mtmd-cli -m Qwen3.8-27B-GSQ-RCO-IQ3_XXS.gguf \
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf \
  --image photo.jpg -p "Describe this image."

Optional -mtp builds add a Multi-Token Prediction head for speculative decoding at about 0.35 GB extra, which connects directly to part two: the drafting engines covered there need a draft source, and an MTP head built into the quantized file is the tidiest one available.

Which of the two approaches should you run

Take GSQ-RCO if you want to keep your existing toolchain. It runs in stock llama.cpp, Ollama and LM Studio today, it ships the allocation logs so the claims are checkable, it has papers behind both halves of the method, and IQ3_S is genuinely hard to distinguish from the full model on reasoning and code.

Take Ternary Bonsai 2 if the binding constraint is disk and memory rather than convenience. 5.95 GB against 11.8 GB is not a small difference on a 16 GB machine, and Bonsai buys that at a cost in knowledge and vision tasks plus a mandatory custom build.

The wider lesson from putting them side by side: the field has stopped treating low-bit quantization as a single quality dial. Where the bits go now matters as much as how many there are, and the two most interesting releases of the month landed on opposite sides of whether you should decide that during training or after it.

Sources and further reading

• GSQ-RCO GGUF builds and the full benchmark table
• GSQ: Gumbel-Softmax Quantization, arXiv 2604.18556
• RCO: Riemannian Constrained Optimization, arXiv 2605.00649
• IST Austria DASLab on Hugging Face
• The Qwen3.8-27B base model

Ten minutes: pull the IQ3_XXS file at 10.1 GB, point your existing llama.cpp or Ollama at it with no other changes, and run whichever reasoning prompt you last used to judge a model. This is the rare case where trying it costs you nothing but the download.

Tested on: not independently tested. Bit-widths, file sizes, perplexity, benchmark scores and the Unsloth Dynamic comparison are all author-reported by IST Austria DASLab on their model card. The method descriptions are summarised from the two linked arXiv papers. Download counts were read from the Hugging Face API on the date below.
Date checked: 2026-09-27