Inkling

On July 15, 2026, Thinking Machines Lab shipped Inkling: 975 billion parameters, Apache 2.0, weights on Hugging Face on day one. That makes it the largest open-weights model ever released by a US lab, and the first foundation model from the company Mira Murati founded after leaving OpenAI. It reasons natively over text, images, and audio, carries a 1-million-token context window, and exposes a thinking effort dial you can turn from 0.2 to 0.99. And in a refreshing break from launch-day tradition, the lab's own announcement admits it is not the strongest model available. Here is what Inkling actually is, what it scores, and how to run or fine-tune it today.

TL;DR

• What it is: a 975B-parameter Mixture-of-Experts model with 41B active per token, multimodal input (text, image, audio), 1M-token context, trained on 45T tokens, released under a clean Apache 2.0 license with weights on Hugging Face at launch.
• Why it matters: it is the US answer to the Chinese open-weight wave, pitched honestly as the best open base for customization rather than a leaderboard king, with a token-saving effort dial and fine-tuning via the Tinker API.
• The catch: you need roughly 600 GB of VRAM even at 4-bit, llama.cpp support is still sitting in an unmerged PR, and on raw benchmarks it trails the closed frontier by a wide margin.

What Thinking Machines actually shipped

Thinking Machines Lab launched in early 2025 with a large chunk of OpenAI's former leadership and a product, Tinker, that sells managed fine-tuning rather than chat subscriptions. Inkling is the model that strategy was waiting for: a big, permissively licensed base that customers can specialize on Tinker and then run anywhere. Two models were announced. Inkling itself, 975B total and 41B active, is downloadable now as a BF16 checkpoint plus an NVFP4 build for Blackwell systems. Inkling-Small, a 276B model with 12B active, was previewed for a later release and will be the one most of us can realistically host.

The positioning is the unusual part. The announcement states plainly that "Inkling is not the strongest overall model available today, open or closed" and sells it instead as a broad, honest base for customization. After a year of launch posts claiming frontier supremacy from every direction, a vendor leading with its own limitations is worth noticing.

A DeepSeek-style recipe with real twists

A Mixture-of-Experts (MoE) model splits its feed-forward layers into many expert subnetworks and routes each token through only a few, so you hold 975B parameters on disk but pay inference for 41B. Inkling follows the recipe popularized by DeepSeek-V4 and GLM-5.2: 66 layers, 256 routed experts plus 2 shared, 6 routed experts active per token, sigmoid routing with auxiliary-loss-free load balancing.

Then it deviates. There is no RoPE: Inkling uses a learned relative-position bias, which the lab claims extrapolates better to long sequences. Attention interleaves 512-token sliding-window layers with global layers at a 5:1 ratio. Short convolutions sit after the key and value projections, an inductive-bias trick you rarely see at this scale. And the multimodal path is encoder-free: images enter as raw 40x40 pixel patches through a four-layer MLP and audio as spectrogram tokens, no bolted-on vision tower. Sebastian Raschka's architecture notes are the best independent walkthrough of these choices, including a caution: with conventional grouped-query attention instead of DeepSeek-style MLA, raw decoding speed is probably not Inkling's advantage.

Benchmarks, with the usual asterisk

All numbers below are first-party, reported at effort 0.99, and not reproduced on our bench. The comparison columns come from Thinking Machines' own model card. What stands out is not any single score but the shape: solid everywhere, dominant nowhere, and honest about the gap to the closed frontier.

All figures are self-reported by Thinking Machines at launch. Treat them as vendor claims until independently reproduced.

Within the open-weight field the picture is friendlier. 77.6% on SWE-Bench Verified clears Nvidia's Nemotron 3 Ultra at 71.9%, and the 91.4% VoiceBench score is unusual for a model whose audio path has no dedicated encoder. The clear weak spot is factual recall: 43.9% on SimpleQA means you want retrieval in front of it for anything knowledge-heavy. Against Kimi K3, announced the very next day at 2.8T parameters, Inkling loses the size contest but wins on availability: its weights are downloadable today, while Moonshot's are promised for July 27.

The effort dial is the actual feature

Most reasoning models give you a binary: thinking on or off. Inkling exposes thinking effort as a continuous parameter from 0.2 to 0.99, and the lab's own charts show it matching Nemotron 3 Ultra on Terminal Bench 2.1 while spending roughly a third of the tokens at lower effort settings. For agent pipelines, that is a budget knob you control per request instead of a model-picker decision. Turn it down for routing and classification calls, up for the hard synthesis step, all against one deployed checkpoint. If the numbers hold up in independent testing, this is the feature other open labs copy next.

Run it in about 10 minutes

Be honest with yourself about hardware first. The BF16 checkpoint wants about 2 TB of aggregated VRAM (think 16x H200), and even the NVFP4 build needs 600 GB or more across at least four Blackwell-class GPUs. This is not a laptop model. It is barely a single-server model. If you have the metal, vLLM works out of the box:

# Download the weights (BF16 ~2TB; NVFP4 for Blackwell systems)
huggingface-cli download thinkingmachines/Inkling

# Serve with vLLM on a multi-GPU node
vllm serve thinkingmachines/Inkling \
  --tensor-parallel-size 8 \
  --max-model-len 262144

# Community GGUF quants exist (unsloth/inkling-GGUF), but llama.cpp
# support is still in PR #25731. Until it merges, build from that PR
# and always pass --jinja so the chat template is applied.

Everyone else calls it hosted. Together, Fireworks, Modal, Databricks, and Baseten all serve Inkling, and any OpenAI-compatible client works:

# Inkling via Together's OpenAI-compatible endpoint.
from openai import OpenAI

client = OpenAI(
    api_key="...",                        # your Together key
    base_url="https://api.together.xyz/v1",
)

resp = client.chat.completions.create(
    model="thinkingmachines/Inkling",
    messages=[{"role": "user",
               "content": "Summarize this repo's auth flow and list its weak points."}],
)
print(resp.choices[0].message.content)

Fine-tuning is the intended path, though: Inkling is live on Thinking Machines' Tinker platform with 64K and 256K context options at a 50% launch discount, plus a free playground for a limited time. Simon Willison's first look runs the multimodal loop through the Tinker API: he had Inkling draw an SVG pelican on a bicycle, then describe its own rendering. It analyzed the image coherently, then called its pelican a stork. Frontier is a spectrum.

Limitations and gotchas

• Hardware floor: 600 GB VRAM minimum at 4-bit, about 2 TB at BF16. Self-hosting means a GPU server, not a workstation. Wait for Inkling-Small if you want local.
• llama.cpp support is not mainline yet. The GGUF quants from Unsloth and others require building llama.cpp from PR #25731 and passing --jinja for the chat template.
• Benchmarks are launch-day and first-party. The honest framing earns goodwill, but the numbers still deserve independent reproduction.
• SimpleQA at 43.9% is weak factual recall for a model this size. Pair it with retrieval for knowledge work.
• Output is text only. It reads images and audio but will not generate them.
• Training data disclosure is thin. The card acknowledges "content that may be subject to intellectual property protection" without detail, a point Willison flags too.

Who should use it

Use Inkling if you fine-tune. That is what it is for: a permissive Apache 2.0 base with multimodal input, a 1M context, and a managed tuning platform attached, from a US lab if your compliance people care about that. It also makes sense as a hosted workhorse for agent pipelines where the effort dial can cut your reasoning-token bill against a single deployment.

Hold off if you want the strongest general model (the lab itself points you elsewhere), if you need something you can run on a workstation today (watch for Inkling-Small, or grab GLM-5.2), or if your workload is pure coding, where Kimi K2.7 Code gives you more per active parameter. For everyone else, the real story is strategic: the US open-weights scene finally has a frontier-scale entrant, after a year in which Qwen, DeepSeek, and Moonshot made the open frontier look like a one-country race.

Sources and further reading

• Thinking Machines: Introducing Inkling (official announcement)
• Inkling model card (hardware requirements, full benchmark table)
• Hugging Face: thinkingmachines/Inkling (BF16 + NVFP4 weights)
• Hugging Face blog: Welcome Inkling
• Simon Willison: first look at Inkling
• Sebastian Raschka: Inkling architecture and benchmark notes
• Unsloth: Inkling GGUF quantizations

Tested on: not independently benchmarked. Inkling needs 600+ GB of VRAM even quantized, which is beyond our local bench. All benchmark figures are Thinking Machines' launch-day numbers; hardware and access details are drawn from the model card and the sources above.
Date checked: 2026-07-22