Cosmos 3

Most "open" AI releases this year were chatbots. NVIDIA's Cosmos 3, announced at GTC Taipei on June 1, 2026, is something different: an open world model that understands and generates not just text and video but ambient audio and robot actions, built for physical AI. The point is not to chat. It is to generate training data for robots, predict what happens next in the physical world, and output the action a robot should take. NVIDIA shipped the weights, the training data, and the recipes under a permissive license. Here is what it is, what is actually open, and what you can run.

What a world foundation model is

A language model predicts the next token. A world foundation model predicts the next state of the world: given a scene and an intent, what the camera sees next, what it sounds like, and what action moves a robot toward its goal. Cosmos 3 unifies three things earlier systems kept separate: physical reasoning (understanding a scene), world generation (producing realistic video and audio of what happens next), and action generation (the trajectory a robot should follow). That combination is why it is aimed at robotics, autonomous vehicles, and simulation rather than at your chat window.

How it is built

Cosmos 3 uses a two-tower "Mixture-of-Transformers" design. One tower is an autoregressive reasoner, a vision-language model initialized from Qwen3-VL (8B in Nano, 32B in Super). The other is a diffusion generator that produces the video, audio, and action output. They share one architecture and pass information from reasoner to generator, aligned across video, audio, and action tokens with 3D rotary position embeddings. So it is a hybrid: autoregressive where it reasons, diffusion where it generates. It was trained on roughly 20 trillion multimodal tokens, including about a billion images, 400 million videos, plus audio and human and robot action data.

It comes in three sizes. Cosmos 3 Super (64B) is the frontier-accuracy model, available now. Cosmos 3 Nano (16B) targets a single workstation GPU for near-real-time use, available now. Cosmos 3 Edge (4B), the on-device low-latency variant, is announced but not yet downloadable, so treat it as a roadmap item.

What is actually open

This is genuine open-weights, and more. The shipped models (Nano and Super) are downloadable on Hugging Face under OpenMDW-1.1, a permissive Linux Foundation license that explicitly allows commercial and non-commercial use and also covers the architecture, code, datasets, benchmarks, and training recipes. NVIDIA released six synthetic datasets and the training scripts alongside the weights, plus a separate DROID policy model for robot manipulation. Like NVIDIA's Nemotron 3 Ultra, the story is full openness, not just a weights drop.

Benchmarks and the honest caveats

NVIDIA claims Cosmos 3 ranks first among open models on a stack of physical-AI benchmarks (Physics-IQ, PAI-Bench, RoboLab, RoboArena, and a traffic-anomaly reasoning leaderboard), and tops Artificial Analysis open leaderboards for text-to-image and image-to-video. These are NVIDIA-reported; there is no independent reproduction yet, and NVIDIA did not publish head-to-head tables against its own Cosmos 2 or against Google's Genie line, so cross-family comparisons are not available.

Limitations and gotchas

• It is NVIDIA-only and Linux-only: Ampere, Hopper, or Blackwell GPUs, CUDA 13, BF16. There is no Apple Silicon or AMD path.
• Super needs a data-center GPU; Nano targets a workstation-class card (NVIDIA cites the RTX PRO 6000) for real-time robotics. This is not a laptop model.
• The on-device Edge (4B) model, the one most edge builders want, is not released yet.
• Benchmarks are NVIDIA-reported and physical-AI specific; if you are not doing robotics, AV, or simulation, this is probably not your model.

Who should use it

Roboticists training manipulation policies, autonomous-vehicle teams generating rare scenarios, and anyone who needs physics-accurate synthetic data instead of expensive real-world capture. The open weights plus the open datasets and recipes make it a genuine base to build on, not just an endpoint to call. If you build chat or coding products, skip it; this is a different branch of the tree.

Run it in about 10 minutes

The fastest taste is the hosted playground on build.nvidia.com or a NIM container; the local path is Hugging Face plus Diffusers.

# Hosted: a NIM container for the Nano reasoner (needs an NGC API key)
docker run --gpus all -e NGC_API_KEY=$NGC_API_KEY \
  -p 8000:8000 nvcr.io/nim/nvidia/cosmos3-reasoner:latest

# Or pull the open weights from Hugging Face
huggingface-cli download nvidia/Cosmos3-Nano

With the weights local, the omni pipeline rolls the world forward from a single image and a prompt:

# Generate a short world rollout from an image
from diffusers import Cosmos3OmniPipeline

pipe = Cosmos3OmniPipeline.from_pretrained("nvidia/Cosmos3-Nano").to("cuda")
out = pipe(image="kitchen.png", prompt="the robot arm picks up the red mug")
out.save("rollout.mp4")

If you do not have an NVIDIA workstation, the ten-minute move is the build.nvidia.com demo and the technical report; the two-tower reasoner-plus-generator design is the part worth studying even if you cannot run the weights yet.

Sources and further reading

• NVIDIA: Cosmos 3 announcement
• Cosmos3-Nano model card (Hugging Face)
• NVIDIA developer blog
• NVIDIA/Cosmos on GitHub

Tested on: not independently tested. Cosmos 3 needs an NVIDIA Ampere, Hopper, or Blackwell GPU on Linux (Super is data-center class), beyond our bench. Benchmarks are NVIDIA-reported with no independent reproduction; the Edge (4B) variant is announced but not yet downloadable.
Date checked: 2026-06-26