LingBot-Map

Point a camera at the world, and LingBot-Map builds the 3D scene while you are still filming. Released in late April 2026 by Robbyant, the robotics arm of Ant Group, this Apache 2.0 model turns an ordinary RGB video stream into camera poses and a dense point cloud in real time: about 20 frames per second, sequences beyond 10,000 frames, and roughly 13.3 GB of VRAM. No LiDAR, no depth sensor, no hours of COLMAP crunching. If you have ever wanted photogrammetry that keeps up with a moving robot, a drone, or just your phone on a walk, this is the most usable open release yet.

Streaming 3D reconstruction from video, explained

Classic 3D reconstruction is an offline job. You record everything first, then a pipeline like COLMAP or a batch model like VGGT chews on the full set of frames at once. That works for scanning a statue, but it is useless for a robot that needs to know where the wall is right now.

Streaming reconstruction flips the constraint: the model sees frames one at a time, in order, and must output geometry and camera pose immediately without ever revisiting old frames. The classic answer is SLAM (simultaneous localization and mapping), which tracks handcrafted feature points and runs constant optimization loops. LingBot-Map replaces that whole machinery with a single feed-forward transformer: frames go in, poses and points come out, no iterative optimization anywhere.

The catch that killed earlier attempts is drift. Streaming models like Spann3R or CUT3R accumulate small pose errors that compound over minutes of video, until the reconstruction folds in on itself. LingBot-Map's paper is essentially one long answer to the drift problem.

How the Geometric Context Transformer works

The architecture is called a Geometric Context Transformer (GCT), and it is a pure autoregressive model, the same generation-one-step-at-a-time regime your LLM uses, applied to geometry. Each incoming frame attends to three kinds of context instead of one big history buffer.

First, an anchor context pins the global coordinate frame, so the scene does not slowly rotate out from under itself. Second, a pose-reference window holds the most recent frames for dense geometric cues, the local detail. Third, a trajectory memory keeps a compact record of where the camera has been, which is what corrects long-range drift when you loop back to a place you have already seen.

Because the state stays compact and attention runs over a paged KV cache (the same key-value caching trick that makes LLM serving fast), per-frame cost stays flat instead of growing with video length. The paper reports the bounded context cuts per-frame compute by roughly 80x versus keeping full history, which is how a 25,000-frame indoor walkthrough demo stays stable. Training covered 29 datasets across synthetic and real scenes.

Benchmarks: paper-reported, but the gap is not subtle

All numbers below come from the LingBot-Map paper, so treat them as vendor-reported until the community reproduces them. The team did release the full evaluation pipeline for KITTI, Oxford Spires, VBR, TUM-D, 7-Scenes, ETH3D, Tanks and Temples, and NRGBD, which makes checking their homework unusually easy.

On Oxford Spires with 320-frame sequences, LingBot-Map reports an absolute trajectory error (ATE, how far the estimated camera path deviates from ground truth) of 6.42 m at 20.29 FPS. The streaming competition is not close: CUT3R lands at 18.16 m and Wint3R at 21.10 m. It also beats the offline batch models VGGT (24.78 m) and DA3 (12.87 m), which get to see every frame at once.

All figures paper-reported (arXiv:2604.14141), not independently reproduced. "n/p" means not published in the paper's headline tables. Offline methods process the whole sequence at once, so FPS is not comparable.

The long-sequence test is the one that matters for real use. Stretched to 3,840 frames on Oxford Spires, LingBot-Map's error grows by just 0.69 m to 7.11 m, while CUT3R and Wint3R blow out past 32 m. Reconstruction quality (F1, the balance of accuracy and completeness of the recovered surface) tells the same story: 98.98 on ETH3D versus 77.28 for the runner-up, plus wins on 7-Scenes (80.39) and NRGBD (64.26).

Run it in about 10 minutes

You need an NVIDIA GPU with around 14 GB of free VRAM and CUDA 12.8 drivers; a used RTX 3090 or any 4090-class card clears the bar. There is no Ollama route here, this is a Python research repo, but an unusually tidy one. If you are still deciding what box to build, our local AI hardware guide covers the trade-offs.

# 1. Environment (Python 3.10 + PyTorch 2.8 with CUDA 12.8)
conda create -n lingbot-map python=3.10 -y
conda activate lingbot-map
pip install torch==2.8.0 torchvision==0.23.0 \
    --index-url https://download.pytorch.org/whl/cu128

# 2. Clone and install, with the browser viewer extras
git clone https://github.com/Robbyant/lingbot-map.git
cd lingbot-map
pip install -e ".[vis]"
pip install flashinfer-python   # recommended, big speedup

# 3. Grab a checkpoint (HF: robbyant/lingbot-map)
#    lingbot-map-long is the one you want for real videos.

# 4. Reconstruct the bundled example scene
python demo.py --model_path /path/to/lingbot-map-long.pt \
    --image_folder example/courthouse --mask_sky

The demo spins up a viser viewer at http://localhost:8080, and you can watch the point cloud grow frame by frame in your browser. Four example scenes ship in the repo (courthouse, university, a loop-closure trajectory, and an outdoor Oxford sequence). For your own footage, pass --video_path video.mp4 instead of an image folder, and add --mask_sky outdoors so the sky does not smear into the geometry (it pulls a small ONNX segmentation model on first run, so pip install onnxruntime first).

Two flags do most of the tuning work. --keyframe_interval 2 halves KV-cache memory on long videos, and --mode windowed --window_size 128 is mandatory past roughly 3,000 frames, resetting state in overlapping chunks so drift never accumulates. Tight on VRAM? --offload_to_cpu trades speed for headroom.

Limitations and gotchas

First, the hard requirement: CUDA 12.8 on NVIDIA silicon. There is no Apple Silicon or ROCm path today, though a community Mac desktop viewer lets a Mac display a reconstruction streamed from a Linux GPU box.

Second, depth range is bounded by the training data distribution. Feed it aerial footage from 200 m up and the geometry gets vague, because the model never saw scenes at that scale. Third, quality degrades past about 320 frames of raw KV cache, so the keyframe and windowed modes are not optional extras for long captures, they are the operating manual. And the output is a point cloud with camera poses, not a textured mesh: for meshes you still hand the result to downstream tooling, or reach for a generative model like Hunyuan3D-2 when the goal is an asset rather than a map.

Who should use it

Robotics and embodied AI builders are the obvious audience, since real-time ego-motion plus geometry from one RGB camera is the input layer for navigation, and it slots naturally under world models like NVIDIA Cosmos 3. Drone and scanning hobbyists get COLMAP-grade capture that finishes when the flight does. And 3D content pipelines can use the poses and points as fast initialization for NeRF or Gaussian splatting runs, the same way LHM-1B shortcuts human reconstruction.

The 10-minute move: clone the repo, download lingbot-map-long, and run the courthouse example. Watching a building assemble itself in your browser at 20 FPS explains this release better than any benchmark table.

Sources and further reading

• LingBot-Map on GitHub
• Model weights on Hugging Face
• Model weights on ModelScope
• Paper: Geometric Context Transformer for Streaming 3D Reconstruction (arXiv)
• Community Mac desktop viewer

Tested on: not independently tested. LingBot-Map requires CUDA 12.8 on NVIDIA hardware, which we could not run for this article; all benchmark figures are paper-reported by the Robbyant team, and install steps are taken verbatim from the official repository.
Date checked: 2026-07-16