LingBot-Map (robbyant/lingbot-map) is an open-source, Apache-2.0 feed-forward 3D foundation model developed by Robbyant for real-time 3D reconstruction and camera tracking directly from streaming RGB video. Powered by a Geometric Context Transformer (GCT) and Paged KV Cache attention, it generates camera poses and dense geometry online at ~20 FPS on a single GPU over sequences exceeding 10,000 frames without requiring test-time optimization (like COLMAP or iterative bundle adjustment).

Traditional 3D reconstruction pipelines (e.g., COLMAP, NeRF, 3DGS) rely on heavy multi-view matching, iterative bundle adjustment, or per-scene neural optimization, making them too slow for live robotics or real-time spatial navigation. LingBot-Map treats spatial reconstruction as an online feed-forward perception task, predicting poses and dense point clouds continuously as new video frames arrive.

Architecture & Technical Profile at a Glance

Feature / Metric	Technical Specification	Core Advantage
Model Type	Feed-Forward 3D Foundation Model	One single forward pass per frame; zero per-scene test-time optimization.
Core Architecture	Geometric Context Transformer (GCT)	Integrates anchor context, pose-reference windows, and trajectory memory.
Streaming Speed	~20 FPS @ 518×378 resolution (Single GPU)	Delivers real-time frame rates suitable for interactive visual odometry and SLAM.
Memory Efficiency	Paged KV Cache Attention (FlashInfer / SDPA)	Keeps memory overhead steady ($O(1)$) across sequences beyond 10,000 frames.
Long-Range Handling	Keyframe subsampling & Windowed Mode	Prevents RoPE degradation on sequences exceeding 320 views.
Output Modalities	Camera Poses, Dense Depth, Point Clouds & Viser exports	Direct export to interactive 3D viewers, flythrough videos, or robotics stacks.
License	Apache 2.0	Fully open-source weights, inference scripts, and benchmark tooling.

Core Breakthroughs & Capabilities

• Unified Geometric Context Transformer (GCT): Unifies three critical spatial components in a single streaming model: an anchor context for global coordinate and scale grounding, a pose-reference window for short-range local geometry, and a trajectory memory to eliminate accumulated spatial drift.

• Hardware-Accelerated Streaming Inference: Utilizes FlashInfer paged KV caching and PyTorch compilation to run continuous, low-latency streaming inference without hitting GPU out-of-memory errors on massive walkthrough sequences.

• Integrated Sky Masking: Includes an automatic ONNX-based sky segmentation preprocessor (--mask_sky) that filters out background sky pixels to produce clean, artifact-free outdoor point clouds (e.g., KITTI, Oxford Spires).

• Embodied AI & World-Model Synergy: Acts as a plug-and-play geometric layer for physical AI systems, converting generated world-model video feeds or live robot camera streams directly into spatial 3D maps and traversable obstacles.