Project Lyra (featuring Lyra 1.0 and Lyra 2.0) is an open-source generative 3D world modeling framework developed by NVIDIA’s Spatial Intelligence Lab (NV-TLabs). Built on a 14-billion parameter Diffusion Transformer backbone (WAN-14B), Lyra transforms a single 2D photograph into a persistent, geometrically consistent, and navigable 3D environment—reconstructing camera-controlled walkthrough video sequences into explicit 3D Gaussian Splats (3DGS) and surface meshes for robotics simulation, real-time rendering, and virtual production.

Early single-image 3D generation models were largely restricted to isolated objects or short novel-view sweeps that deteriorated when the camera moved beyond a narrow field of view. Project Lyra 2.0 solves two foundational bottlenecks in long-horizon scene synthesis: spatial forgetting (hallucinating new structures when revisiting previous viewpoints) and temporal drifting (accumulating reconstruction errors over extended camera paths).

Model Specifications & Technical Overview

Dimension / Feature	Project Lyra 2.0 Technical Specification	Core Architectural Benefit
Foundation Backbone	14B Parameter Diffusion Transformer (WAN-14B base)	Scalable generative video synthesis across diverse environments.
Primary Input	Single Image (480×832) + 6-DoF Camera Trajectory	Generate fully custom scene paths from any still photo.
Output Formats	3D Gaussian Splats (3DGS) & Surface Meshes	Direct export into real-time renderers, physics engines, and Isaac Sim.
Temporal Drift Fix	Self-Augmented History Training	Exposes the model to its own degraded outputs to actively correct drift.
Spatial Memory Fix	Geometry-Guided Memory Retrieval	Routes historical frames via 3D coordinate warping to prevent forgetting.
Interactive Tools	Web GUI & Camera Path Planner	Visual walkthrough controls for path drawing and interactive scene expansion.
Open-Source Code	Apache 2.0 Codebase & Weights on Hugging Face	Open research artifacts for robotics, physical AI, and spatial computing.

Core Breakthroughs & Capabilities

• Two-Stage World Generation Pipeline: Lyra decouples world generation into two specialized phases: first, it generates a long-range, camera-conditioned video sequence with global geometric coherence; second, it lifts the synthesized frames into an explicit 3D Gaussian representation and polygon meshes for real-time navigation.

• Geometry-Routed Spatial Memory (Eliminating Forgetting): Rather than blindly appending raw past frames into attention context, Lyra maintains a lightweight 3D geometric proxy. When the camera loops back to a previously visited area, the system uses coordinate warping to retrieve relevant viewpoints and enforce dense correspondence without baking in visual artifacts.

• Self-Augmented History Training (Eliminating Drift): To prevent small frame-by-frame errors from snowballing over long camera runs, the model is trained on its own synthetic imperfections. This self-correcting dynamic closes the train-inference distribution gap across multi-hundred-frame trajectories.

• Robotics Simulation & Embodied AI Ready: Generated 3D assets load natively into physics platforms like NVIDIA Isaac Sim, allowing robotics researchers to rapidly convert indoor/outdoor photos into interactive training environments for navigation and manipulation agents.

• Interactive GUI & Trajectory Editor: Ships with an interactive interface where users can draw camera paths in 3D space, preview multi-view walkthroughs, and iteratively expand spatial boundaries as the camera moves forward.