Meta SAM 3D ([ai.meta.com/research/sam3d/](https://ai.meta.com/research/sam3d/), developed by Meta AI Research) is an open foundation model family for visually grounded single-image 3D reconstruction. Expanding the Segment Anything ecosystem from 2D pixel masks to 3D spatial representations, it turns a single 2D photo into fully textured 3D meshes, camera-relative spatial layouts, and articulated human body poses. Shipped as SAM 3D Objects (general scenes/objects) and SAM 3D Body (human mesh recovery with Momentum Human Rig), the models achieve a 5:1 human preference win rate over prior methods and power production features like “View in Room” on Facebook Marketplace.

Traditional single-view 3D reconstruction pipelines often fail on in-the-wild photographs containing complex occlusions, clutter, varied lighting, and multi-object interactions. Meta SAM 3D overcomes the “3D data barrier” by using a multi-stage training framework combining synthetic data pretraining with large-scale human- and model-in-the-loop real-world alignment. By segmenting and “3D-fying” individual elements within an image, it recovers accurate geometry, surface textures, and physical spatial relationships directly from standard RGB inputs.

Core Architecture & Technical Profile at a Glance

Feature / Dimension	Meta SAM 3D Specification	Core Research & Industrial Advantage
Model Family	SAM 3D Objects & SAM 3D Body	Specialized architectures for both generic objects/scenes and articulated humans.
Input Modalities	Single 2D RGB Image + Segment Prompts (or Point Clouds)	Generates full 3D assets from photos without multi-view camera arrays.
Output Deliverables	3D Meshes, PBR Textures, Gaussian Splats, & 3D Layouts	Production-ready assets compatible with game engines and AR/VR tools.
Body Representation	Momentum Human Rig (MHR) Integration	Decouples skeleton and surface shape for precise hands, feet, and pose editing.
Benchmark Performance	5:1 Win Rate in Human Preference / 62.9 MPJPE on EMDB	Robust reconstruction in cluttered scenes and complex body poses.
Evaluation Suite	SAM 3D Artist Objects & In-the-Wild 3D Benchmarks	Standardized, rigorous evaluation dataset for the 3D research community.
Open Availability	Open Weights, Inference Code & Web Demo (Segment Anything Playground)	Open-weights research release under Meta’s permissive model license.

Key Capabilities and Practical Applications

• Visually Grounded Single-View Object Reconstruction (SAM 3D Objects): Predicts hidden geometries, textures, and depth orientations the way humans infer 3D shapes from 2D scenes, maintaining camera-relative layout so reconstructed items sit naturally on ground planes.

• Detailed Human Body Recovery (SAM 3D Body): Recovers full-body human meshes from single photos or broadcast video frames. It accurately models complex articulations, hands, and feet even under partial occlusion.

• Integration with SAM 3 Concept Segmentation: Pairs with SAM 3’s promptable concept detection (text or click prompts) to isolate specific items in cluttered photos and turn them into standalone 3D digital assets.

• E-Commerce & Spatial Computing Deployments:

  • Facebook Marketplace “View in Room”: Powers interactive AR visualization, enabling shoppers to place 3D reconstructions of furniture and home decor into their rooms.

  • Game & AR/VR Asset Creation: Accelerates 3D prototyping by converting concept art or reference photos into editable meshes.

  • Robotics Simulation & Spatial AI: Generates 3D digital twins and proxy physics objects from camera feeds for robot training environments.