Vidi (bytedance.github.io/vidi-website/, developed by ByteDance Research) is a family of Large Multimodal Models (LMMs) engineered for fine-grained video understanding and intelligent video creation/editing (VUE). Across its evolution through Vidi, Vidi2, and Vidi2.5 / Vidi2.5-Think, the system combines temporal retrieval (TR), 2D/3D spatio-temporal grounding (STG bounding boxes across frames), deep plot/character reasoning, and structured automated video editing planning (Vidi-Edit)—outperforming proprietary frontier models across long-form video evaluation benchmarks like VUE-TR-V2, VUE-STG, and VUE-PLOT.

Most commercial AI video pipelines focus strictly on generative synthesis (creating raw clips from text/images), while the bottleneck in professional production remains post-production: searching hours of raw footage, tracking dynamic subjects across time, and generating structured edit decisions. ByteDance Vidi bridges this divide by functioning like an expert AI film editor that analyzes raw, multi-hour video assets to pinpoint moments, locate on-screen entities, and formulate end-to-end editing plans.

Architecture & Benchmark Profile at a Glance

Feature / Dimension	ByteDance Vidi Specification	Core Production & Research Advantage
Model Family & Versions	Vidi-7B, Vidi1.5-9B, Vidi2 (12B), Vidi2.5 & Vidi2.5-Think	Multi-tier progression optimized via multimodal RL post-training.
Temporal Retrieval (TR)	Natural Language Timestamp Identification (VUE-TR-V2)	Locates exact start/end scene ranges from complex text or audio-based queries.
Spatio-Temporal Grounding (STG)	Simultaneous Timestamp Extraction + Bounding Box Trajectories	Locates when an event occurs and tracks where the target object moves over time.
Plot & Character Reasoning	Vidi2.5-Think & VUE-PLOT Benchmark (Character & Reasoning Tracks)	Evaluates long-form narrative arcs, character motivations, and dialogue subtext.
Automated Editing Planner	Vidi-Edit (Structured JSON/Plan Generation)	Converts raw clips into actionable edit decision lists (EDLs), cuts, and audio pacing.
Benchmark Validation	VUE-TR, VUE-STG, and VUE-PLOT	High-accuracy retrieval, tracking, and plot comprehension on multi-hour raw videos.

Core Capabilities and Technical Innovations

• Simultaneous Spatio-Temporal Grounding (STG): Unlike models that separate spatial object detection from temporal clipping, Vidi solves both in a single pass. Given a query (e.g., “the red sports car overtaking on the turn”), it outputs exact video timestamp ranges along with continuous spatial bounding box coordinates across every frame.

• Autonomous Video Editing Planning (Vidi-Edit): Ingests multi-camera raw takes, B-roll, and user creative constraints to output a machine-readable editing specification. This includes cut timestamps, crop coordinates, speaker-aware multi-view camera switching, and soundtrack/voiceover ducking attributes.

• Long-Form Plot Reasoning with Vidi2.5-Think: Incorporates chain-of-thought video reasoning to solve nuanced narrative queries (such as identifying subtle character interactions, continuity errors, and timeline relationships) across long-duration feature footage.

• Standardized Evaluation Ecosystem (VUE Benchmarks): ByteDance open-sourced comprehensive evaluation suites for the broader research community:

  • VUE-TR-V2: Evaluates long-duration multimodal temporal retrieval.

  • VUE-STG: Standardized benchmark for spatio-temporal object grounding in video.

  • VUE-PLOT: Multi-track benchmark assessing character recognition and complex plot reasoning.