Live Avatar (liveavatar.github.io) is an open-source, algorithm-system co-designed framework for real-time, interactive, and infinite-length audio-driven talking avatar generation. Powered by a 14-billion-parameter video diffusion foundation model distilled into a 4-step streaming pipeline, Live Avatar achieves over 45 FPS real-time rendering and low Time-to-First-Frame (TTFF ~1.21s). It introduces Timestep-forcing Pipeline Parallelism (TPP), Rolling RoPE, and Adaptive Attention Sinks (AAS) to enable continuous streaming exceeding 10,000+ seconds without identity drift or color degradation.

Traditional diffusion-based avatar generation methods suffer from two primary bottlenecks: slow sequential multi-step denoising that cannot keep up with real-time playback, and long-horizon autoregressive degradation (such as face morphing, color shift, and identity drift). Live Avatar addresses both sides of the problem by pairing distillation and pipeline parallelism with long-horizon architectural stability techniques.

Core Architecture & Technical Profile at a Glance

Feature / Dimension	Live Avatar Specification	Core Technical Advantage
Foundation Model Size	14 Billion Parameters (14B Diffusion Backbone)	Rich facial micro-expressions, clothing textures, and photorealistic skin rendering.
Sampling & Distillation	Distribution Matching Distillation (DMD) to 4 steps	Compresses multi-step bidirectional diffusion into an ultra-fast 4-step streaming model.
Pipeline Acceleration	Timestep-forcing Pipeline Parallelism (TPP)	Assigns GPUs fixed denoising timesteps for linear multi-device throughput scaling.
Frame Rate & Latency	45+ FPS with ~1.21s Time-to-First-Frame (TTFF)	Outpaces video playback speed for zero-lag conversational live streaming.
Long-Horizon Stability	Block-wise Autoregressive Processing (10,000+ seconds)	Generates hours of continuous video without face collapse or quality loss.
System Optimizations	FP8 Quantization, FlashAttention-3, cuDNN fused kernels, VAE caching	Yields a 189× FPS boost over raw baselines and fits on 48GB/80GB GPUs.
Licensing & Availability	Open Source (Apache 2.0) / Hugging Face & GitHub	Free for research, integration with multimodal dialogue agents (e.g., Qwen-Omni).

Key Capabilities and Architectural Innovations

• Timestep-forcing Pipeline Parallelism (TPP): Decouples the sequential diffusion denoising chain across multiple GPUs by fixing each device to a specific denoising step, converting sequential computation into an asynchronous spatial pipeline with linear hardware scaling.

• Eliminating Long-Term Identity & Distribution Drift:

  • Adaptive Attention Sink (AAS): Replaces static reference initializations with dynamically selected generated frames to prevent long-term color and lighting distortion.

  • Rolling RoPE: Continuously updates Rotary Position Embeddings on sink frames to maintain relative spatial-temporal positions.

  • History Corrupt: Injects controlled noise into the KV cache during training to build resilience against compounding error accumulation.

• Full-Duplex Conversational Agent Integration: When paired with speech-to-speech multimodal models (such as Qwen-Omni), Live Avatar enables human-like visual conversation where virtual assistants listen and speak with synchronized lip motion and dynamic head gestures in real time.

• GenBench Benchmark: Introduces a standardized long-form evaluation suite designed to evaluate long-duration talking avatar coherence, temporal consistency, and lip-sync accuracy across extended timeframes.