Real Time Video Generation Inference On Baseten

Real-time video generation inference on Baseten Announcing our Series F . Learn more

Model performance

Real-time video generation inference on Baseten

We run Wan 2.2 video generation in 2.75 seconds per clip, a 53.6x improvement over the baseline implementation.

Authors

Ali Taha

Brendan Duke

Yikai Zhu

Faraz Shahsavan

Pankaj Gupta

Philip Kiely

Last updated July 16, 2026

Share

TL;DR We run Wan 2.2 video generation in 2.75 seconds per clip, a 53.6x improvement over the baseline implementation. Our runtime optimizations across custom kernels, NVFP4 quantization, and timestep distillation deliver these performance gains. This video generation runtime is available as a limited-time demo through July 31 behind our custom-built content guardrails.

Baseten’s model performance team has built a new video inference runtime that delivers SOTA performance on Wan 2.2, the leading open-source model for text-to-video generation.

This performance came from multiple model-level and runtime-level optimizations: Timestep distillation to four-step video generation (~20x faster than baseline)

Custom kernel optimization and fusion (~1.5x faster than original kernels)

Linear quantization in NVFP4 (~1.5x faster than original weights)

These improvements are multiplicative and together result in a 53.6x speedup in video generation speed, going from more than two minutes to less than three seconds. This also brings the cost per video generated down materially, from five cents to less than one-sixth of a cent per video (a 31x reduction in cost). This blog provides a high-level overview of the runtime performance work we completed, the infrastructure required to deliver that work at scale, and the guardrails we built to safely open the work to public usage. You can test the fast video inference for yourself at our public demo platform through July 31, 2026. Fast video inference runtime Unlike LLM inference, video generation is not autoregressive. Instead, diffusion works by iterating over the entire latent space of the final video with bi-directional attention for consistency across the entire sequence. The techniques and tooling for video generation are accordingly quite different than for LLM inference, and the overall stack is less mature. This leaves room for substantial performance improvement via both post-training and inference optimization. Timestep distillation The biggest performance lever was model-level optimization via timestep distillation. By default, Wan 2.2 uses 40 to 80 steps to complete the diffusion process from noise to video. By post-training the model with timestep distillation, we can reduce that to four steps. Timestep distillation is a lossy optimization. With four steps, the model is more likely to produce visual artifacts and less likely to properly handle consistency and physics in active scenes, like a video of a player dunking a basketball over an opponent. However, reaching real-time video generation currently requires this sacrifice. Timestep distillation brings video generation from >120 seconds to ~6 seconds, bringing it in range of the other optimization methods to get below the five-second real-time mark. For video models, we apply Distribution Matching Distillation (DMD2) as our core training algorithm. Rather than matching individual denoising steps, DMD2 trains a student model to match the distribution of outputs from a teacher model. Training involves three components updated in alternation: the student generator, a fake score model that prevents mode collapse, and a GAN discriminator that sharpens fine-grained details. To keep attention over the full temporal-spatial context tractable, we use sequence parallelism, distributing the long context across multiple devices during training. Custom kernel engineering Unlike LLMs, we don’t use a pre-built inference engine to run video generation models. While there are now excellent open-source options like SGLang video, frontier performance still comes from building from scratch in PyTorch. Wan 2.2 inference is bottlenecked on Self-Attention. Using Video Sparse Attention (VSA) would allow the model to only attend to pieces of the sequence, saving substantial time. However, the default VSA Triton kernel did not offer sufficient performance. After 10 iterations, we created a custom kernel with 1.58x better performance. We also optimized an additional nine kernels that contributed less to the end-to-end time. Fast individual kernels are not enough. The bottleneck shifts from executing kernels to launching them. However, via kernel fusion, we were able to fuse Adaptive LayerNorm modulation, RMS Norm, and Gated Residual Add together into a single triton kernel, saving hundreds of launches across inference. For a deep dive on the kernel work, read the dev notes by Ali Taha on building frontier kernels. NVFP4 quantization Going from 16 to 4 bits is the last big jump in performance. On an NVIDIA B200 GPU, 4-bit inference accesses roughly 4x higher tensor core throughput than BF16. After applying our attention optimizations, the diffusion transformer becomes dominated by operations on the linear layers, so moving the GEMM operations to NVFP4 alleviates the bottleneck.

After quantization to NVFP4, output quality remains visually comparable to BF16. For more on quantization, see our writeup on quantizing FLUX.2 [dev] . Scalable inference infrastructure No matter how fast an individual instance serving the model is, it will be overwhelmed with enough traffic. Video generation infrastructure is just as important as runtime optimizations for building a stable, scalable system. This is an especially important problem for video as video generation runs with a batch size of one. Even with faster than real time inference, horizontal scale is essential for running production services. Of course, one essential mechanism is autoscaling. As traffic to the system increases, extra GPU capacity should be spun up to support the increased traffic. This requires a cold start. For a video generation model, the cold start has four parts: Provisioning and scheduling the compute

Pulling the inference container image onto the compute

Downloading the model weights into the container

Starting up the model inference server

Given the size of the images and weights involved, the second and third steps often take the longest. We built the Baseten Delivery Network to optimize this process for large models, including Wan 2.2. Regardless of...