CineOrchestra: One model to rule all cinematic video controls

CineOrchestra tackles the long-standing challenge of fine-grained control in cinematic video generation. Current text-to-video models handle multi-subject personalization, temporal control, multi-shot synthesis, or camera control in isolation, but never together. The key insight from Sharath Girish and co-authors is that all cinematic elements—subjects, events, cameras, shot transitions—share a fundamental structure: each is an entity acting over a specific temporal interval. By expressing them through a unified entity-centric conditioning framework with reference images, the architectural challenge reduces to a single positional encoding problem.

To solve that, CineOrchestra introduces two parameter-free coordinate rotary embeddings: interval-sampled temporal RoPE ensures consistent attention across events of wildly different durations, and 2D entity-temporal cross-attention RoPE disambiguates per-entity conditions and routes each to its correct spatiotemporal region. On two new benchmarks, the model beats six per-axis specialists in dense caption following and shot-transition timing, with consistent gains shown in pairwise user studies and ablation experiments. The project page includes demos and code links.