GimbalDiffusion: Gravity-Aware Camera Control for Video Generation
Frédéric Fortier-Chouinard, Yannick Hold-Geoffroy, Valentin Deschaintre, Matheus Gadelha, Jean-François Lalonde
2025-12-11
Summary
This paper introduces a new method, GimbalDiffusion, for creating videos from text descriptions with much more precise control over the camera's movement and viewpoint.
What's the problem?
Current text-to-video systems struggle with accurately controlling the camera. They usually describe camera movements relative to where the camera *was*, not in relation to the real world, making it hard to get the exact camera angles and movements you want. Existing videos used to train these systems also don't have a lot of variety in camera angles, mostly showing straight-on views.
What's the solution?
GimbalDiffusion solves this by defining camera paths using a real-world coordinate system, using gravity as a constant reference point. This means the camera's position is described as if you were looking at it from above, rather than relative to the previous frame. They also used 360-degree videos to train the system, giving it a much wider range of camera movements to learn from. Finally, they developed a technique called 'null-pitch conditioning' which helps the system prioritize camera instructions over potentially conflicting text prompts, like preventing it from generating grass when the camera is pointed straight up.
Why it matters?
This research is important because it makes text-to-video generation much more controllable and realistic. Being able to precisely control the camera opens up possibilities for creating more dynamic and visually interesting videos, and it’s a step towards being able to direct virtual scenes as easily as a real-world film director.
Abstract
Recent progress in text-to-video generation has achieved remarkable realism, yet fine-grained control over camera motion and orientation remains elusive. Existing approaches typically encode camera trajectories through relative or ambiguous representations, limiting explicit geometric control. We introduce GimbalDiffusion, a framework that enables camera control grounded in physical-world coordinates, using gravity as a global reference. Instead of describing motion relative to previous frames, our method defines camera trajectories in an absolute coordinate system, allowing precise and interpretable control over camera parameters without requiring an initial reference frame. We leverage panoramic 360-degree videos to construct a wide variety of camera trajectories, well beyond the predominantly straight, forward-facing trajectories seen in conventional video data. To further enhance camera guidance, we introduce null-pitch conditioning, an annotation strategy that reduces the model's reliance on text content when conflicting with camera specifications (e.g., generating grass while the camera points towards the sky). Finally, we establish a benchmark for camera-aware video generation by rebalancing SpatialVID-HQ for comprehensive evaluation under wide camera pitch variation. Together, these contributions advance the controllability and robustness of text-to-video models, enabling precise, gravity-aligned camera manipulation within generative frameworks.