The method trains a single DiT on sparse depth data with separate noise levels per modality and per-modality decoders. This lets the model perform conditional and joint generation of image and depth in different permutations, while learning from sparse real-world depth supervision.
Modality Forcing is useful for spatial generation, monocular depth estimation, and image-depth synthesis workflows. The project reports scaling behavior from 300M to 3B parameter T2I models and states that its strongest model is competitive with state-of-the-art monocular depth estimators.
