MB-ORES: A Multi-Branch Object Reasoner for Visual Grounding in Remote Sensing
Karim Radouane, Hanane Azzag, Mustapha lebbah
2025-04-02
Summary
This paper is about creating a system that can automatically identify and locate objects in satellite images, using both object detection and visual grounding techniques.
What's the problem?
It's challenging to accurately find specific objects in satellite images, as it requires both recognizing the object and understanding its location in the image.
What's the solution?
The researchers developed a system called MB-ORES that combines object detection with a special network that reasons about the relationships between objects and their locations.
Why it matters?
This work matters because it can help with tasks like monitoring deforestation, tracking urban growth, and responding to natural disasters.
Abstract
We propose a unified framework that integrates object detection (OD) and visual grounding (VG) for remote sensing (RS) imagery. To support conventional OD and establish an intuitive prior for VG task, we fine-tune an open-set object detector using referring expression data, framing it as a partially supervised OD task. In the first stage, we construct a graph representation of each image, comprising object queries, class embeddings, and proposal locations. Then, our task-aware architecture processes this graph to perform the VG task. The model consists of: (i) a multi-branch network that integrates spatial, visual, and categorical features to generate task-aware proposals, and (ii) an object reasoning network that assigns probabilities across proposals, followed by a soft selection mechanism for final referring object localization. Our model demonstrates superior performance on the OPT-RSVG and DIOR-RSVG datasets, achieving significant improvements over state-of-the-art methods while retaining classical OD capabilities. The code will be available in our repository: https://github.com/rd20karim/MB-ORES.