Backward-Compatible Aligned Representations via an Orthogonal Transformation Layer
Simone Ricci, Niccolò Biondi, Federico Pernici, Alberto Del Bimbo
2024-08-22
Summary
This paper discusses a new method for updating visual retrieval systems without needing to redo all the previous work when a new model is introduced, using a technique called orthogonal transformation.
What's the problem?
When visual retrieval systems update their models to improve performance, they often face issues because the new and old models represent information differently. This means that every time a new model is added, all the existing data must be recalculated, which is time-consuming and resource-intensive.
What's the solution?
The authors propose a method called Orthogonal Compatible Aligned (OCA) representations. This approach expands the representation space of the model and uses an orthogonal transformation to ensure that the new model can work with data from the old model without needing to redo everything. This transformation keeps the original structure intact while allowing the model to learn new information effectively.
Why it matters?
This research is important because it simplifies the process of updating visual retrieval systems, making them more efficient and less resource-intensive. By allowing models to remain compatible over time, it enhances the reliability of these systems in practical applications, such as image searching and recognition.
Abstract
Visual retrieval systems face significant challenges when updating models with improved representations due to misalignment between the old and new representations. The costly and resource-intensive backfilling process involves recalculating feature vectors for images in the gallery set whenever a new model is introduced. To address this, prior research has explored backward-compatible training methods that enable direct comparisons between new and old representations without backfilling. Despite these advancements, achieving a balance between backward compatibility and the performance of independently trained models remains an open problem. In this paper, we address it by expanding the representation space with additional dimensions and learning an orthogonal transformation to achieve compatibility with old models and, at the same time, integrate new information. This transformation preserves the original feature space's geometry, ensuring that our model aligns with previous versions while also learning new data. Our Orthogonal Compatible Aligned (OCA) approach eliminates the need for re-indexing during model updates and ensures that features can be compared directly across different model updates without additional mapping functions. Experimental results on CIFAR-100 and ImageNet-1k demonstrate that our method not only maintains compatibility with previous models but also achieves state-of-the-art accuracy, outperforming several existing methods.