KD-OCT: Efficient Knowledge Distillation for Clinical-Grade Retinal OCT Classification
Erfan Nourbakhsh, Nasrin Sanjari, Ali Nourbakhsh
2025-12-16
Summary
This research focuses on improving how quickly and efficiently doctors can diagnose age-related macular degeneration (AMD), a major cause of vision loss, using a type of eye scan called optical coherence tomography, or OCT.
What's the problem?
Currently, the best computer programs for analyzing these eye scans are very large and require a lot of computing power, making them difficult to use in real-time clinical settings like a doctor's office. It's hard to quickly get a diagnosis when the software takes a long time to process the images.
What's the solution?
The researchers developed a new technique called KD-OCT, which essentially 'teaches' a smaller, faster program (EfficientNet-B2) to perform almost as well as a much larger, more powerful program (ConvNeXtV2-Large). They did this by having the smaller program learn from the larger one, using a combination of the larger program’s knowledge and direct examples of what’s normal and what indicates disease. This 'teaching' process happens in real-time during training.
Why it matters?
This work is important because it allows for the creation of AMD screening tools that can be used more easily and quickly, even on less powerful computers. This could lead to earlier detection and treatment of AMD, potentially saving people from vision loss, and making screening more accessible in places without access to high-end computing resources.
Abstract
Age-related macular degeneration (AMD) and choroidal neovascularization (CNV)-related conditions are leading causes of vision loss worldwide, with optical coherence tomography (OCT) serving as a cornerstone for early detection and management. However, deploying state-of-the-art deep learning models like ConvNeXtV2-Large in clinical settings is hindered by their computational demands. Therefore, it is desirable to develop efficient models that maintain high diagnostic performance while enabling real-time deployment. In this study, a novel knowledge distillation framework, termed KD-OCT, is proposed to compress a high-performance ConvNeXtV2-Large teacher model, enhanced with advanced augmentations, stochastic weight averaging, and focal loss, into a lightweight EfficientNet-B2 student for classifying normal, drusen, and CNV cases. KD-OCT employs real-time distillation with a combined loss balancing soft teacher knowledge transfer and hard ground-truth supervision. The effectiveness of the proposed method is evaluated on the Noor Eye Hospital (NEH) dataset using patient-level cross-validation. Experimental results demonstrate that KD-OCT outperforms comparable multi-scale or feature-fusion OCT classifiers in efficiency- accuracy balance, achieving near-teacher performance with substantial reductions in model size and inference time. Despite the compression, the student model exceeds most existing frameworks, facilitating edge deployment for AMD screening. Code is available at https://github.com/erfan-nourbakhsh/KD- OCT.