LSNet: See Large, Focus Small
Ao Wang, Hui Chen, Zijia Lin, Jungong Han, Guiguang Ding
2025-04-03
Summary
This paper is about creating a new type of computer vision network that's both efficient and accurate, inspired by how humans see.
What's the problem?
Existing computer vision networks are either too complex to use in real-time applications or not accurate enough because they can't capture a wide range of visual information.
What's the solution?
The researchers developed a new type of convolution called LS (Large-Small) convolution, which combines large-kernel perception and small-kernel aggregation. This allows the network to efficiently capture a wide range of perceptual information and achieve precise feature aggregation for dynamic and complex visual representations.
Why it matters?
This work matters because it can lead to more efficient and accurate computer vision systems, which could be used in applications like self-driving cars, robotics, and medical imaging.
Abstract
Vision network designs, including Convolutional Neural Networks and Vision Transformers, have significantly advanced the field of computer vision. Yet, their complex computations pose challenges for practical deployments, particularly in real-time applications. To tackle this issue, researchers have explored various lightweight and efficient network designs. However, existing lightweight models predominantly leverage self-attention mechanisms and convolutions for token mixing. This dependence brings limitations in effectiveness and efficiency in the perception and aggregation processes of lightweight networks, hindering the balance between performance and efficiency under limited computational budgets. In this paper, we draw inspiration from the dynamic heteroscale vision ability inherent in the efficient human vision system and propose a ``See Large, Focus Small'' strategy for lightweight vision network design. We introduce LS (Large-Small) convolution, which combines large-kernel perception and small-kernel aggregation. It can efficiently capture a wide range of perceptual information and achieve precise feature aggregation for dynamic and complex visual representations, thus enabling proficient processing of visual information. Based on LS convolution, we present LSNet, a new family of lightweight models. Extensive experiments demonstrate that LSNet achieves superior performance and efficiency over existing lightweight networks in various vision tasks. Codes and models are available at https://github.com/jameslahm/lsnet.