Hybrid Design of CNN and Vision Transformer: A Review
He Long
Abstract
Convolutional Neural Network (CNN), with its remarkable feature extraction capabilities, has proven to perform superior in computer vision applications. Conversely, transformer-based models have transformed conventional neural network topologies, showing remarkable results in multiple application areas. In recent years, hybrid models combining CNN and Vision Transformer have been increasingly emerging, with extensive research continuously overcoming the weaknesses of both models, effectively leveraging their respective strengths to deliver outstanding performance in various vision tasks. In this paper, we offer an exhaustive examination of hybrid models that amalgamate the functionalities of CNN and Vision Transformer. Firstly, we provide a comprehensive overview of fundamental concepts of Vision Transformer and summarize the concepts and advantages of hybrid models. Secondly, we systematically review the design philosophy of hybrid models and highlight major representative hybrid models. Lastly, we conduct an in-depth analysis of the future research directions for hybrid models. Through this paper, we aim to provide insightful and valuable references for the design of hybrid models combining CNN and Vision Transformer.