Understanding the architecture of vision transformer and its variants: A review

The impressive performance of Transformer models in natural language tasks has sparked considerable interest within the computer vision community, leading to a focused exploration of their application to address challenges in computer vision problems. Transformers excel in modeling long-range dependencies among input sequence elements and enable parallel processing, which makes them different from traditional recurrent networks like Long Short-Term Memory (LSTM). Notably, Transformers exhibit minimal need for inductive biases in their design, making them naturally suited for diverse tasks. Their straightforward design allows the processing of multiple modalities, such as images, videos, text, and speech, using similar processing blocks, demonstrating exceptional scalability to large-capacity networks and extensive datasets. Leveraging these strengths has led to significant advancements in various vision tasks, marking a departure from convolutional networks in image classification. This survey provides a comprehensive overview of Transformer models in the field of computer vision, starting with an introduction to the core concepts that contributed to their success. The exploration includes a comparative analysis of the advantages and limitations of popular techniques, both in terms of architectural design and experimental value. Additionally, the survey delves into open research directions and potential future works, aiming to stimulate further interest within the community and address current challenges in the application of Transformer models in computer vision.