HiViT: A Simpler and More Efficient Design of Hierarchical Vision Transformer

A new architecture named HiViT (short for hierarchical ViT, which is simpler and more efficient than Swin yet further improves its performance on fully-supervised and self-supervised visual representation learning), after pre-trained using masked autoencoder on ImageNet-1K.