Meta-learning Fine-tuned Feature Extractor for Few-shot Image Classification: A Case Study on Fine-tuning CNN Backbone with Transformer for Few-shot Learning

The need for a massive number of labeled training data has become a challenge for applying current deep neural networks to real-world applications. For deep neural networks, prior knowledge gained from previous training tasks can hardly adapt to new tasks with only few training samples. In this case, few-shot learning has received significant attention. Supervised pre-training is a well-known approach in visual recognition to endow a neural network with the discrimination ability of different classes. Meta-learning is another recently emerging training paradigm, which improves the generalization ability of neural networks, leading a better adaptation to new tasks with few-shot samples. In this work, we leverage meta-learning with a transformer to fine-tune a pre-trained CNN backbone, which improves the generalizability while maintaining the discriminability gained from the vanilla supervised pre-training. Experiments show that our proposed approach achieves state-of-the-art performance on current few-shot learning benchmark datasets.