Automatic Text Summarization Based on Pre-trained Models

Due to the current technological eras, increased accessibility of textual information. Numerous people search and read vast amounts of information to ascertain what is necessary, comprehensive, and pertinent to their goals. Most published documents entice readers to read the rest but frequently omit the most important details. As a result, recent years have seen several attempts to automate this process. Despite advances in deep learning and machine learning, the need for many training samples has made achieving good results on low-resource datasets difficult. Additionally, transfer learning cannot be used in conventional training methods and used the pre-trained models to address a related problem rather than creating a new model from scratch to address the same problem and improve the dataset. Promising results were obtained by assessing the text summaries created by adjusting the dataset using already-trained models based on the transformer architecture with reduced training time. Natural language processing has significantly benefited from transfer learning, where a model is initially pre-trained on a data-rich job before being fine-tuned on a downstream task (NLP). Because of its usefulness, transfer learning generated various methods, strategies, and techniques.