Deep learning-based modified transformer model for automated news article summarization

The amount of textual data on the internet is increasing enormously, so data

summarization into text has become essential. As generating text summaries

manually is an arduous task and humans are generally prone to make mistakes,

deep learning techniques have evolved to overcome this problem. Modified

transformer-based deep learning models with varying encoder-decoder and

feed-forward network layers are proposed to develop an abstractive summary

of the news articles. The proposed transformer model provides the advantage

of parallelization with the help of multiple attention head layers to

process long sentences, and hence, better text summarization performance is

achieved. These models are trained on an ?in-shorts? dataset, and the

proposed model is compared with the PEGASUS-CNNdaily-mail, BART-large-CNN,

and DistilBART-CNN-12-6 models on the CNN/DailyMail dataset. The performance

is evaluated in terms of the ROUGE score by comparing it with the existing

Recurrent Neural Network (RNN) model. The suggested transformer model

achieved a ROUGE score of 0.33, surpassing the RNN model score of 0.17. This

innovative approach can be employed on extensive textual data to extract

summaries or headlines.