Leveraging Transformer Models and eXplainable Reinforcement Learning Methods for Advanced Intrusion Detection and Response System

As cyber threats become increasingly sophisticated and frequent, the need for advanced Intrusion Detection and Response Systems (IDRS) is more critical than ever. These systems are essential for defending networks against external and internal intrusions by detecting potential threats and applying appropriate responses. However, a significant challenge faced by existing IDRSs is the overwhelming volume of alerts generated by IDSs, which makes manual response impractical. Additionally, the effectiveness of automated IRS is often undermined by difficulties in accurately estimating response costs, assessing the network situation, and providing clear explanations for chosen responses. These challenges often lead to suboptimal responses, negatively impacting network performance and pushing administrators towards inefficient manual methods. To overcome these challenges, this paper introduces a novel approach that integrates transformer encoder, decision transformer, and eXplainable Reinforcement Learning (XRL) methods to build an AI-powered IDRS. Experiments show that the proposed transformer-based IDS achieves performance on par with or surpass traditional treebased and neural models on tabular network flows. Despite similar performance, transformers offer greater flexibility and adaptability to shifting feature distributions and evolving attack patterns, making them well-suited for real-world deployment. I In addition, the proposed Decision Transformer-based IRS enables dynamic selection of optimal responses tailored to the current network situation, detected attack type, and contextual constraints, offering explainable and adaptive defense in real time.