Enhancing AIOps with LILT: A LoRA-Based Incremental Learning Transformer for Robust Adaptation to Concept Drift

With the widespread application of artificial intelligence technology in the field of operations and maintenance, AIOps (Artificial Intelligence for IT Operations) has become a key technology for improving operational efficiency and automated decision-making. However, traditional machine learning models often struggle to maintain stable performance when dealing with concept drift and catastrophic forgetting. To address this issue, this paper proposes the LILT model (LoRA-based Incremental Learning Transformer), which implements incremental learning in the Transformer neural network through LoRA fine-tuning technology. Additionally, the model incorporates regularization methods to constrain the model parameters, effectively addressing concept drift and knowledge retention issues in hard disk failure prediction. The LoRA adapter facilitates incremental learning by fine-tuning the feedforward layer parameters, improving the model’s adaptability when new tasks arise. Meanwhile, freezing the parameters of the self-attention layer ensures the stability of the task and promotes the accumulation of knowledge over long-term learning. Regularization techniques add constraints to control model parameter changes, ensuring stability between tasks and supporting long-term knowledge retention. Moreover, the integrated gradient (IG) method is employed to analyze the feature importance of the model, specifically demonstrating the contribution of each input feature to the model’s predictions by outputting the tensor of feature contributions. Experimental results show that LILT, when applied to hard disk failure prediction tasks with concept drift, demonstrates stronger task adaptability and stability compared to traditional Transformer models. Furthermore, LILT has a lower forgetting rate compared to the LoRA-based incremental learning method. The model has made significant progress in retaining the memory of old tasks, effectively mitigating catastrophic forgetting. At the same time, the integrated gradient method enhances the interpretability of LILT. This study provides a novel solution for the fault prediction domain, demonstrating strong stability and interpretability in a drift-driven streaming data environment.