AI-Powered Intelligent System through context aware log scrutiny for Anomaly detection
AI-Powered Intelligent System through context aware log scrutiny for Anomaly detection
Shilpi Gupta,George Asirvatharaj,R. Talluri
Abstract
System logs are digital signatures which play prominent role in system diagnostic, security analysis and performance monitoring and optimization. The extraction of right amount of information from raw logs is critical and depends on log processing process. Large language models (LLM) demonstrated impressive performance in natural language processing but still exhibit hallucination when analyzing logs which is due to implicit knowledge of rules in logs with which LLM is not capable. Therefore, we propose AI powered anomaly detection through intelligent system log analysis framework based on GPT 4.1 via GitHub copilot along with pre-trained knowledge injection to LLMs. This process includes three steps which consist of preprocessing and cleansing, fuzzy logic matcher along with semantic and AI based prediction for analysis. At initial stages, GPT 4.1 is tasked to generate series of rules related to system logs. LLM performs fault detection analysis and generates summary which includes possible hypotheses, identification of potential root cause, suspected faulty components list and recommendation for debug actions or additional tests. These are based on prior knowledge which we fed initially and based on model’s pretrained knowledge. Experiment demonstrates that proposed method outperforms existing LLM in log formulation and anomaly detection tasks and achieves improvements of +1.3% in GA, +7.2% in PA and +1.0% in ED compared to best baselines.
