Design AI Agent for Auditing: Applying Large Language Models (LLMs) and Retrieval Augmented Generations (RAG) to Audit Workflows

Foundation large language models (LLMs) face limitations in specialized auditing domains, including accuracy issues, contextual memory constraints, and manual document management requirements. This proposal introduces an AI agent framework specifically designed for auditing workflows, integrating three core components: retrieval augmented generation (RAG) for seamless access to private knowledge bases, customizable workflows with intelligent query classification and multiagent coordination, and orchestrated prompts that embed standardized audit methodologies. The proposed framework reduces workflow disruptions and token consumption while maintaining accuracy. The proposal demonstrates the agent's workflow and its capabilities in document retrieval and analytical calculations. The evaluation plans to compare foundation LLM applications with customized AI agents using both baseline RAG and graph RAG approaches across auditing tasks, measuring accuracy against manually generated ground truth and efficiency through time and token consumption metrics. JEL Classifications: M42; O33; C88; C63; D83.