TAAF: A Trace Abstraction and Analysis Framework Synergizing Knowledge Graphs and LLMs

Abstract

Execution trace data contains a rich source of information crucial for understanding, debugging, and optimizing software executions. However, traces generated from operating systems or complex applications like Chrome or MySQL are often extremely large and difficult to analyze. Existing trace analysis and visualization tools typically rely on predefined analyses. When users need specific, customized insights, they often find it either impossible with these tools or must develop their own analyses, which is time-consuming and requires significant domain knowledge. Even when suitable predefined analyses exist, users must manually locate the specific analysis, open it, scroll through extensive kernel events to identify particular timestamps, and finally interpret values that often still require expert knowledge to fully comprehend. This research introduces the Trace Abstraction and Analysis Framework (TAAF), a novel approach that integrates large language models (LLMs), knowledge graphs (KGs), and time-indexing techniques to bridge the gap between raw trace data and actionable insights. TAAF constructs a time-indexed knowledge graph from execution trace events, capturing structural and contextual relationships among threads, CPUs, and key system attributes. Generative AI models then utilize these knowledge graphs to answer user queries expressed in natural language, significantly reducing the manual effort and specialized expertise traditionally required in trace analysis. To validate TAAF we present TraceQA-100, a 100-question benchmark built on kernel traces, and run extensive experiments on this suite. Results demonstrate that combining knowledge graphs with generative AI within TAAF improves answer quality and accuracy up to 31.2% compared to manual methods or raw data alone, particularly in tasks involving multi-hop reasoning and causal analysis. We also examine the strengths and limitations of applying LLMs and KGs to trace analysis.