TokenScope gives LLMs token-level explainability for code generation

TokenScope, developed by researchers Amirreza Esmaeili and Fatemeh Fard, is an interactive interpretability tool for decoder-based large language models (LLMs) focused on code generation. It provides fine-grained, token-level metrics such as probability distributions, logits, and attention patterns during the decoding process. Unlike existing tools that offer post-generation analysis or static attention maps, TokenScope works in real-time as tokens are generated, allowing users to see exactly which input tokens influence each output token. It also supports interactive token replacement: users can manually change a generated token and observe how the model adjusts the rest of the sequence. This enables counterfactual branching, where alternative generation paths can be explored to understand model decision-making under different conditions.

A key innovation is TokenScope's integration with abstract syntax trees (ASTs) for code. Instead of treating code as flat text, it aggregates token-level signals into structural components like functions, loops, or variable declarations. This code-aware aggregation helps developers pinpoint where LLMs might introduce logical errors or style inconsistencies. The tool is designed to work with any decoder-based LLM (e.g., GPT, CodeLlama) and is intended for both researchers studying model behavior and practitioners debugging AI-generated code. By making the token-level decision process transparent, TokenScope addresses a critical need for explainability in AI-assisted software development, potentially reducing the risk of subtle bugs and increasing trust in automated code generation.