New Method Reveals How AI Thinks, Step by Step
Finally, a way to see inside AI's decision-making process—could make AI safer and more trustworthy.
Artificial intelligence networks, like those used in image recognition or self-driving cars, are often black boxes—we see what goes in and what comes out, but not how the decision is made. Now, scientists have created a tool called Complete Path Contribution that breaks down the network's output into the contributions of every possible path from input to output. This gives a complete, mathematically guaranteed explanation of how the network computes.
Previous methods only looked at interactions between two components at a time, like two layers or two neurons. But real decisions involve many components working together in complex chains. This new method captures those higher-order interactions, revealing, for example, how inhibitory signals (which suppress) and excitatory signals (which activate) combine to shape the final output. The researchers also developed an efficient algorithm to find the most important paths without checking every single one, making it practical even for deep networks.
They tested the method on image classification networks, quantifying how excitatory and inhibitory pathways interact to construct decisions. In a model of the mouse retina, they showed how dynamic receptive fields—how a neuron's response changes over time—emerge from interactions among pathways between different cell types. This level of detail was previously unmeasurable.
The upshot: we can now understand how AI networks transform inputs to outputs at a mechanistic level. This could lead to more transparent and trustworthy AI systems, and also help neuroscientists understand how biological brains compute. While the method is currently a research tool, it's a step toward AI that can explain itself.
- Complete Path Contribution traces all paths information takes through a network, giving a complete explanation of how it makes decisions.
- It reveals how excitatory and inhibitory signals interact, which was previously impossible to measure.
- The method is efficient and works on real networks, including models of the mouse retina.
Why It Matters
This could make AI more transparent and trustworthy, and help us understand how brains compute.