Researchers pinpoint LLM neurons that detect malicious code

A team of researchers from Vietnam has published a paper accepted at ESEM 2026 that uses mechanistic interpretability to identify which neurons inside large language models (LLMs) are responsible for detecting malicious code. They probed three instruction-tuned models—Llama3.1-8B-Instruct, Mistral-v0.3-7B-Instruct, and Qwen2.5-7B-Instruct—using 1,500 malicious and 1,500 benign PyPI packages from the PyPI Malregistry. By attributing the malware-detection behavior to specific feed-forward network (FFN) neurons, they discovered that amplifying 'facilitating' neurons while suppressing 'inhibiting' ones can improve classification accuracy. However, the reverse operation—amplifying inhibiting neurons—collapses predictions toward a single class, highlighting the fragility of these internal circuits.

The study found that the guardrail detection mechanism varies significantly across models, meaning each LLM encodes malicious programming concepts differently within its FFN layers. This model-dependent behavior has important implications for security alignment: it suggests that a one-size-fits-all approach to unlearning harmful knowledge won't work. Instead, the authors advocate for neuron-level editing, selective unlearning, and security-aware alignment tailored to each model's internal architecture. The work opens the door to more reliable defense mechanisms for code-focused LLMs, potentially allowing developers to surgically remove or strengthen malware-detection capabilities without retraining.