Selective Neuron Amplification in Transformer Language Models

About

Large language models often fail on tasks they seem to already understand. In our experiments, this appears to be less about missing knowledge and more about certain internal circuits not being strongly activated during inference. We explore Selective Neuron Amplification, which increases the influence of task relevant neurons without changing the model's parameters. The method works at inference time and does not permanently alter the model. SNA helps mainly when the model is uncertain, while having low effect when the model is already confident. This suggests that some model failures are due to weak activation rather than lack of capability.

Ryyan Akhtar, Payal Pahwa, Monika Arora · 2026

Related benchmarks

Task	Dataset	Result
Coding	Coding Easy	Baseline—	1
Coding	Coding Medium	Baseline Score—	1
Coding	Coding Hard	Baseline Score—	1
Logic	Logic Easy	Baseline Score—	1
Logic	Logic Medium	Baseline Score—	1
Logic	Logic Hard	Baseline Score—	1
Math	MATH Easy	Baseline Score—	1
Math	Math Medium	Baseline Score—	1
Math	MATH Hard	Baseline Score—	1
Poetry	Poetry Easy	Baseline Score—	1