Truthfulness on TruthfulQA (test)
48AccuracyNUDGING
Evaluation results
| Method | Links | |
|---|---|---|
| NUDGINGPair=L→G, Intervention Rate=16.6%2026.04 | 48 | |
| BlendInPair=L→G, Intervention Rate=16.6%2026.04 | 48 | |
| BlendInPair=Q→L, Intervention Rate=31.5%2026.04 | 47 | |
| ART-beamLLM=Llama3-8B-Instruct2026.04 | 46.4 | |
| Beam SearchLLM=Llama3-8B-Instruct2026.04 | 46 | |
| ITILLM=Llama3-8B-Instruct2026.04 | 46 | |
| NUDGINGPair=Q→L, Intervention Rate=31.5%2026.04 | 46 | |
| ART-greedyLLM=Llama3-8B-Instruct2026.04 | 45.8 | |
| DoLaLLM=Llama3-8B-Instruct2026.04 | 45.3 | |
| ACTLLM=Llama3-8B-Instruct2026.04 | 45.2 | |
| VanillaLLM=Llama3-8B-Instruct2026.04 | 45.1 | |
| NUDGINGPair=Q→G, Intervention Rate=30.9%2026.04 | 44 | |
| BlendInPair=Q→G, Intervention Rate=30.9%2026.04 | 44 | |
| AdaGCSteps (tokens)=60k (1T), AdamW eps=1e-152025.02 | 35.02 | |
| GlobalGCSteps (tokens)=60k (1T), AdamW eps=1e-82025.02 | 30.73 | |
| GlobalGCSteps (tokens)=60k (1T), AdamW eps=1e-152025.02 | 28.29 | |
| AdaGCSteps (tokens)=21k (350B), AdamW eps=1e-152025.02 | 24.92 | |
| GlobalGCSteps (tokens)=21k (350B), AdamW eps=1e-152025.02 | 23.39 | |
| ART-beamLLM=Llama2-7B-Chat2026.04 | 22.7 | |
| GlobalGCSteps (tokens)=21k (350B), AdamW eps=1e-82025.02 | 22.02 | |
| DoLaLLM=Llama2-7B-Chat2026.04 | 21.5 | |
| ART-greedyLLM=Llama2-7B-Chat2026.04 | 21.5 | |
| Beam SearchLLM=Llama2-7B-Chat2026.04 | 21.4 | |
| ITILLM=Llama2-7B-Chat2026.04 | 19.9 | |
| VanillaLLM=Llama2-7B-Chat2026.04 | 19 | |
| ACTLLM=Llama2-7B-Chat2026.04 | 18.7 |