Error Detection on PaperAudit-Dataset ICML branch 1.0
6.4EC (Detection@1)Llama3.2-3B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Llama3.2-3BTraining Protocol=Zero-shot, Model Size=3B2026.01 | 6.4 | 7.1 | 6.7 | 34.2 | 5.6 | 10.1 | |
| Qwen3-8BTraining Protocol=Zero-shot, Model Size=8B2026.01 | 13.9 | 20.5 | 16.6 | 40.5 | 38.5 | 39.5 | |
| Qwen3-14BTraining Protocol=Zero-shot, Model Size=14B2026.01 | 16.9 | 20.5 | 18.5 | 43.7 | 40.5 | 42 | |
| Llama3.2-3B + SFTTraining Protocol=SFT, Model Size=3B2026.01 | 27.3 | 30.4 | 28.7 | 36.8 | 41 | 38.8 | |
| DeepSeek-V3.1Training Protocol=Zero-shot, Model Size=Frontier-scale2026.01 | 29 | 32.3 | 30.6 | 67.1 | 18.6 | 29.1 | |
| Llama3.2-3B + SFT + RLTraining Protocol=SFT + RL, Model Size=3B2026.01 | 30.4 | 33.9 | 32.1 | 37.8 | 42.1 | 39.8 | |
| Qwen3-8B + SFTTraining Protocol=SFT, Model Size=8B2026.01 | 38.4 | 42.8 | 40.5 | 43.1 | 47.6 | 45.2 | |
| Qwen3-14B + SFTTraining Protocol=SFT, Model Size=14B2026.01 | 42.4 | 47.2 | 44.7 | 48.5 | 52.7 | 50.5 | |
| Qwen3-235B-A22BTraining Protocol=Zero-shot, Model Size=235B2026.01 | 43.5 | 48.6 | 45.9 | 79.1 | 20.6 | 32.6 | |
| Claude-Sonnet-4.5Training Protocol=Zero-shot, Model Size=Frontier-scale2026.01 | 45.8 | 51 | 48.3 | 82 | 20.5 | 32.8 | |
| Gemini-2.5-ProTraining Protocol=Zero-shot, Model Size=Frontier-scale2026.01 | 48 | 53.8 | 50.6 | 75.4 | 27.3 | 40.1 | |
| Qwen3-8B + SFT + RLTraining Protocol=SFT + RL, Model Size=8B2026.01 | 52.3 | 58.3 | 55.1 | 56.6 | 63.1 | 59.7 | |
| Qwen3-14B + SFT + RLTraining Protocol=SFT + RL, Model Size=14B2026.01 | 52.9 | 59 | 55.8 | 57.1 | 63.7 | 60.2 |