Mean Accuracy Evaluation on NORMAD-ETI (test)
91.6AccuracyOracle model selection
Evaluation results
| Method | Links | |
|---|---|---|
| Oracle model selectionM1=EXAONE-3, M2=AYA-232025.05 | 91.6 | |
| Oracle model selectionM1=EXAONE-3, M2=INTERNLM-2.52025.05 | 91.3 | |
| Oracle model selectionM1=EXAONE-3, M2=YI-1.52025.05 | 90.8 | |
| Oracle model selectionM1=LLAMA-3, M2=EXAONE-32025.05 | 87.5 | |
| Oracle model selectionM1=YI-1.5, M2=SEALLM-32025.05 | 83.9 | |
| Oracle model selectionM1=GEMMA-2, M2=SEALLM-32025.05 | 83.8 | |
| Oracle model selectionM1=GEMMA-2, M2=YI-1.52025.05 | 83.3 | |
| Oracle model selectionM1=INTERNLM-2.5, M2=SEALLM-32025.05 | 83.2 | |
| Oracle model selectionM1=GEMMA-2, M2=INTERNLM-2.52025.05 | 82.8 | |
| Oracle model selectionM1=EXAONE-3, M2=SEALLM-32025.05 | 82.8 | |
| Oracle model selectionM1=LLAMA-3, M2=GEMMA-22025.05 | 82.6 | |
| Oracle model selectionM1=GEMMA-2, M2=AYA-232025.05 | 82.6 | |
| Oracle model selectionM1=AYA-23, M2=SEALLM-32025.05 | 82.5 | |
| Oracle model selectionM1=LLAMA-3, M2=SEALLM-32025.05 | 82.4 | |
| Oracle model selectionM1=GEMMA-2, M2=EXAONE-32025.05 | 82.3 | |
| Oracle model selectionProtocol=Average across all pairs2025.05 | 81.9 | |
| Self-Reflect+DebateM1=GEMMA-2, M2=EXAONE-32025.05 | 80.4 | |
| GEMMA-2-27BProtocol=Self-reflection baseline2025.05 | 80.1 | |
| Debate-OnlyM1=LLAMA-3, M2=GEMMA-22025.05 | 79.7 | |
| Debate-OnlyM1=GEMMA-2, M2=AYA-232025.05 | 79.7 | |
| Self-Reflect+DebateM1=EXAONE-3, M2=SEALLM-32025.05 | 79.3 | |
| GEMMA-2-27BProtocol=Judge LLM Single-model baseline2025.05 | 79.2 | |
| Debate-OnlyM1=GEMMA-2, M2=SEALLM-32025.05 | 79 | |
| Debate-OnlyM1=GEMMA-2, M2=EXAONE-32025.05 | 78.6 | |
| Self-Reflect+DebateM1=GEMMA-2, M2=SEALLM-32025.05 | 78.6 | |
| Debate-OnlyM1=GEMMA-2, M2=YI-1.52025.05 | 78.5 | |
| Debate-OnlyM1=GEMMA-2, M2=INTERNLM-2.52025.05 | 78.5 | |
| Debate-OnlyM1=EXAONE-3, M2=INTERNLM-2.52025.05 | 78.5 | |
| Self-Reflect+DebateM1=EXAONE-3, M2=AYA-232025.05 | 78.5 | |
| Self-Reflect+DebateM1=LLAMA-3, M2=EXAONE-32025.05 | 78.2 | |
| Self-Reflect+DebateM1=GEMMA-2, M2=INTERNLM-2.52025.05 | 77.7 | |
| Self-Reflect+DebateM1=EXAONE-3, M2=YI-1.52025.05 | 77.7 | |
| Self-Reflect+DebateM1=GEMMA-2, M2=YI-1.52025.05 | 77.6 | |
| Debate-OnlyM1=EXAONE-3, M2=SEALLM-32025.05 | 77.6 | |
| Debate-OnlyM1=EXAONE-3, M2=AYA-232025.05 | 77.5 | |
| Oracle model selectionM1=LLAMA-3, M2=YI-1.52025.05 | 77.3 | |
| Debate-OnlyM1=LLAMA-3, M2=AYA-232025.05 | 77 | |
| Debate-OnlyProtocol=Average across all pairs2025.05 | 76.3 | |
| Self-Reflect+DebateM1=GEMMA-2, M2=AYA-232025.05 | 76 | |
| Self-Reflect+DebateM1=LLAMA-3, M2=GEMMA-22025.05 | 75.9 | |
| Debate-OnlyM1=LLAMA-3, M2=SEALLM-32025.05 | 75.7 | |
| Oracle model selectionM1=YI-1.5, M2=INTERNLM-2.52025.05 | 75.7 | |
| Self-Reflect+DebateProtocol=Average across all pairs2025.05 | 75.6 | |
| Debate-OnlyM1=EXAONE-3, M2=YI-1.52025.05 | 75.5 | |
| Debate-OnlyM1=LLAMA-3, M2=EXAONE-32025.05 | 75.4 | |
| Debate-OnlyM1=INTERNLM-2.5, M2=SEALLM-32025.05 | 75 | |
| Oracle model selectionM1=LLAMA-3, M2=INTERNLM-2.52025.05 | 74.8 | |
| Self-Reflect+DebateM1=LLAMA-3, M2=INTERNLM-2.52025.05 | 74.8 | |
| Debate-OnlyM1=LLAMA-3, M2=YI-1.52025.05 | 74.7 | |
| Debate-OnlyM1=LLAMA-3, M2=INTERNLM-2.52025.05 | 74.7 | |
| Self-Reflect+DebateM1=LLAMA-3, M2=SEALLM-32025.05 | 74.7 | |
| Self-Reflect+DebateM1=INTERNLM-2.5, M2=SEALLM-32025.05 | 74.7 | |
| Self-Reflect+DebateM1=LLAMA-3, M2=YI-1.52025.05 | 74.5 | |
| Self-Reflect+DebateM1=YI-1.5, M2=INTERNLM-2.52025.05 | 74.4 | |
| Debate-OnlyM1=AYA-23, M2=SEALLM-32025.05 | 74.4 | |
| Oracle model selectionM1=YI-1.5, M2=AYA-232025.05 | 74.1 | |
| Self-Reflect+DebateM1=YI-1.5, M2=SEALLM-32025.05 | 74.1 | |
| Debate-OnlyM1=INTERNLM-2.5, M2=AYA-232025.05 | 74.1 | |
| Debate-OnlyM1=YI-1.5, M2=SEALLM-32025.05 | 74 | |
| Debate-OnlyM1=YI-1.5, M2=INTERNLM-2.52025.05 | 73.9 | |
| Self-Reflect+DebateM1=LLAMA-3, M2=AYA-232025.05 | 73.8 | |
| Oracle model selectionM1=LLAMA-3, M2=AYA-232025.05 | 72.9 | |
| Self-Reflect+DebateM1=YI-1.5, M2=AYA-232025.05 | 72.8 | |
| Self-Reflect+DebateM1=INTERNLM-2.5, M2=AYA-232025.05 | 72.6 | |
| Self-Reflect+DebateM1=EXAONE-3, M2=INTERNLM-2.52025.05 | 70.9 | |
| Oracle model selectionM1=INTERNLM-2.5, M2=AYA-232025.05 | 70.8 | |
| YI-1.5Protocol=Single-model baseline2025.05 | 70.7 | |
| Self-Reflect+DebateM1=AYA-23, M2=SEALLM-32025.05 | 70.1 | |
| Debate-OnlyM1=YI-1.5, M2=AYA-232025.05 | 69.9 | |
| GEMMA-2Protocol=Single-model baseline2025.05 | 68.9 | |
| SEALLM-3Protocol=Single-model baseline2025.05 | 68.1 | |
| INTERNLM-2.5Protocol=Single-model baseline2025.05 | 67.8 | |
| AYA-23Protocol=Single-model baseline2025.05 | 65.8 | |
| LLAMA-3Protocol=Single-model baseline2025.05 | 63.7 | |
| EXAONE-3Protocol=Single-model baseline2025.05 | 63.5 |