Question Answering on Concept Questions Multiple-Choice
84.4Concept AccuracyBaseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BaselineBackbone=Llama-3.1-8B-Instruct2026.06 | 84.4 | 92.4 | |
| NoiseBackbone=Llama-3.1-8B-Instruct2026.06 | 76.9 | 90.6 | |
| BaselineBackbone=Gemma-2-2B-it2026.06 | 73.3 | 89.2 | |
| MeanBackbone=Llama-3.1-8B-Instruct2026.06 | 69.6 | 88.4 | |
| MeanBackbone=Gemma-2-2B-it2026.06 | 62.8 | 87.6 | |
| NoiseBackbone=Gemma-2-2B-it2026.06 | 62.2 | 87.6 | |
| EMBERBackbone=Gemma-2-2B-it2026.06 | 49.4 | 86.4 | |
| EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 45.8 | 85.4 | |
| SNMFBackbone=Llama-3.1-8B-Instruct2026.06 | 43.8 | 75.1 | |
| SNMFBackbone=Gemma-2-2B-it2026.06 | 43.7 | 69.9 | |
| RMUBackbone=Gemma-2-2B-it2026.06 | 43.4 | 72.4 | |
| SNMF + EMBERBackbone=Gemma-2-2B-it2026.06 | 37.7 | 77.8 | |
| CRISPBackbone=Llama-3.1-8B-Instruct2026.06 | 36.9 | 76.6 | |
| CRISPBackbone=Gemma-2-2B-it2026.06 | 32.9 | 70.9 | |
| RMU + EMBERBackbone=Gemma-2-2B-it2026.06 | 29.9 | 80.8 | |
| CRISP + EMBERBackbone=Gemma-2-2B-it2026.06 | 29.2 | 82 | |
| SNMF + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 29.1 | 76.4 | |
| CRISP + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 26.9 | 79.2 | |
| RMUBackbone=Llama-3.1-8B-Instruct2026.06 | 22.8 | 78 | |
| RMU + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 22.2 | 81.6 |