Question Answering on Concept Questions Open-Ended
67.1Concept AccuracyBaseline
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BaselineBackbone=Llama-3.1-8B-Instruct2026.06 | 67.1 | 79.3 | |
| BaselineBackbone=Gemma-2-2B-it2026.06 | 52.3 | 81.9 | |
| NoiseBackbone=Llama-3.1-8B-Instruct2026.06 | 47.4 | 75.9 | |
| MeanBackbone=Llama-3.1-8B-Instruct2026.06 | 40.9 | 74 | |
| EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 35.8 | 71.9 | |
| NoiseBackbone=Gemma-2-2B-it2026.06 | 30.7 | 78.4 | |
| MeanBackbone=Gemma-2-2B-it2026.06 | 29.6 | 78.6 | |
| EMBERBackbone=Gemma-2-2B-it2026.06 | 25.3 | 77.1 | |
| SNMFBackbone=Gemma-2-2B-it2026.06 | 12.9 | 48.2 | |
| RMU + EMBERBackbone=Gemma-2-2B-it2026.06 | 12 | 63.8 | |
| CRISP + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 10.8 | 56 | |
| SNMFBackbone=Llama-3.1-8B-Instruct2026.06 | 10.7 | 53.2 | |
| CRISP + EMBERBackbone=Gemma-2-2B-it2026.06 | 10.1 | 69.1 | |
| SNMF + EMBERBackbone=Gemma-2-2B-it2026.06 | 9.7 | 59.6 | |
| RMU + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 8.1 | 56.4 | |
| RMUBackbone=Gemma-2-2B-it2026.06 | 7.3 | 46 | |
| SNMF + EMBERBackbone=Llama-3.1-8B-Instruct2026.06 | 6.7 | 56.1 | |
| CRISPBackbone=Llama-3.1-8B-Instruct2026.06 | 4.8 | 43.4 | |
| RMUBackbone=Llama-3.1-8B-Instruct2026.06 | 4.2 | 44.9 | |
| CRISPBackbone=Gemma-2-2B-it2026.06 | 2.1 | 41.9 |