Question Answering on 8-concept
86.2Concept Accuracy (Con)Baseline
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| BaselineBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 86.2 | 87.2 | |
| NoiseBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=Embedding-only2026.06 | 78.5 | 86.5 | |
| BaselineBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 74.5 | 78.5 | |
| BaselineBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 74.2 | 86 | |
| MeanBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=Embedding-only2026.06 | 69.5 | 84.2 | |
| BaselineBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 62 | 80.8 | |
| MeanBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=Embedding-only2026.06 | 61.5 | 83.5 | |
| PISCESBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 60.5 | 78.8 | |
| NoiseBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=Embedding-only2026.06 | 59.7 | 83.5 | |
| NoiseBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=Embedding-only2026.06 | 56.2 | 75.8 | |
| PISCESBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 51.5 | 82 | |
| EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=Embedding-only2026.06 | 50.5 | 81.2 | |
| MeanBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=Embedding-only2026.06 | 50.5 | 73.8 | |
| SNMFBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 45.2 | 61.5 | |
| EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=Embedding-only2026.06 | 45 | 82.5 | |
| EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=Embedding-only2026.06 | 44 | 71.8 | |
| RMUBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 43.2 | 68.2 | |
| CRISPBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 38.3 | 72.8 | |
| SNMFBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 38.2 | 67 | |
| SNMF + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 37.5 | 75.8 | |
| PISCES + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 37 | 80.5 | |
| PISCES + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 33.5 | 80 | |
| SNMF + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 30 | 72.5 | |
| MeanBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=Embedding-only2026.06 | 29 | 77 | |
| NoiseBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=Embedding-only2026.06 | 29 | 76.2 | |
| CRISPBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 28.7 | 62 | |
| CRISP + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 28.2 | 73 | |
| CRISP + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 27.5 | 79 | |
| RMUBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 25.2 | 74.8 | |
| RMU + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 25.2 | 75.8 | |
| EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=Embedding-only2026.06 | 25 | 72.5 | |
| RMU + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=MC, Method Category=MLP-based2026.06 | 21.2 | 77.5 | |
| SNMFBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 18.2 | 41 | |
| CRISP + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 12 | 69.2 | |
| RMU + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 12 | 59.8 | |
| SNMFBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 10 | 46.2 | |
| SNMF + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 10 | 58 | |
| CRISP + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 9.7 | 50.7 | |
| RMU + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 9 | 53.5 | |
| PISCES + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 8.5 | 53 | |
| RMUBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 8.2 | 36.2 | |
| PISCESBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 6.5 | 42 | |
| PISCES + EMBERBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 5.8 | 51 | |
| CRISPBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 5.5 | 34.2 | |
| PISCESBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 5.3 | 40 | |
| SNMF + EMBERBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 4.8 | 55.2 | |
| RMUBackbone=Llama-3.1-8B-Instruct, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 4.8 | 39.5 | |
| CRISPBackbone=Gemma-2-2B-it, Evaluation Protocol=OE, Method Category=MLP-based2026.06 | 2.5 | 33.5 |