Reading Comprehension on TriviaQA
70.3AccuracyMistral
Evaluation Results
| Method | Links | |
|---|---|---|
| MistralSize=7B, Tokens=?, Shot(s)=5, Training Strategy=trained from scratch, Attention Type=softmax-attention2024.09 | 70.3 | |
| MSA-CPTTraining Strategy=sparse continued pretraining2026.06 | 67.7 | |
| MambaSize=7B, Tokens=1.2T, Shot(s)=5, Training Strategy=trained from scratch2024.09 | 66.2 | |
| FullTraining Strategy=Full-Attention baseline2026.06 | 66 | |
| GSASize=7B, Tokens=+100B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.09 | 65.8 | |
| MSA-PTTraining Strategy=from-scratch sparse pretraining2026.06 | 65.5 | |
| Llama2Size=7B, Tokens=2T, Shot(s)=5, Training Strategy=trained from scratch, Attention Type=softmax-attention2024.09 | 64.2 | |
| GemmaSize=7B, Tokens=6T, Shot(s)=5, Training Strategy=trained from scratch, Attention Type=softmax-attention2024.09 | 63.7 | |
| GSASize=7B, Tokens=+20B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.09 | 60.7 | |
| SUPRASize=7B, Tokens=+100B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.09 | 60.4 | |
| RWKV6Size=7B, Tokens=1.4T, Shot(s)=5, Training Strategy=trained from scratch2024.09 | 59.5 | |
| GLASize=7B, Tokens=+20B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.09 | 57.8 | |
| LLaMA2Model=LLaMA2, Parameters=7B2025.02 | 45.4 | |
| RetNetSize=7B, Tokens=+20B, Shot(s)=5, Training Strategy=finetuned from Mistral 7B2024.09 | 43 | |
| DiffuLLaMA + P2-selfModel=DiffuLLaMA, Parameters=7B, Sampling Strategy=P2-self2025.02 | 18.8 | |
| DiffuLLaMAModel=DiffuLLaMA, Parameters=7B, Sampling Strategy=ancestral sampling2025.02 | 18.5 | |
| GPT2-MModel=GPT2-M, Parameters=355M2025.02 | 6.7 | |
| GPT2-SModel=GPT2-S, Parameters=127M2025.02 | 4 | |
| DiffuGPT-MModel=DiffuGPT-M, Parameters=355M2025.02 | 3.8 | |
| DiffuGPT-SModel=DiffuGPT-S, Parameters=127M2025.02 | 2 | |
| SEDD-MModel=SEDD-M, Parameters=424M2025.02 | 1.8 | |
| SEDD-SModel=SEDD-S, Parameters=170M2025.02 | 1.5 | |
| Plaid1BModel=Plaid1B, Parameters=1.3B2025.02 | 1.2 |