STEM Question Answering on SciQ (first-token accuracy)
98.3First-Token AccuracyLlama-3.1-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| Llama-3.1-8BStrategy=Prefilling2025.05 | 98.3 | |
| Gemma-2-9BStrategy=Prefilling2025.05 | 98.3 | |
| Phi-4-14BStrategy=Prefilling2025.05 | 98.3 | |
| Phi-4-14BStrategy=Prompt Engineering2025.05 | 98.2 | |
| Gemma-2-9BStrategy=Prompt Engineering2025.05 | 98.1 | |
| Ministral-8BStrategy=Prefilling2025.05 | 98 | |
| Llama-3.1-8BStrategy=Prompt Engineering2025.05 | 97.8 | |
| Qwen-2-7BStrategy=FTP2025.05 | 97.5 | |
| Qwen-2-7BStrategy=Prefilling2025.05 | 97.4 | |
| Ministral-8BStrategy=FTP2025.05 | 97.4 | |
| Ministral-8BStrategy=Prompt Engineering2025.05 | 97.3 | |
| Qwen-2-7BStrategy=Prompt Engineering2025.05 | 97.2 | |
| Mistral-Nemo-12BStrategy=Prefilling2025.05 | 97.2 | |
| Llama-3.1-8BStrategy=FTP2025.05 | 97.1 | |
| Mistral-Nemo-12BStrategy=FTP2025.05 | 96.8 | |
| Mistral-Nemo-12BStrategy=Prompt Engineering2025.05 | 95.9 | |
| Phi-4-14BStrategy=FTP2025.05 | 95.4 | |
| Gemma-7BStrategy=Prefilling2025.05 | 95.1 | |
| Gemma-7BStrategy=Prompt Engineering2025.05 | 93.6 | |
| Zephyr-7BStrategy=Prefilling2025.05 | 93.1 | |
| Zephyr-7BStrategy=Prompt Engineering2025.05 | 89.6 | |
| Gemma-7BStrategy=FTP2025.05 | 89 | |
| Zephyr-7BStrategy=FTP2025.05 | 88.5 | |
| Gemma-2-9BStrategy=FTP2025.05 | 31.8 |