Reading Comprehension on RACE (First-Token Accuracy)
87.3First-Token AccuracyZephyr-7B
Evaluation Results
| Method | Links | |
|---|---|---|
| Zephyr-7BStrategy=Prefilling2025.05 | 87.3 | |
| Qwen-2-7BStrategy=FTP2025.05 | 87 | |
| Qwen-2-7BStrategy=Prompt Engineering2025.05 | 86.7 | |
| Qwen-2-7BStrategy=Prefilling2025.05 | 86.7 | |
| Zephyr-7BStrategy=FTP2025.05 | 86.5 | |
| Gemma-2-9BStrategy=Prefilling2025.05 | 86.3 | |
| Gemma-2-9BStrategy=Prompt Engineering2025.05 | 86.2 | |
| Ministral-8BStrategy=FTP2025.05 | 84.7 | |
| Ministral-8BStrategy=Prefilling2025.05 | 84.7 | |
| Ministral-8BStrategy=Prompt Engineering2025.05 | 83.8 | |
| Llama-3.1-8BStrategy=Prefilling2025.05 | 83.2 | |
| Mistral-Nemo-12BStrategy=Prefilling2025.05 | 83.1 | |
| Mistral-Nemo-12BStrategy=FTP2025.05 | 82.7 | |
| Mistral-Nemo-12BStrategy=Prompt Engineering2025.05 | 79.5 | |
| Llama-3.1-8BStrategy=Prompt Engineering2025.05 | 78.2 | |
| Llama-3.1-8BStrategy=FTP2025.05 | 78.1 | |
| Phi-4-14BStrategy=Prompt Engineering2025.05 | 77.5 | |
| Phi-4-14BStrategy=Prefilling2025.05 | 73.4 | |
| Phi-4-14BStrategy=FTP2025.05 | 73.3 | |
| Zephyr-7BStrategy=Prompt Engineering2025.05 | 72.3 | |
| Gemma-7BStrategy=Prefilling2025.05 | 70.5 | |
| Gemma-7BStrategy=Prompt Engineering2025.05 | 65.2 | |
| Gemma-7BStrategy=FTP2025.05 | 50.4 | |
| Gemma-2-9BStrategy=FTP2025.05 | 49.5 |