Reasoning on LQA
40.6First-Token AccuracyZephyr-7B
Evaluation Results
| Method | Links | |
|---|---|---|
| Zephyr-7BStrategy=Prefilling2025.05 | 40.6 | |
| Gemma-2-9BStrategy=Prefilling2025.05 | 35.8 | |
| Qwen-2-7BStrategy=Prefilling2025.05 | 35.5 | |
| Ministral-8BStrategy=Prefilling2025.05 | 34.6 | |
| Phi-4-14BStrategy=Prefilling2025.05 | 34.6 | |
| Phi-4-14BStrategy=Prompt Engineering2025.05 | 34.1 | |
| Llama-3.1-8BStrategy=Prefilling2025.05 | 34 | |
| Qwen-2-7BStrategy=FTP2025.05 | 33 | |
| Gemma-2-9BStrategy=Prompt Engineering2025.05 | 32.1 | |
| Zephyr-7BStrategy=FTP2025.05 | 32.1 | |
| Mistral-Nemo-12BStrategy=Prefilling2025.05 | 32 | |
| Mistral-Nemo-12BStrategy=Prompt Engineering2025.05 | 31.9 | |
| Phi-4-14BStrategy=FTP2025.05 | 31.9 | |
| Qwen-2-7BStrategy=Prompt Engineering2025.05 | 31.6 | |
| Gemma-2-9BStrategy=FTP2025.05 | 31 | |
| Zephyr-7BStrategy=Prompt Engineering2025.05 | 30.1 | |
| Ministral-8BStrategy=Prompt Engineering2025.05 | 29 | |
| Llama-3.1-8BStrategy=FTP2025.05 | 28.9 | |
| Mistral-Nemo-12BStrategy=FTP2025.05 | 28.7 | |
| Llama-3.1-8BStrategy=Prompt Engineering2025.05 | 28.4 | |
| Ministral-8BStrategy=FTP2025.05 | 28.4 | |
| Gemma-7BStrategy=Prompt Engineering2025.05 | 27.3 | |
| Gemma-7BStrategy=FTP2025.05 | 26.4 | |
| Gemma-7BStrategy=Prefilling2025.05 | 26.1 |