Scientific Reasoning on TheoremQA (test)
48.4AccuracyGPT-4-Turbo-0409
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4-Turbo-0409Few-shot CoT=5-shot2024.05 | 48.4 | |
| Qwen-1.5-110BParameter Size=110B, Few-shot CoT=5-shot2024.05 | 34.9 | |
| MAmmoTH2-8x7B-PlusParameter Size=8x7B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=5-shot2024.05 | 34.1 | |
| MAmmoTH2-8B-PlusParameter Size=8B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=5-shot2024.05 | 32.5 | |
| MAmmoTH2-8x7BParameter Size=8x7B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=5-shot2024.05 | 32.2 | |
| MAmmoTH2-8BParameter Size=8B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=5-shot2024.05 | 32.2 | |
| MAmmoTH2-34BParameter Size=34B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=5-shot2024.05 | 30.4 | |
| MAmmoTH2-7B-PlusParameter Size=7B, Training Stage=Continue trained with additional instruction datasets, Few-shot CoT=5-shot2024.05 | 29.2 | |
| MAmmoTH2-7BParameter Size=7B, Training Stage=Trained only with WEBINSTRUCT, Few-shot CoT=5-shot2024.05 | 29 |