Scientific Reasoning on MMLU-Sci
54.96ScoreGalactica-30B
Evaluation results
| Method | Links | |
|---|---|---|
| Galactica-30BParameter Scale=30B~32B2024.01 | 54.96 | |
| Llama3-8B-InstructEvaluation Protocol=few-shot, Parameter Scale=6B~7B2024.01 | 52.67 | |
| ChatGLM3-32B-BaseParameter Scale=30B~32B2024.01 | 50.3 | |
| SciGLMBackbone=ChatGLM3-32B-Base, Parameter Scale=30B~32B2024.01 | 49.38 | |
| SciGLMBackbone=ChatGLM3-6B-Base, Parameter Scale=6B~7B2024.01 | 45.34 | |
| Mistral-7B: MetaMATHEvaluation Protocol=few-shot, Parameter Scale=6B~7B2024.01 | 44.74 | |
| Mistral-7B: MetaMATH + SciInstructFine-tuning=SciInstruct, Parameter Scale=6B~7B2024.01 | 42.16 | |
| ChatGLM3-6BParameter Scale=6B~7B2024.01 | 41.78 | |
| Llama3-8B-Instruct + SciInstructFine-tuning=SciInstruct, Parameter Scale=6B~7B2024.01 | 40.86 | |
| ChatGLM3-6B-BaseParameter Scale=6B~7B2024.01 | 40.16 | |
| ChatGLM2-6B-BaseParameter Scale=6B~7B2024.01 | 38.06 | |
| ChatGLM2-6BParameter Scale=6B~7B2024.01 | 37.09 | |
| LLaMA-2-13BParameter Scale=12B~13B2024.01 | 35.85 | |
| Vicuna-13BParameter Scale=12B~13B2024.01 | 32.13 | |
| Galactica-6.7BParameter Scale=6B~7B2024.01 | 30.68 | |
| LLaMA-2-7BParameter Scale=6B~7B2024.01 | 30.41 | |
| Mistral-7B: MetaMATHEvaluation Protocol=zero-shot, Parameter Scale=6B~7B2024.01 | 28.25 | |
| Llama3-8B-InstructEvaluation Protocol=zero-shot, Parameter Scale=6B~7B2024.01 | 26.9 |