Domain Reasoning on MMLU-Pro Chemistry
19.5AccuracyKARMA
Evaluation results
| Method | Links | |
|---|---|---|
| KARMABackbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=LoRA, Prompting Strategy=zero-shot chain-of-thought, Training Data Source=KARMA-Data, Training Objective=Slot-Parallel Alignment (SPA)2026.07 | 19.5 | |
| Simple SFT (Open-Data)Backbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=LoRA, Prompting Strategy=zero-shot chain-of-thought, Training Data Source=Open-Data, Training Objective=SFT2026.07 | 17.8 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=None, Prompting Strategy=zero-shot chain-of-thought, Training Objective=Pre-trained Instruction Tuned2026.07 | 14.8 | |
| Simple SFT (KARMA-Data)Backbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=LoRA, Prompting Strategy=zero-shot chain-of-thought, Training Data Source=KARMA-Data, Training Objective=SFT2026.07 | 12.6 |