Domain Reasoning on MMLU CS
70.1MMLU CS Domain Reasoning AccuracyKARMA
Evaluation results
| Method | Links | |
|---|---|---|
| KARMABackbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=LoRA, Prompting Strategy=zero-shot chain-of-thought, Training Data Source=KARMA-Data, Training Objective=Slot-Parallel Alignment (SPA)2026.07 | 70.1 | |
| Simple SFT (Open-Data)Backbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=LoRA, Prompting Strategy=zero-shot chain-of-thought, Training Data Source=Open-Data, Training Objective=SFT2026.07 | 65.5 | |
| Qwen2.5-7B-InstructBackbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=None, Prompting Strategy=zero-shot chain-of-thought, Training Objective=Pre-trained Instruction Tuned2026.07 | 65.1 | |
| Simple SFT (KARMA-Data)Backbone=Qwen2.5-7B-Instruct, Fine-tuning Strategy=LoRA, Prompting Strategy=zero-shot chain-of-thought, Training Data Source=KARMA-Data, Training Objective=SFT2026.07 | 45.1 |