General Multi-Choice Question Answering on MMLU-Pro
49.4Science AccuracyAutoMR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AutoMRBackbone=Qwen, Skeleton Structure=DAG2025.10 | 49.4 | 33.7 | 57.4 | 45.6 | |
| AutoMRBackbone=LLaMA, Skeleton Structure=DAG2025.10 | 48.9 | 33.2 | 51 | 38.8 | |
| MaASBackbone=Qwen, Skeleton Structure=Sequential2025.10 | 45.5 | 31 | 56 | 41.7 | |
| Meta-ReasonerBackbone=Qwen, Skeleton Structure=Sequential2025.10 | 45.4 | 31.9 | 55 | 42.2 | |
| MaASBackbone=LLaMA, Skeleton Structure=Sequential2025.10 | 44.6 | 29.7 | 46.2 | 35.6 | |
| Meta-ReasonerBackbone=LLaMA, Skeleton Structure=Sequential2025.10 | 44.3 | 30.6 | 47.2 | 36.4 | |
| rStarBackbone=Qwen, Skeleton Structure=Tree2025.10 | 43.6 | 30.8 | 55.4 | 36 | |
| MRPBackbone=Qwen, Skeleton Structure=-2025.10 | 42.8 | 30.1 | 53.5 | 41.6 | |
| rStarBackbone=LLaMA, Skeleton Structure=Tree2025.10 | 42.6 | 30 | 46.8 | 34.8 | |
| CoTBackbone=Qwen, Skeleton Structure=Sequential2025.10 | 41.6 | 28.3 | 51.5 | 39.8 | |
| MRPBackbone=LLaMA, Skeleton Structure=-2025.10 | 36.4 | 24.2 | 40.6 | 32.8 | |
| Direct-I/OBackbone=Qwen, Skeleton Structure=-2025.10 | 32.7 | 25.1 | 39 | 29.1 | |
| CoTBackbone=LLaMA, Skeleton Structure=Sequential2025.10 | 31.5 | 22.4 | 37.3 | 31.3 | |
| Direct-I/OBackbone=LLaMA, Skeleton Structure=-2025.10 | 16.3 | 11.5 | 15.8 | 14.5 |