Abstract Reasoning on AbsR
83.41AbsR AccuracyKALE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| KALEBackbone=Orca2 7B, Category=Augmented-based2026.01 | 83.41 | — | |
| KALEBackbone=OLMOE 7B, Category=Augmented-based2026.01 | 81.99 | — | |
| KALEBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 81.52 | — | |
| KG-SFTBackbone=Orca2 7B, Category=SFT-based2026.01 | 78.91 | — | |
| DMTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 77.13 | — | |
| GPT-3.5Size=>20B2024.03 | 77.08 | 84 | |
| MeanLearnBackbone=Orca2 7B, Category=SFT-based2026.01 | 77.01 | — | |
| STaRBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 76.66 | — | |
| SDFTBackbone=Orca2 7B, Category=SFT-based2026.01 | 76.66 | — | |
| SDFTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 75.83 | — | |
| DMTBackbone=Orca2 7B, Category=SFT-based2026.01 | 75.24 | — | |
| KG-SFTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 74.76 | — | |
| MeanLearnSize=8B2024.03 | 73.96 | 84.5 | |
| LLaMA-3Size=8B2024.03 | 72.92 | 82.5 | |
| TOGBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 72.04 | — | |
| MeanLearnBackbone=Gemma2 9B, Category=SFT-based2026.01 | 72.04 | — | |
| STaRBackbone=Orca2 7B, Category=Augmented-based2026.01 | 71.68 | — | |
| MeanLearnBackbone=OLMOE 7B, Category=SFT-based2026.01 | 71.09 | — | |
| GPT3MixBackbone=Orca2 7B, Category=Augmented-based2026.01 | 69.79 | — | |
| CoTBackbone=Orca2 7B, Category=Prompt-based2026.01 | 67.77 | — | |
| CoTBackbone=Gemma2 9B, Category=Prompt-based2026.01 | 67.54 | — | |
| GraphRAGBackbone=Orca2 7B, Category=Retrieval-based2026.01 | 67.06 | — | |
| StructGPTBackbone=Orca2 7B, Category=Retrieval-based2026.01 | 65.17 | — | |
| PaLM-2Size=>20B2024.03 | 64.58 | 75.76 | |
| MeanLearnSize=7B2024.03 | 64.58 | 77 | |
| GraphRAGBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 64.57 | — | |
| SFTBackbone=Orca2 7B, Category=SFT-based2026.01 | 63.98 | — | |
| GPT3MixBackbone=OLMOE 7B, Category=Augmented-based2026.01 | 62.2 | — | |
| AugGPTBackbone=OLMOE 7B, Category=Augmented-based2026.01 | 61.73 | — | |
| AugGPTBackbone=Orca2 7B, Category=Augmented-based2026.01 | 61.73 | — | |
| LLaMA-2Size=13B2024.03 | 61.46 | 73.5 | |
| VicunaSize=13B2024.03 | 61.46 | 74 | |
| SFTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 61.37 | — | |
| VanillaBackbone=Orca2 7B, Category=Prompt-based2026.01 | 61.37 | — | |
| KG-SFTBackbone=OLMOE 7B, Category=SFT-based2026.01 | 61.26 | — | |
| DMTBackbone=OLMOE 7B, Category=SFT-based2026.01 | 60.43 | — | |
| Orca-2Size=7B2024.03 | 60.42 | 73.5 | |
| SDFTBackbone=OLMOE 7B, Category=SFT-based2026.01 | 59.95 | — | |
| GPT3MixBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 59.72 | — | |
| AugGPTBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 59.6 | — | |
| TOGBackbone=Orca2 7B, Category=Retrieval-based2026.01 | 59.6 | — | |
| WizardLMSize=13B2024.03 | 59.38 | 74 | |
| StructGPTBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 59.24 | — | |
| STaRBackbone=OLMOE 7B, Category=Augmented-based2026.01 | 59.24 | — | |
| VicunaSize=7B2024.03 | 58.33 | 75 | |
| GraphRAGBackbone=OLMOE 7B, Category=Retrieval-based2026.01 | 57.82 | — | |
| StructGPTBackbone=OLMOE 7B, Category=Retrieval-based2026.01 | 56.87 | — | |
| TOGBackbone=OLMOE 7B, Category=Retrieval-based2026.01 | 54.5 | — | |
| SFTBackbone=OLMOE 7B, Category=SFT-based2026.01 | 53.31 | — | |
| WizardLMSize=7B2024.03 | 53.24 | 65 | |
| VanillaBackbone=Gemma2 9B, Category=Prompt-based2026.01 | 52.49 | — | |
| CoTBackbone=OLMOE 7B, Category=Prompt-based2026.01 | 51.06 | — | |
| VanillaBackbone=OLMOE 7B, Category=Prompt-based2026.01 | 49.88 | — | |
| MeanLearnSize=13B2024.03 | 45.82 | 67.17 | |
| LLaMA-2Size=7B2024.03 | 35.42 | 50 | |
| Orca-2Size=13B2024.03 | 29.5 | 66.5 | |
| COT-DistillBackbone=Llama3 8B2026.01 | — | 80.64 | |
| Distill-SBSBackbone=Llama3 8B2026.01 | — | 77.61 | |
| KALEBackbone=Llama3 8B2026.01 | — | 83.62 | |
| KALEBackbone=Qwen3-32B-thinking2026.01 | — | 93.6 | |
| PDBackbone=Llama3 8B2026.01 | — | 75.31 | |
| SFTBackbone=Qwen3-32B-thinking2026.01 | — | 85.82 | |
| VanillaBackbone=Qwen3-32B-thinking2026.01 | — | 82.33 |