Reading Comprehension on RACE high
94.5AccuracyOracle
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| OracleType=Single LLM with Routing2026.01 | 94.5 | — | — | |
| Human CeilingType=Human baseline2021.12 | 94.2 | — | — | |
| ALBERT (ensemble)Setting=Supervised2022.01 | 91.4 | — | — | |
| ALBERT (ensemble)Evaluation Protocol=supervised2021.12 | 90.5 | — | — | |
| Qwen-14BModel size=13 ~ 20B, shot=0-shot2024.03 | 90.3 | — | — | |
| Fine-tuned SOTASetting=Fine-tuned2020.05 | 90 | — | — | |
| FlexoraBackbone=Gemma-7B2024.08 | 88.19 | — | — | |
| InternLM2-Chat-20B-SFTParameter Scale=13~20B, Evaluation Setup=0-shot, Training Protocol=SFT2024.03 | 87.3 | — | — | |
| InternLM2-Chat-20BParameter Scale=13~20B, Evaluation Setup=0-shot, Training Protocol=RLHF2024.03 | 87.1 | — | — | |
| FlexoraBackbone=Qwen1.5-7B2024.08 | 87.08 | — | — | |
| FlexoraBackbone=Yi-6B2024.08 | 86.91 | — | — | |
| Gemma-2-9B-itType=Single LLM2026.01 | 85.6 | — | — | |
| Flexora (w/ rsLoRA)Backbone Model=Llama3-8B, LoRA Algorithm=rsLoRA, Flexora Integration=true2024.08 | 85.46 | — | — | |
| LoReFTBackbone Model=Llama3-8B, LoRA Algorithm=LoReFT, Flexora Integration=false2024.08 | 85.33 | — | — | |
| InternLM2-Chat-7B-SFTParameter Scale=< 7B, Evaluation Setup=0-shot, Training Protocol=SFT2024.03 | 85.2 | — | — | |
| LoRABackbone=Yi-6B2024.08 | 85.13 | — | — | |
| InternLM2-Chat-7BParameter Scale=< 7B, Evaluation Setup=0-shot, Training Protocol=RLHF2024.03 | 85.1 | — | — | |
| Flexora (w/ LoReFT)Backbone Model=Llama3-8B, LoRA Algorithm=LoReFT, Flexora Integration=true2024.08 | 84.97 | — | — | |
| ChatGLM3-6B-BaseModel size=< 7B, shot=0-shot2024.03 | 84.8 | — | — | |
| Qwen-14B-ChatParameter Scale=13~20B, Evaluation Setup=0-shot2024.03 | 84.7 | — | — | |
| KALEBackbone=Qwen3-32B-thinking2026.01 | 84.61 | — | — | |
| Flexora (w/ DoRA)Backbone Model=Llama3-8B, LoRA Algorithm=DoRA, Flexora Integration=true2024.08 | 84.45 | — | — | |
| rsLoRABackbone Model=Llama3-8B, LoRA Algorithm=rsLoRA, Flexora Integration=false2024.08 | 84.36 | — | — | |
| FlexoraBackbone=Zephyr-7B-beta2024.08 | 84.3 | — | — | |
| FlexoraBackbone=Qwen2.5-32B2024.08 | 84.27 | — | — | |
| FlexoraBackbone=Mistral-7B-v0.12024.08 | 84.25 | — | — | |
| SMoAType=Multi-LLMs2026.01 | 84 | 2.22 | 11.02 | |
| LoRABackbone=Gemma-7B2024.08 | 83.96 | — | — | |
| Qwen-7BModel size=< 7B, shot=0-shot2024.03 | 83.9 | — | — | |
| MoSLoRABackbone Model=Llama3-8B, LoRA Algorithm=MoSLoRA, Flexora Integration=false2024.08 | 83.75 | — | — | |
| DoRABackbone Model=Llama3-8B, LoRA Algorithm=DoRA, Flexora Integration=false2024.08 | 83.39 | — | — | |
| KALEBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 83.3 | — | — | |
| FlexoraBackbone=Llama2-7B2024.08 | 83.19 | — | — | |
| FlexoraLoRA Rank (r)=162024.08 | 83.03 | — | — | |
| LoRABackbone=Mistral-7B-v0.12024.08 | 82.39 | — | — | |
| Flexora (w/ LoRA)Backbone Model=Llama3-8B, LoRA Algorithm=LoRA, Flexora Integration=true2024.08 | 82.36 | — | — | |
| FlexoraBackbone=Llama3-8B2024.08 | 82.36 | — | — | |
| FlexoraLoRA Rank (r)=82024.08 | 82.36 | — | — | |
| FlexoraFine-tuning Strategy=LoRA, Enhanced=Flexora2024.08 | 82.36 | — | — | |
| Flexora(w/ Base Model)Backbone=Meta-Llama-3-8B, Model variant=Base, Adaptation=Flexora2024.08 | 82.36 | — | — | |
| Chinchillaevaluation_mode=Few-Shot2022.03 | 82.3 | — | — | |
| LoRABackbone=Zephyr-7B-beta2024.08 | 82.25 | — | — | |
| Flexora(w/ Instruct Model)Backbone=Meta-Llama-3-8B-Instruct, Model variant=Instruct, Adaptation=Flexora2024.08 | 82.19 | — | — | |
| Flexora (w/ MoSLoRA)Backbone Model=Llama3-8B, LoRA Algorithm=MoSLoRA, Flexora Integration=true2024.08 | 82.07 | — | — | |
| FlexoraLoRA Rank (r)=322024.08 | 81.97 | — | — | |
| FlexoraBackbone=Vicuna-7B-v1.52024.08 | 81.9 | — | — | |
| KALEBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 81.76 | — | — | |
| Mixtral-8x7B-Instruct-v0.1Parameter Scale=13~20B, Evaluation Setup=0-shot2024.03 | 81.4 | — | — | |
| InternLM2-7BModel size=< 7B, shot=0-shot2024.03 | 81.4 | — | — | |
| LoRABackbone=Qwen1.5-7B2024.08 | 81.36 | — | — | |
| Random (Greedy)Selection=randomly selected layers, LoRA Rank (r)=82024.08 | 81.22 | — | — | |
| LoRABackbone=Qwen2.5-32B2024.08 | 81.03 | — | — | |
| RouteMoAType=Ours2026.01 | 81 | 1.82 | 4.45 | |
| LoRABackbone Model=Llama3-8B, LoRA Algorithm=LoRA, Flexora Integration=false2024.08 | 80.99 | — | — | |
| LoRABackbone=Llama3-8B2024.08 | 80.99 | — | — | |
| LoRALoRA Rank (r)=82024.08 | 80.99 | — | — | |
| LoRALoRA Rank (r)=322024.08 | 80.99 | — | — | |
| LoRAFine-tuning Strategy=LoRA2024.08 | 80.99 | — | — | |
| FlexoraBackbone=Xuan Yuan-6B2024.08 | 80.7 | — | — | |
| Qwen2.5-Coder-7B-InstructType=Single LLM2026.01 | 80.6 | — | — | |
| LoRAPruneRatio=0.5, LoRA Rank (r)=82024.08 | 80.42 | — | — | |
| FlexoraFine-tuning Strategy=Full Fine-Tuning, Enhanced=Flexora2024.08 | 80.37 | — | — | |
| Ministral-8B-Instruct-2410Type=Single LLM2026.01 | 80.3 | — | — | |
| Mixtral-8x7B-v0.1Model size=13 ~ 20B, shot=0-shot2024.03 | 80.2 | — | — | |
| MoAType=Multi-LLMs2026.01 | 80.1 | 8.53 | 13.45 | |
| SFTBackbone=Qwen3-32B-thinking2026.01 | 80.05 | — | — | |
| StructGPTBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 79.87 | — | — | |
| LoRALoRA Rank (r)=162024.08 | 79.68 | — | — | |
| Instruct ModelBackbone=Meta-Llama-3-8B-Instruct, Model variant=Instruct2024.08 | 79.48 | — | — | |
| RouteLLMType=Single LLM with Routing2026.01 | 79.4 | — | — | |
| Full FTFine-tuning protocol=full parameter fine-tuning2024.08 | 79.36 | — | — | |
| Full FTFine-tuning Strategy=Full Fine-Tuning2024.08 | 79.36 | — | — | |
| Engram-40BShots=5-shot2026.01 | 79.2 | — | — | |
| ChatGLM3-6BParameter Scale=< 7B, Evaluation Setup=0-shot2024.03 | 79.1 | — | — | |
| GPT3MixBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 79.07 | — | — | |
| LoRA-FALoRA Rank (r)=82024.08 | 79.03 | — | — | |
| LoRA-SPLoRA Rank (r)=82024.08 | 79.01 | — | — | |
| AugGPTBackbone=Gemma2 9B, Category=Augmented-based2026.01 | 78.88 | — | — | |
| VeRALoRA Rank (r)=82024.08 | 78.84 | — | — | |
| FlexoraBackbone=Llama-7B2024.08 | 78.62 | — | — | |
| KALEBackbone=Orca2 7B, Category=Augmented-based2026.01 | 78.62 | — | — | |
| RouterDCType=Single LLM with Routing2026.01 | 78.3 | — | — | |
| Engram-27BShots=5-shot2026.01 | 78.2 | — | — | |
| AdaLoRAro=4, LoRA Rank (r)=82024.08 | 77.93 | — | — | |
| DenseLLM=Baichuan2-13B, Ratio=0.00%2024.03 | 77.89 | — | — | |
| VanillaBackbone=Qwen3-32B-thinking2026.01 | 77.47 | — | — | |
| Bio-Medical-Llama-3-8BType=Single LLM2026.01 | 77.3 | — | — | |
| AugGPTBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 77.16 | — | — | |
| KG-SFTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 77.16 | — | — | |
| DMTBackbone=Gemma2 9B, Category=SFT-based2026.01 | 76.99 | — | — | |
| GraphRAGBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 76.8 | — | — | |
| FlexoraBackbone=Chatglm3-6B2024.08 | 76.33 | — | — | |
| STaRBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 76.3 | — | — | |
| GPT3MixBackbone=Qwen2.5 32B, Category=Augmented-based2026.01 | 76.21 | — | — | |
| GraphRAGBackbone=Orca2 7B, Category=Retrieval-based2026.01 | 76.02 | — | — | |
| StructGPTBackbone=Orca2 7B, Category=Retrieval-based2026.01 | 76.01 | — | — | |
| TOGBackbone=Gemma2 9B, Category=Retrieval-based2026.01 | 75.99 | — | — | |
| GraphRAGBackbone=Qwen2.5 32B, Category=Retrieval-based2026.01 | 75.84 | — | — | |
| DMTBackbone=Qwen2.5 32B, Category=SFT-based2026.01 | 75.72 | — | — | |
| Baichuan2-13B-BaseModel size=13 ~ 20B, shot=0-shot2024.03 | 75.6 | — | — |