List your company
ResearchFormerly WizWand

Reading Comprehension on RACE high

94.5AccuracyOracle
74.84479.94785.0590.153May 28, 2020May 7, 2021Apr 17, 2022Mar 28, 2023Mar 7, 2024Feb 15, 2025Jan 26, 2026
Updated 8mo ago

Evaluation results

MethodLinks
OracleType=Single LLM with Routing2026.0194.5——
Human CeilingType=Human baseline2021.1294.2——
ALBERT (ensemble)Setting=Supervised2022.0191.4——
ALBERT (ensemble)Evaluation Protocol=supervised2021.1290.5——
Qwen-14BModel size=13 ~ 20B, shot=0-shot2024.0390.3——
Fine-tuned SOTASetting=Fine-tuned2020.0590——
FlexoraBackbone=Gemma-7B2024.0888.19——
InternLM2-Chat-20B-SFTParameter Scale=13~20B, Evaluation Setup=0-shot, Training Protocol=SFT2024.0387.3——
InternLM2-Chat-20BParameter Scale=13~20B, Evaluation Setup=0-shot, Training Protocol=RLHF2024.0387.1——
FlexoraBackbone=Qwen1.5-7B2024.0887.08——
FlexoraBackbone=Yi-6B2024.0886.91——
Gemma-2-9B-itType=Single LLM2026.0185.6——
Flexora (w/ rsLoRA)Backbone Model=Llama3-8B, LoRA Algorithm=rsLoRA, Flexora Integration=true2024.0885.46——
LoReFTBackbone Model=Llama3-8B, LoRA Algorithm=LoReFT, Flexora Integration=false2024.0885.33——
InternLM2-Chat-7B-SFTParameter Scale=< 7B, Evaluation Setup=0-shot, Training Protocol=SFT2024.0385.2——
LoRABackbone=Yi-6B2024.0885.13——
InternLM2-Chat-7BParameter Scale=< 7B, Evaluation Setup=0-shot, Training Protocol=RLHF2024.0385.1——
Flexora (w/ LoReFT)Backbone Model=Llama3-8B, LoRA Algorithm=LoReFT, Flexora Integration=true2024.0884.97——
ChatGLM3-6B-BaseModel size=< 7B, shot=0-shot2024.0384.8——
Qwen-14B-ChatParameter Scale=13~20B, Evaluation Setup=0-shot2024.0384.7——
KALEBackbone=Qwen3-32B-thinking2026.0184.61——
Flexora (w/ DoRA)Backbone Model=Llama3-8B, LoRA Algorithm=DoRA, Flexora Integration=true2024.0884.45——
rsLoRABackbone Model=Llama3-8B, LoRA Algorithm=rsLoRA, Flexora Integration=false2024.0884.36——
FlexoraBackbone=Zephyr-7B-beta2024.0884.3——
FlexoraBackbone=Qwen2.5-32B2024.0884.27——
FlexoraBackbone=Mistral-7B-v0.12024.0884.25——
SMoAType=Multi-LLMs2026.01842.2211.02
LoRABackbone=Gemma-7B2024.0883.96——
Qwen-7BModel size=< 7B, shot=0-shot2024.0383.9——
MoSLoRABackbone Model=Llama3-8B, LoRA Algorithm=MoSLoRA, Flexora Integration=false2024.0883.75——
DoRABackbone Model=Llama3-8B, LoRA Algorithm=DoRA, Flexora Integration=false2024.0883.39——
KALEBackbone=Gemma2 9B, Category=Augmented-based2026.0183.3——
FlexoraBackbone=Llama2-7B2024.0883.19——
FlexoraLoRA Rank (r)=162024.0883.03——
LoRABackbone=Mistral-7B-v0.12024.0882.39——
Flexora (w/ LoRA)Backbone Model=Llama3-8B, LoRA Algorithm=LoRA, Flexora Integration=true2024.0882.36——
FlexoraBackbone=Llama3-8B2024.0882.36——
FlexoraLoRA Rank (r)=82024.0882.36——
FlexoraFine-tuning Strategy=LoRA, Enhanced=Flexora2024.0882.36——
Flexora(w/ Base Model)Backbone=Meta-Llama-3-8B, Model variant=Base, Adaptation=Flexora2024.0882.36——
Chinchillaevaluation_mode=Few-Shot2022.0382.3——
LoRABackbone=Zephyr-7B-beta2024.0882.25——
Flexora(w/ Instruct Model)Backbone=Meta-Llama-3-8B-Instruct, Model variant=Instruct, Adaptation=Flexora2024.0882.19——
Flexora (w/ MoSLoRA)Backbone Model=Llama3-8B, LoRA Algorithm=MoSLoRA, Flexora Integration=true2024.0882.07——
FlexoraLoRA Rank (r)=322024.0881.97——
FlexoraBackbone=Vicuna-7B-v1.52024.0881.9——
KALEBackbone=Qwen2.5 32B, Category=Augmented-based2026.0181.76——
Mixtral-8x7B-Instruct-v0.1Parameter Scale=13~20B, Evaluation Setup=0-shot2024.0381.4——
InternLM2-7BModel size=< 7B, shot=0-shot2024.0381.4——
LoRABackbone=Qwen1.5-7B2024.0881.36——
Random (Greedy)Selection=randomly selected layers, LoRA Rank (r)=82024.0881.22——
LoRABackbone=Qwen2.5-32B2024.0881.03——
RouteMoAType=Ours2026.01811.824.45
LoRABackbone Model=Llama3-8B, LoRA Algorithm=LoRA, Flexora Integration=false2024.0880.99——
LoRABackbone=Llama3-8B2024.0880.99——
LoRALoRA Rank (r)=82024.0880.99——
LoRALoRA Rank (r)=322024.0880.99——
LoRAFine-tuning Strategy=LoRA2024.0880.99——
FlexoraBackbone=Xuan Yuan-6B2024.0880.7——
Qwen2.5-Coder-7B-InstructType=Single LLM2026.0180.6——
LoRAPruneRatio=0.5, LoRA Rank (r)=82024.0880.42——
FlexoraFine-tuning Strategy=Full Fine-Tuning, Enhanced=Flexora2024.0880.37——
Ministral-8B-Instruct-2410Type=Single LLM2026.0180.3——
Mixtral-8x7B-v0.1Model size=13 ~ 20B, shot=0-shot2024.0380.2——
MoAType=Multi-LLMs2026.0180.18.5313.45
SFTBackbone=Qwen3-32B-thinking2026.0180.05——
StructGPTBackbone=Gemma2 9B, Category=Retrieval-based2026.0179.87——
LoRALoRA Rank (r)=162024.0879.68——
Instruct ModelBackbone=Meta-Llama-3-8B-Instruct, Model variant=Instruct2024.0879.48——
RouteLLMType=Single LLM with Routing2026.0179.4——
Full FTFine-tuning protocol=full parameter fine-tuning2024.0879.36——
Full FTFine-tuning Strategy=Full Fine-Tuning2024.0879.36——
Engram-40BShots=5-shot2026.0179.2——
ChatGLM3-6BParameter Scale=< 7B, Evaluation Setup=0-shot2024.0379.1——
GPT3MixBackbone=Gemma2 9B, Category=Augmented-based2026.0179.07——
LoRA-FALoRA Rank (r)=82024.0879.03——
LoRA-SPLoRA Rank (r)=82024.0879.01——
AugGPTBackbone=Gemma2 9B, Category=Augmented-based2026.0178.88——
VeRALoRA Rank (r)=82024.0878.84——
FlexoraBackbone=Llama-7B2024.0878.62——
KALEBackbone=Orca2 7B, Category=Augmented-based2026.0178.62——
RouterDCType=Single LLM with Routing2026.0178.3——
Engram-27BShots=5-shot2026.0178.2——
AdaLoRAro=4, LoRA Rank (r)=82024.0877.93——
DenseLLM=Baichuan2-13B, Ratio=0.00%2024.0377.89——
VanillaBackbone=Qwen3-32B-thinking2026.0177.47——
Bio-Medical-Llama-3-8BType=Single LLM2026.0177.3——
AugGPTBackbone=Qwen2.5 32B, Category=Augmented-based2026.0177.16——
KG-SFTBackbone=Gemma2 9B, Category=SFT-based2026.0177.16——
DMTBackbone=Gemma2 9B, Category=SFT-based2026.0176.99——
GraphRAGBackbone=Gemma2 9B, Category=Retrieval-based2026.0176.8——
FlexoraBackbone=Chatglm3-6B2024.0876.33——
STaRBackbone=Qwen2.5 32B, Category=Augmented-based2026.0176.3——
GPT3MixBackbone=Qwen2.5 32B, Category=Augmented-based2026.0176.21——
GraphRAGBackbone=Orca2 7B, Category=Retrieval-based2026.0176.02——
StructGPTBackbone=Orca2 7B, Category=Retrieval-based2026.0176.01——
TOGBackbone=Gemma2 9B, Category=Retrieval-based2026.0175.99——
GraphRAGBackbone=Qwen2.5 32B, Category=Retrieval-based2026.0175.84——
DMTBackbone=Qwen2.5 32B, Category=SFT-based2026.0175.72——
Baichuan2-13B-BaseModel size=13 ~ 20B, shot=0-shot2024.0375.6——