Dialogue on MT-Bench
9.3MT-Bench ScoreGPT-4o
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-4oEvaluation Protocol=Closed-Source API, Emission (gCO2/q)=4.52†, Throughput (Tok/s)=55.22026.03 | 9.3 | |
| Claude 3.5 SonnetEvaluation Protocol=Closed-Source API, Emission (gCO2/q)=3.85†, Throughput (Tok/s)=62.12026.03 | 9.1 | |
| Gemini 2.5 ProEvaluation Protocol=Closed-Source API, Emission (gCO2/q)=3.90†, Throughput (Tok/s)=58.42026.03 | 9 | |
| EcoThinkEvaluation Protocol=Adaptive Inference, Emission (gCO2/q)=1.32, Throughput (Tok/s)=148.62026.03 | 8.9 | |
| Qwen-3-8B-InstructEvaluation Protocol=Standard CoT, Emission (gCO2/q)=2.12, Throughput (Tok/s)=98.52026.03 | 8.6 | |
| FrugalGPT (Cascade)Evaluation Protocol=Standard CoT, Emission (gCO2/q)=1.95, Throughput (Tok/s)=88.52026.03 | 8.5 | |
| Llama-3.1-8B-InstructEvaluation Protocol=Standard CoT, Emission (gCO2/q)=2.15, Throughput (Tok/s)=95.82026.03 | 8.2 | |
| SPARDBackbone=Qwen2.5-7B-Instruct2026.04 | 8.138 | |
| + GRPOavgBackbone=Qwen2.5-7B-Instruct2026.04 | 7.975 | |
| + GRPOimpBackbone=Qwen2.5-7B-Instruct2026.04 | 7.862 | |
| + DPOBackbone=Qwen2.5-7B-Instruct2026.04 | 7.837 | |
| SPARDBackbone=Qwen3-8B2026.04 | 7.831 | |
| + GRPOrmBackbone=Qwen2.5-7B-Instruct2026.04 | 7.812 | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.04 | 7.793 | |
| + DPOBackbone=Qwen3-8B2026.04 | 7.762 | |
| + SFTBackbone=Qwen2.5-7B-Instruct2026.04 | 7.756 | |
| + GRPOimpBackbone=Qwen3-8B2026.04 | 7.725 | |
| + GRPOrmBackbone=Qwen3-8B2026.04 | 7.681 | |
| + GRPOavgBackbone=Qwen3-8B2026.04 | 7.681 | |
| BaseBackbone=Qwen3-8B2026.04 | 7.506 | |
| + SFTBackbone=Qwen3-8B2026.04 | 7.162 | |
| NA-LoRABackbone=Llama-3.1-8B-Base, Rank (r)=82026.06 | 6.39 | |
| DoRABackbone=Llama-3.1-8B-Base2026.06 | 6.32 | |
| GoRABackbone=Llama-3.1-8B-Base2026.06 | 6.3 | |
| LoRA+Backbone=Llama-3.1-8B-Base2026.06 | 6.27 | |
| Bi-LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 6.26 | |
| LoRA-ProBackbone=Llama-3.1-8B-Base2026.06 | 6.21 | |
| NA-LoRABackbone=Llama-3.1-8B-Base, Rank (r)=322026.06 | 6.2 | |
| AdaLoRABackbone=Llama-3.1-8B-Base2026.06 | 6.18 | |
| Full FTCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 6.16 | |
| rsLoRABackbone=Llama-3.1-8B-Base2026.06 | 6.16 | |
| NA-LoRABackbone=Llama-3.1-8B-Base, Rank (r)=1282026.06 | 6.12 | |
| LoRA-nBARCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 6.1 | |
| PiSSABackbone=Llama-3.1-8B-Base2026.06 | 6.1 | |
| LoRABackbone=Llama-3.1-8B-Base2026.06 | 6.08 | |
| ROPCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 6.05 | |
| GOATBackbone=Llama-2-7B2025.02 | 6.01 | |
| RWP_fullCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 6.01 | |
| LoRA-GABackbone=Llama-3.1-8B-Base2026.06 | 6 | |
| Flat-LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 5.98 | |
| DoRABackbone=Llama-2-7B2025.02 | 5.97 | |
| LoRA-SAMCost=×2, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 5.97 | |
| LoRA-oBARCost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 5.97 | |
| DoRABackbone=Llama-2-7B-Base2026.06 | 5.97 | |
| LoRA-GABackbone=Llama-2-7B-Base2026.06 | 5.95 | |
| LoRA-LookSAMCost=×1.x, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 5.94 | |
| LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 5.92 | |
| NA-LoRABackbone=Llama-2-7B-Base, Rank (r)=82026.06 | 5.92 | |
| FullBackbone=Llama-3.1-8B-Base2026.06 | 5.88 | |
| MoLoRABackbone=Llama-2-7B2025.02 | 5.84 | |
| HydraLoRABackbone=Llama-2-7B2025.02 | 5.82 | |
| RWP_LoRACost=×1, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 5.8 | |
| LoRA-ESAMCost=×1.x, Backbone=Llama 2-7B, Evaluation Protocol=Fine-tuning2025.08 | 5.8 | |
| NA-LoRABackbone=Llama-2-7B-Base, Rank (r)=1282026.06 | 5.78 | |
| LoRA-ProBackbone=Llama-2-7B-Base2026.06 | 5.72 | |
| NA-LoRABackbone=Llama-2-7B-Base, Rank (r)=322026.06 | 5.72 | |
| LoRA+Backbone=Llama-2-7B-Base2026.06 | 5.71 | |
| LoRABackbone=Llama-2-7B2025.02 | 5.61 | |
| LoRABackbone=Llama-2-7B-Base2026.06 | 5.61 | |
| GoRABackbone=Llama-2-7B-Base2026.06 | 5.61 | |
| AdaLoRABackbone=Llama-2-7B-Base2026.06 | 5.57 | |
| Full FTBackbone=Llama-2-7B2025.02 | 5.56 | |
| PiSSABackbone=Llama-2-7B2025.02 | 5.3 | |
| FullBackbone=Llama-2-7B-Base2026.06 | 5.3 | |
| PiSSABackbone=Llama-2-7B-Base2026.06 | 5.3 | |
| rsLoRABackbone=Llama-2-7B-Base2026.06 | 5.25 | |
| MiLoRABackbone=Llama-2-7B2025.02 | 5.23 |