Dialogue Reasoning on MUTUAL
80.8AccuracyLED
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LEDBackbone=Qwen2.5-7B-Instruct2026.05 | 80.8 | — | |
| SafeDecodingBackbone=Qwen2.5-7B-Instruct2026.05 | 80.2 | — | |
| EVA (text)Backbone=Qwen2.5-7B-Instruct2026.05 | 79.8 | — | |
| VanillaBackbone=Qwen2.5-7B-Instruct2026.05 | 79 | — | |
| LoRABackbone=Qwen2.5-7B-Instruct2026.05 | 79 | — | |
| Circuit BreakersBackbone=Qwen2.5-7B-Instruct2026.05 | 78.8 | — | |
| EVA (text)Backbone=Llama3.1-8B-Instruct2026.05 | 74.4 | — | |
| VanillaBackbone=Llama3.1-8B-Instruct2026.05 | 73.7 | — | |
| LoRABackbone=Llama3.1-8B-Instruct2026.05 | 73.7 | — | |
| DenseBase Model=DeepSeek-7B, Sparsity=Dense2025.06 | 71.3 | — | |
| DenseBase Model=LLaMA-2-7B, Sparsity=Dense2025.06 | 70.84 | — | |
| MaskProBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 68.53 | — | |
| LEDBackbone=Llama3.1-8B-Instruct2026.05 | 68.4 | — | |
| MaskProBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 68.33 | — | |
| SparsegptBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 68.01 | — | |
| Pruner-ZBase Model=LLaMA-2-7B, Sparsity=2:42025.06 | 67.44 | — | |
| SparsegptBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 67.43 | — | |
| Pruner-ZBase Model=DeepSeek-7B, Sparsity=2:42025.06 | 67.24 | — | |
| Circuit BreakersBackbone=Mistral-7B-Instruct-v0.22026.05 | 66.9 | — | |
| LoRABackbone=Mistral-7B-Instruct-v0.22026.05 | 66.8 | — | |
| VanillaBackbone=Mistral-7B-Instruct-v0.22026.05 | 66.4 | — | |
| LEDBackbone=Mistral-7B-Instruct-v0.22026.05 | 66.2 | — | |
| Circuit BreakersBackbone=Llama3.1-8B-Instruct2026.05 | 66 | — | |
| EVA (text)Backbone=Mistral-7B-Instruct-v0.22026.05 | 61.4 | — | |
| VanillaBackbone=Vicuna-7B-v1.52026.05 | 48.3 | — | |
| LoRABackbone=Llama2-7B-chat2026.05 | 48 | — | |
| LEDBackbone=Vicuna-7B-v1.52026.05 | 47.8 | — | |
| EVA (text)Backbone=Vicuna-7B-v1.52026.05 | 47 | — | |
| VanillaBackbone=Llama2-7B-chat2026.05 | 46.5 | — | |
| Circuit BreakersBackbone=Vicuna-7B-v1.52026.05 | 46.3 | — | |
| EVA (text)Backbone=Llama2-7B-chat2026.05 | 46.2 | — | |
| LoRABackbone=Vicuna-7B-v1.52026.05 | 45.9 | — | |
| Circuit BreakersBackbone=Llama2-7B-chat2026.05 | 44.4 | — | |
| SafeDecodingBackbone=Mistral-7B-Instruct-v0.22026.05 | 43.9 | — | |
| LEDBackbone=Llama2-7B-chat2026.05 | 42.5 | — | |
| SafeDecodingBackbone=Llama3.1-8B-Instruct2026.05 | 34 | — | |
| SafeDecodingBackbone=Llama2-7B-chat2026.05 | 32.7 | — | |
| SafeDecodingBackbone=Vicuna-7B-v1.52026.05 | 31.4 | — | |
| AutoMix + PSLM=MISTRAL-7B, Router=POMDP-based2023.10 | — | 0.467 | |
| AutoMix + PSLM=LLAMA2-13B, Router=POMDP-based2023.10 | — | 0.124 | |
| AutoMix + PSLM=GPT-3.5, Router=POMDP-based2023.10 | — | 0.188 | |
| AutoMix + TSLM=MISTRAL-7B, Router=Thresholding2023.10 | — | 0.467 | |
| AutoMix + TSLM=LLAMA2-13B, Router=Thresholding2023.10 | — | 0.118 | |
| AutoMix + TSLM=GPT-3.5, Router=Thresholding2023.10 | — | 0.183 | |
| FrugalGPTSLM=MISTRAL-7B2023.10 | — | 0.203 | |
| FrugalGPTSLM=LLAMA2-13B2023.10 | — | -0.087 | |
| FrugalGPTSLM=GPT-3.52023.10 | — | 0.111 | |
| HybridLLMSLM=MISTRAL-7B2023.10 | — | 0.034 | |
| HybridLLMSLM=LLAMA2-13B2023.10 | — | 0.022 | |
| HybridLLMSLM=GPT-3.52023.10 | — | 0.018 |