Exact Match Performance on Big-Bench Hard (BBH)
77.3Boolean Expressions EMBase
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| BaseBackbone=Qwen-2.5-7B2025.11 | 77.3 | — | 59.8 | 53.3 | 50.7 | 41.3 | 22.7 | 5.3 | 0 | 38.7 | |
| LoRAHubBackbone=Qwen-2.5-7B2025.11 | 77.3 | — | 51.5 | 53.2 | 52 | 35.2 | 73.1 | 41.6 | 1.9 | 48.1 | |
| LoRA RetrieverBackbone=LLaMA-3.1-8B2025.11 | 76.7 | — | 54 | 49.3 | 56.7 | 20.7 | 0 | 32 | 34 | 40.4 | |
| LOGO (Entropy)Backbone=LLaMA-3.1-8B2025.11 | 76.7 | — | 47.1 | 50 | 50 | 24 | 3.3 | 26.7 | 42 | 40 | |
| Adapter SoupBackbone=Qwen-2.5-7B2025.11 | 76.7 | — | 59.8 | 55.3 | 50.7 | 44 | 76.7 | 49.3 | 4.7 | 52.2 | |
| LOGO (Norm)Backbone=Qwen-2.5-7B2025.11 | 74.7 | — | 62.1 | 54.7 | 50.7 | 44 | 80 | 49.3 | 12 | 53.3 | |
| LoRA RetrieverBackbone=Qwen-2.5-7B2025.11 | 74 | — | 59.8 | 56.7 | 50.7 | 40.7 | 76.7 | 54 | 12.7 | 53.6 | |
| Adapter SoupBackbone=LLaMA-3.1-8B2025.11 | 72.7 | — | 48.3 | 52.7 | 54.7 | 38 | 8.7 | 30.7 | 34.7 | 42.5 | |
| LOGO (Norm)Backbone=LLaMA-3.1-8B2025.11 | 71.3 | — | 48.3 | 52 | 50 | 27.3 | 1.3 | 14.7 | 41.3 | 38.3 | |
| Adapter SoupBackbone=DeepSeek-LLM-7B-Base2025.11 | 68 | — | 49.4 | 48.7 | 52 | 46.7 | 0 | 0 | 6 | 33.8 | |
| LOGO (Norm)Backbone=DeepSeek-LLM-7B-Base2025.11 | 68 | — | 48.3 | 48.7 | 51.3 | 44 | 0 | 0 | 5.3 | 33.2 | |
| LoRA RetrieverBackbone=DeepSeek-LLM-7B-Base2025.11 | 66.7 | — | 48.3 | 48.7 | 54 | 44 | 0 | 0 | 7.3 | 33.6 | |
| LOGO (Entropy)Backbone=Qwen-2.5-7B2025.11 | 66 | — | 59.8 | 53.3 | 50.7 | 43.3 | 78.7 | 51.3 | 9.3 | 52.6 | |
| BaseBackbone=DeepSeek-LLM-7B-Base2025.11 | 66 | — | 27.6 | 0.7 | 46 | 46 | 0 | 0 | 3.3 | 23.7 | |
| LOGO (Entropy)Backbone=DeepSeek-LLM-7B-Base2025.11 | 66 | — | 14.9 | 0 | 53.3 | 38.7 | 0.7 | 0 | 1.3 | 21.9 | |
| LoRAHubBackbone=LLaMA-3.1-8B2025.11 | 64.5 | — | 54 | 42.7 | 48.5 | 39.1 | 8.9 | 21.6 | 16.7 | 37 | |
| BaseBackbone=LLaMA-3.1-8B2025.11 | 58 | — | 41.4 | 25.3 | 45.3 | 35.3 | 0 | 2 | 11.3 | 27.3 | |
| LoRAHubBackbone=DeepSeek-LLM-7B-Base2025.11 | 55.9 | — | 43.4 | 31.7 | 51.7 | 40.9 | 26.8 | 13.9 | 4.4 | 33.6 |