Discrete Reasoning on DROP
71.59Exact Match (EM)GPT-4
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GPT-4Setting=Zero-shot2023.11 | 71.59 | — | |
| Orca 2-13BSetting=Zero-shot, System Message=Cautious2023.11 | 70.88 | — | |
| Orca 2-7BSetting=Zero-shot, System Message=Cautious2023.11 | 69.09 | — | |
| ChatGPTSetting=Zero-shot2023.11 | 64.39 | — | |
| Orca 2-7BSetting=Zero-shot, System Message=Default2023.11 | 60.26 | — | |
| WizardLM-70BSetting=Zero-shot2023.11 | 59.62 | — | |
| Orca 2-13BSetting=Zero-shot, System Message=Default2023.11 | 57.97 | — | |
| LLaMA-2-Chat-70BSetting=Zero-shot2023.11 | 54.11 | — | |
| Orca-1-13BSetting=Zero-shot2023.11 | 53.63 | — | |
| WizardLM-13BSetting=Zero-shot2023.11 | 45.97 | — | |
| LLaMA-2-Chat-13BSetting=Zero-shot2023.11 | 40.73 | — | |
| FedSDR2026.05 | 37.94 | — | |
| FedSD2026.05 | 37.9 | — | |
| Dual Upload2026.05 | 37.69 | — | |
| Dual-LoRA(FedDPA)2026.05 | 37.19 | — | |
| Distill Rect2026.05 | 36.57 | — | |
| Baseline(FedAvg)2026.05 | 36.48 | — | |
| MMKEBackbone=Qwen2.5-7B Instruct, Consolidation=false2025.12 | 10.3 | 24.3 | |
| MMKE+Con.Backbone=Qwen2.5-7B Instruct, Consolidation=true2025.12 | 8.5 | 21.6 | |
| FT-M+Con.Backbone=Qwen2.5-7B Instruct, Consolidation=true2025.12 | 2.8 | 14.6 | |
| EtConBackbone=Qwen2.5-7B Instruct2025.12 | 2.5 | 8.6 | |
| FT-MBackbone=Qwen2.5-7B Instruct, Consolidation=false2025.12 | 2.4 | 13.3 | |
| PreBackbone=Qwen2.5-7B Instruct2025.12 | 2.2 | 9.9 | |
| ALPHABackbone=Qwen2.5-7B Instruct, Consolidation=false2025.12 | 0 | 0 | |
| ALPHA+Con.Backbone=Qwen2.5-7B Instruct, Consolidation=true2025.12 | 0 | 0 |