Natural Language Reasoning on DROP
89.62AccuracyInfiGFusion
Evaluation Results
| Method | Links | |
|---|---|---|
| InfiGFusionModel Size=14B, GPU Hour=1952025.05 | 89.62 | |
| Pivot-SFTModel Size=14B, GPU Hour=1202025.05 | 89.44 | |
| InfiFusionModel Size=14B, GPU Hour=1602025.05 | 89.27 | |
| FuseChatModel Size=14B, GPU Hour=6502025.05 | 89.23 | |
| GPT-4o + QuaSARBackbone=GPT-4o, Prompting Strategy=QuaSAR2025.02 | 88.9 | |
| FuseLLMModel Size=14B, GPU Hour=2252025.05 | 88.74 | |
| Phi-4Model Size=14B, GPU Hour=∼1.0M2025.05 | 88.67 | |
| MiniLogitModel Size=14B, GPU Hour=2202025.05 | 88.56 | |
| Llama-3-70B + QuaSARBackbone=Llama-3-70B, Prompting Strategy=QuaSAR2025.02 | 88 | |
| Mistral-SmallModel Size=24B, GPU Hour=∼1.6M2025.05 | 86.52 | |
| Qwen2.5-InstructModel Size=14B, GPU Hour=∼1.8M2025.05 | 85.56 | |
| Qwen2.5-CoderModel Size=14B, GPU Hour=∼1.8M2025.05 | 84.34 | |
| GPT-4o + CoTBackbone=GPT-4o, Prompting Strategy=CoT2025.02 | 84.2 | |
| GPT-4oBackbone=GPT-4o, Prompting Strategy=Zero-shot2025.02 | 83.4 | |
| Llama-3-70BBackbone=Llama-3-70B, Prompting Strategy=Zero-shot2025.02 | 81.4 | |
| Llama-3-70B + CoTBackbone=Llama-3-70B, Prompting Strategy=CoT2025.02 | 80.2 | |
| Qwen2-72B + QuaSARBackbone=Qwen2-72B, Prompting Strategy=QuaSAR2025.02 | 69 | |
| Qwen2-72BBackbone=Qwen2-72B, Prompting Strategy=Zero-shot2025.02 | 66.4 | |
| Qwen2-72B + CoTBackbone=Qwen2-72B, Prompting Strategy=CoT2025.02 | 64 | |
| QuaSARBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.02 | 63.9 | |
| Llama-3-8BStrategy=Base, Protocol=Annotation Tuning2025.02 | 60.2 | |
| QuaSARBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.02 | 60 | |
| COTBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.02 | 59.3 | |
| QuaSARBackbone=Llama-3-8B, Protocol=ICL2025.02 | 58.7 | |
| Llama-3-8BStrategy=Base, Protocol=ICL2025.02 | 58.4 | |
| COTBackbone=Llama-3-8B, Protocol=ICL2025.02 | 57.9 | |
| QuaSARBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.02 | 57.3 | |
| QuaSARBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.02 | 57.2 | |
| COTBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.02 | 56.8 | |
| COTBackbone=Qwen2-7B, Protocol=ICL2025.02 | 56 | |
| Qwen2-7BStrategy=Base, Protocol=ICL2025.02 | 55.3 | |
| COTBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.02 | 55 | |
| QuaSARBackbone=Qwen2-7B, Protocol=ICL2025.02 | 54.6 | |
| COTBackbone=Llama-3-1B, Protocol=ICL2025.02 | 54.4 | |
| Qwen2-7BStrategy=Base, Protocol=Annotation Tuning2025.02 | 54.2 | |
| QuaSARBackbone=Llama-3-1B, Protocol=ICL2025.02 | 54.1 | |
| Llama-3-1BStrategy=Base, Protocol=Annotation Tuning2025.02 | 53 | |
| Llama-3-1BStrategy=Base, Protocol=ICL2025.02 | 52.5 | |
| COTBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.02 | 52.5 | |
| COTBackbone=Qwen2-1.5B, Protocol=ICL2025.02 | 51.2 | |
| Qwen2-1.5BStrategy=Base, Protocol=Annotation Tuning2025.02 | 50.8 | |
| QuaSARBackbone=Qwen2-1.5B, Protocol=ICL2025.02 | 50.8 | |
| Qwen2-1.5BStrategy=Base, Protocol=ICL2025.02 | 49.8 |