List your company
ResearchFormerly WizWand

Natural Language Reasoning on DROP

89.62AccuracyInfiGFusion
48.207258.958669.7180.4614Feb 18, 2025Mar 5, 2025Mar 20, 2025Apr 4, 2025Apr 19, 2025May 4, 2025May 20, 2025
Updated 5mo ago

Evaluation results

MethodLinks
InfiGFusionModel Size=14B, GPU Hour=1952025.0589.62
Pivot-SFTModel Size=14B, GPU Hour=1202025.0589.44
InfiFusionModel Size=14B, GPU Hour=1602025.0589.27
FuseChatModel Size=14B, GPU Hour=6502025.0589.23
GPT-4o + QuaSARBackbone=GPT-4o, Prompting Strategy=QuaSAR2025.0288.9
FuseLLMModel Size=14B, GPU Hour=2252025.0588.74
Phi-4Model Size=14B, GPU Hour=∼1.0M2025.0588.67
MiniLogitModel Size=14B, GPU Hour=2202025.0588.56
Llama-3-70B + QuaSARBackbone=Llama-3-70B, Prompting Strategy=QuaSAR2025.0288
Mistral-SmallModel Size=24B, GPU Hour=∼1.6M2025.0586.52
Qwen2.5-InstructModel Size=14B, GPU Hour=∼1.8M2025.0585.56
Qwen2.5-CoderModel Size=14B, GPU Hour=∼1.8M2025.0584.34
GPT-4o + CoTBackbone=GPT-4o, Prompting Strategy=CoT2025.0284.2
GPT-4oBackbone=GPT-4o, Prompting Strategy=Zero-shot2025.0283.4
Llama-3-70BBackbone=Llama-3-70B, Prompting Strategy=Zero-shot2025.0281.4
Llama-3-70B + CoTBackbone=Llama-3-70B, Prompting Strategy=CoT2025.0280.2
Qwen2-72B + QuaSARBackbone=Qwen2-72B, Prompting Strategy=QuaSAR2025.0269
Qwen2-72BBackbone=Qwen2-72B, Prompting Strategy=Zero-shot2025.0266.4
Qwen2-72B + CoTBackbone=Qwen2-72B, Prompting Strategy=CoT2025.0264
QuaSARBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.0263.9
Llama-3-8BStrategy=Base, Protocol=Annotation Tuning2025.0260.2
QuaSARBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.0260
COTBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.0259.3
QuaSARBackbone=Llama-3-8B, Protocol=ICL2025.0258.7
Llama-3-8BStrategy=Base, Protocol=ICL2025.0258.4
COTBackbone=Llama-3-8B, Protocol=ICL2025.0257.9
QuaSARBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.0257.3
QuaSARBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.0257.2
COTBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.0256.8
COTBackbone=Qwen2-7B, Protocol=ICL2025.0256
Qwen2-7BStrategy=Base, Protocol=ICL2025.0255.3
COTBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.0255
QuaSARBackbone=Qwen2-7B, Protocol=ICL2025.0254.6
COTBackbone=Llama-3-1B, Protocol=ICL2025.0254.4
Qwen2-7BStrategy=Base, Protocol=Annotation Tuning2025.0254.2
QuaSARBackbone=Llama-3-1B, Protocol=ICL2025.0254.1
Llama-3-1BStrategy=Base, Protocol=Annotation Tuning2025.0253
Llama-3-1BStrategy=Base, Protocol=ICL2025.0252.5
COTBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.0252.5
COTBackbone=Qwen2-1.5B, Protocol=ICL2025.0251.2
Qwen2-1.5BStrategy=Base, Protocol=Annotation Tuning2025.0250.8
QuaSARBackbone=Qwen2-1.5B, Protocol=ICL2025.0250.8
Qwen2-1.5BStrategy=Base, Protocol=ICL2025.0249.8