List your company
ResearchFormerly WizWand

Symbolic Reasoning on MMLU Redux

80.3AccuracyQuaSAR
26.2240.2654.368.34Feb 18, 2025
Updated 8mo ago

Evaluation results

MethodLinks
QuaSARBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.0280.3
COTBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.0280
COMATBackbone=Qwen2-7B, Protocol=ICL2025.0279.8
COTBackbone=Qwen2-7B, Protocol=ICL2025.0279.2
QuaSARBackbone=Qwen2-7B, Protocol=ICL2025.0275.9
QuaSARBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.0267.2
Qwen2-7BStrategy=Base, Protocol=Annotation Tuning2025.0266.3
COTBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.0265.9
Qwen2-7BStrategy=Base, Protocol=ICL2025.0265.5
COTBackbone=Llama-3-8B, Protocol=ICL2025.0264.5
Llama-3-8BStrategy=Base, Protocol=Annotation Tuning2025.0263
QuaSARBackbone=Llama-3-8B, Protocol=ICL2025.0263
Llama-3-8BStrategy=Base, Protocol=ICL2025.0260.2
QuaSARBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.0249.5
COTBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.0247.8
COTBackbone=Qwen2-1.5B, Protocol=ICL2025.0246.3
QuaSARBackbone=Qwen2-1.5B, Protocol=ICL2025.0244.8
Qwen2-1.5BStrategy=Base, Protocol=Annotation Tuning2025.0242.4
Qwen2-1.5BStrategy=Base, Protocol=ICL2025.0241.7
QuaSARBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.0240
COTBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.0234.7
QuaSARBackbone=Llama-3-1B, Protocol=ICL2025.0234.2
COTBackbone=Llama-3-1B, Protocol=ICL2025.0234
Llama-3-1BStrategy=Base, Protocol=Annotation Tuning2025.0230.1
Llama-3-1BStrategy=Base, Protocol=ICL2025.0228.3