List your company
ResearchFormerly WizWand

Symbolic Reasoning on OlyBench

36.5AccuracyQuaSAR
5.313.421.529.6Feb 18, 2025
Updated 8mo ago

Evaluation results

MethodLinks
QuaSARBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.0236.5
COMATBackbone=Qwen2-7B, Protocol=ICL2025.0232.2
QuaSARBackbone=Qwen2-7B, Protocol=ICL2025.0227.8
QuaSARBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.0216.6
COTBackbone=Llama-3-8B, Protocol=Annotation Tuning2025.0214.7
QuaSARBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.0214.6
Llama-3-8BStrategy=Base, Protocol=Annotation Tuning2025.0213.2
QuaSARBackbone=Llama-3-8B, Protocol=ICL2025.0213
COTBackbone=Llama-3-8B, Protocol=ICL2025.0212.4
QuaSARBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.0211.8
Llama-3-8BStrategy=Base, Protocol=ICL2025.0210.9
Qwen2-7BStrategy=Base, Protocol=Annotation Tuning2025.0210.9
COTBackbone=Qwen2-7B, Protocol=Annotation Tuning2025.0210.7
COTBackbone=Llama-3-1B, Protocol=Annotation Tuning2025.0210.6
Qwen2-7BStrategy=Base, Protocol=ICL2025.0210.5
QuaSARBackbone=Llama-3-1B, Protocol=ICL2025.029.8
COTBackbone=Qwen2-7B, Protocol=ICL2025.029.8
COTBackbone=Qwen2-1.5B, Protocol=Annotation Tuning2025.029.1
COTBackbone=Llama-3-1B, Protocol=ICL2025.028.2
QuaSARBackbone=Qwen2-1.5B, Protocol=ICL2025.028.2
COTBackbone=Qwen2-1.5B, Protocol=ICL2025.027.8
Qwen2-1.5BStrategy=Base, Protocol=Annotation Tuning2025.027.4
Llama-3-1BStrategy=Base, Protocol=Annotation Tuning2025.027.1
Qwen2-1.5BStrategy=Base, Protocol=ICL2025.026.9
Llama-3-1BStrategy=Base, Protocol=ICL2025.026.5