List your company
ResearchFormerly WizWand

STEM Task Evaluation on MMLU Physics

60.78AccuracyOne-shot CoT
29.174437.379745.58553.7903Aug 13, 2024
Updated 8mo ago

Evaluation results

MethodLinks
One-shot CoTModel=Llama3-8B2024.0860.783.59
Re-TASK (Lite)Model=Llama3-8B2024.0860.786.21
Zero-shot CoT+SCModel=Llama3-8B2024.0858.82-0.65
Zero-shot CoTModel=Llama3-8B2024.0857.84—
Plan-and-SolveModel=Llama3-8B2024.0855.880.65
Re-TASK (Lite)Model=Qwen1.5-7B2024.0850.986.21
One-shot CoTModel=Mistral-7B2024.0845.13.59
Re-TASK (Lite)Model=Mistral-7B2024.0844.126.21
Zero-shot CoTModel=Qwen1.5-7B2024.0842.16—
One-shot CoTModel=Qwen1.5-7B2024.0842.163.59
Plan-and-SolveModel=Mistral-7B2024.0842.160.65
Plan-and-SolveModel=Qwen1.5-7B2024.0841.180.65
Zero-shot CoT+SCModel=Mistral-7B2024.0839.22-0.65
Zero-shot CoTModel=Mistral-7B2024.0837.25—
Zero-shot CoT+SCModel=Qwen1.5-7B2024.0837.25-0.65
Step-BackModel=Mistral-7B2024.0834.31-14.05
Step-BackModel=Llama3-8B2024.0830.39-14.05
Step-BackModel=Qwen1.5-7B2024.0830.39-14.05