List your company
ResearchFormerly WizWand

STEM Question Answering on SciQ (first-token accuracy)

98.3First-Token AccuracyLlama-3.1-8B
29.1447.09565.0583.005May 21, 2025
Updated 7mo ago

Evaluation results

MethodLinks
Llama-3.1-8BStrategy=Prefilling2025.0598.3
Gemma-2-9BStrategy=Prefilling2025.0598.3
Phi-4-14BStrategy=Prefilling2025.0598.3
Phi-4-14BStrategy=Prompt Engineering2025.0598.2
Gemma-2-9BStrategy=Prompt Engineering2025.0598.1
Ministral-8BStrategy=Prefilling2025.0598
Llama-3.1-8BStrategy=Prompt Engineering2025.0597.8
Qwen-2-7BStrategy=FTP2025.0597.5
Qwen-2-7BStrategy=Prefilling2025.0597.4
Ministral-8BStrategy=FTP2025.0597.4
Ministral-8BStrategy=Prompt Engineering2025.0597.3
Qwen-2-7BStrategy=Prompt Engineering2025.0597.2
Mistral-Nemo-12BStrategy=Prefilling2025.0597.2
Llama-3.1-8BStrategy=FTP2025.0597.1
Mistral-Nemo-12BStrategy=FTP2025.0596.8
Mistral-Nemo-12BStrategy=Prompt Engineering2025.0595.9
Phi-4-14BStrategy=FTP2025.0595.4
Gemma-7BStrategy=Prefilling2025.0595.1
Gemma-7BStrategy=Prompt Engineering2025.0593.6
Zephyr-7BStrategy=Prefilling2025.0593.1
Zephyr-7BStrategy=Prompt Engineering2025.0589.6
Gemma-7BStrategy=FTP2025.0589
Zephyr-7BStrategy=FTP2025.0588.5
Gemma-2-9BStrategy=FTP2025.0531.8