List your company
ResearchFormerly WizWand

Reading Comprehension on SQuAD (Accuracy)

84.36AccuracyMeZO
30.727244.651158.57572.4989Feb 21, 2024Jul 1, 2024Nov 10, 2024Mar 21, 2025Jul 31, 2025Dec 9, 2025Apr 20, 2026
Updated 6mo ago

Evaluation results

MethodLinks
MeZOBackbone=OPT-13B, Evaluation Protocol=ZO-tuning2026.0484.36
MixtralParameters=8x22B2024.0784.1
AdaLeZOBackbone=OPT-13B, Evaluation Protocol=ZO-tuning2026.0483.52
GemmaParameters=7B2024.0781.8
Llama 3Parameters=70B2024.0781.8
Llama 3Parameters=405B2024.0781.8
MeZOBackbone=OPT-30B, Evaluation Protocol=ZO-tuning2026.0480.31
AdaLeZOBackbone=OPT-30B, Evaluation Protocol=ZO-tuning2026.0480.19
Llama 3Parameters=8B2024.0777
MistralParameters=7B2024.0773.2
Zero-shotBackbone=OPT-30B, Evaluation Protocol=Zero-shot2026.0446.5
Zero-shotBackbone=OPT-13B, Evaluation Protocol=Zero-shot2026.0446.2
INTRADOCChunk Length (L)=8K2024.0241.32
INTRADOCChunk Length (L)=2K2024.0239.91
BM25ChunkChunk Length (L)=8K2024.0239.45
BM25ChunkChunk Length (L)=2K2024.0237.91
UNIChunkChunk Length (L)=2K2024.0237.33
MIXChunkChunk Length (L)=2K2024.0236.7
UNIChunkChunk Length (L)=8K2024.0234.94
MIXChunkChunk Length (L)=8K2024.0232.79