Reading Comprehension on SQuAD (Accuracy)
84.36AccuracyMeZO
Evaluation Results
| Method | Links | |
|---|---|---|
| MeZOBackbone=OPT-13B, Evaluation Protocol=ZO-tuning2026.04 | 84.36 | |
| MixtralParameters=8x22B2024.07 | 84.1 | |
| AdaLeZOBackbone=OPT-13B, Evaluation Protocol=ZO-tuning2026.04 | 83.52 | |
| GemmaParameters=7B2024.07 | 81.8 | |
| Llama 3Parameters=70B2024.07 | 81.8 | |
| Llama 3Parameters=405B2024.07 | 81.8 | |
| MeZOBackbone=OPT-30B, Evaluation Protocol=ZO-tuning2026.04 | 80.31 | |
| AdaLeZOBackbone=OPT-30B, Evaluation Protocol=ZO-tuning2026.04 | 80.19 | |
| Llama 3Parameters=8B2024.07 | 77 | |
| MistralParameters=7B2024.07 | 73.2 | |
| Zero-shotBackbone=OPT-30B, Evaluation Protocol=Zero-shot2026.04 | 46.5 | |
| Zero-shotBackbone=OPT-13B, Evaluation Protocol=Zero-shot2026.04 | 46.2 | |
| INTRADOCChunk Length (L)=8K2024.02 | 41.32 | |
| INTRADOCChunk Length (L)=2K2024.02 | 39.91 | |
| BM25ChunkChunk Length (L)=8K2024.02 | 39.45 | |
| BM25ChunkChunk Length (L)=2K2024.02 | 37.91 | |
| UNIChunkChunk Length (L)=2K2024.02 | 37.33 | |
| MIXChunkChunk Length (L)=2K2024.02 | 36.7 | |
| UNIChunkChunk Length (L)=8K2024.02 | 34.94 | |
| MIXChunkChunk Length (L)=8K2024.02 | 32.79 |