Question Answering on Natural Questions (NQ) (EM/F1 Metrics)
36.71EMAdaCAD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| AdaCADModel=Mistral-7B-Inst2026.06 | 36.71 | 54.3 | |
| GreedyModel=Mistral-7B-Inst2026.06 | 35.45 | 53.25 | |
| COIECDModel=Mistral-7B-Inst2026.06 | 33.85 | 52.57 | |
| COIECDModel=Llama3-8B-Inst2026.06 | 32.93 | 52.32 | |
| CoCoAModel=Llama3-8B-Inst2026.06 | 32.11 | 51.32 | |
| AdaCADModel=Llama3-8B-Inst2026.06 | 31.86 | 51.26 | |
| ARRModel=Mistral-7B-Inst2026.06 | 31.43 | 48.26 | |
| CoCoAModel=Mistral-7B-Inst2026.06 | 31.14 | 50.09 | |
| GreedyModel=Llama3-8B-Inst2026.06 | 31.08 | 50.8 | |
| CADModel=Mistral-7B-Inst2026.06 | 26.12 | 45.54 | |
| ARRModel=Llama3-8B-Inst2026.06 | 24.98 | 44.8 | |
| COIECDModel=Llama2-13B-Inst2026.06 | 22.39 | 42.86 | |
| AdaCADModel=Llama2-13B-Inst2026.06 | 22.39 | 42.48 | |
| CADModel=Llama3-8B-Inst2026.06 | 21.17 | 41.57 | |
| GreedyModel=Llama2-13B-Inst2026.06 | 20.57 | 41.36 | |
| ARRModel=Llama2-13B-Inst2026.06 | 14.53 | 34.16 | |
| CADModel=Llama2-13B-Inst2026.06 | 13.87 | 33.32 | |
| CADModel=Qwen2.5-7B-Inst2026.06 | 4.59 | 20.33 | |
| CoCoAModel=Qwen2.5-7B-Inst2026.06 | 3.64 | 20.55 | |
| CoCoAModel=Llama2-13B-Inst2026.06 | 3.18 | 26.93 | |
| AdaCADModel=Qwen2.5-7B-Inst2026.06 | 1.92 | 19.02 | |
| COIECDModel=Qwen2.5-7B-Inst2026.06 | 1.32 | 17.95 | |
| GreedyModel=Qwen2.5-7B-Inst2026.06 | 0.88 | 18.18 | |
| ARRModel=Qwen2.5-7B-Inst2026.06 | 0.5 | 16.02 |