Question Answering on NQ-Swap
73AccuracyAdaCAD
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AdaCADModel=Llama-3.1-70B2026.04 | 73 | — | — | — | |
| NWCADModel=Llama-3.1-70B2026.04 | 73 | — | — | — | |
| CADModel=Llama-3.1-70B2026.04 | 71 | — | — | — | |
| CoCoAModel=Llama-3.1-70B2026.04 | 70.88 | — | — | — | |
| NWCADModel=Llama-3.1-8B2026.04 | 67 | — | — | — | |
| NWCADModel=Ministral-3-8B2026.04 | 66.9 | — | — | — | |
| CoCoAModel=Llama-3.1-8B2026.04 | 66.4 | — | — | — | |
| AdaCADModel=Llama-3.1-8B2026.04 | 66.1 | — | — | — | |
| DCRDModel=Llama2-13B2026.05 | 65.5 | — | — | — | |
| DCRDModel=Llama3-8B2026.05 | 65.3 | — | — | — | |
| With-contextModel=Llama-3.1-70B2026.04 | 64.75 | — | — | — | |
| CADModel=Llama-3.1-8B2026.04 | 64 | — | — | — | |
| CADModel=Ministral-3-8B2026.04 | 63.9 | — | — | — | |
| ADACADModel=Llama2-13B2026.05 | 62.6 | — | — | — | |
| With-contextModel=Llama-3.1-8B2026.04 | 62.2 | — | — | — | |
| AdaCADModel=Ministral-3-8B2026.04 | 62.1 | — | — | — | |
| With-contextModel=Ministral-3-8B2026.04 | 62 | — | — | — | |
| ADACADModel=Llama3-8B2026.05 | 58.3 | — | — | — | |
| DCRDModel=Mistral-7B2026.05 | 58.3 | — | — | — | |
| CADModel=Llama3-8B2026.05 | 56 | — | — | — | |
| COIECDModel=Mistral-7B2026.05 | 56 | — | — | — | |
| CoCoAModel=Ministral-3-8B2026.04 | 54.6 | — | — | — | |
| DCRDModel=Llama2-7B2026.05 | 54.2 | — | — | — | |
| COIECDModel=Llama2-13B2026.05 | 54 | — | — | — | |
| CADModel=Llama2-13B2026.05 | 53.9 | — | — | — | |
| COIECDModel=Llama3-8B2026.05 | 52.1 | — | — | — | |
| ADACADModel=Mistral-7B2026.05 | 51.7 | — | — | — | |
| COIECDModel=Llama2-7B2026.05 | 48.9 | — | — | — | |
| CADModel=Mistral-7B2026.05 | 48.1 | — | — | — | |
| CADModel=Llama2-7B2026.05 | 47.5 | — | — | — | |
| GreedyModel=Llama3-8B2026.05 | 47.2 | — | — | — | |
| ADACADModel=Llama2-7B2026.05 | 46.1 | — | — | — | |
| GreedyModel=Mistral-7B2026.05 | 45.6 | — | — | — | |
| GreedyModel=Llama2-7B2026.05 | 36.5 | — | — | — | |
| GreedyModel=Llama2-13B2026.05 | 36.4 | — | — | — | |
| BaselineModel=Ministral-3-8B2026.04 | 6.2 | — | — | — | |
| BaselineModel=Llama-3.1-70B2026.04 | 5.88 | — | — | — | |
| BaselineModel=Llama-3.1-8B2026.04 | 4.5 | — | — | — | |
| AdaCADModel=Qwen2.5-7b-instruct2026.02 | — | 73.14 | — | — | |
| AdaCADModel=LLaMA3-8b-instruct2026.02 | — | 67.37 | — | — | |
| AdaCADModel=Mistral-7b2026.02 | — | 63.99 | — | — | |
| AdaCADModel=LLaMA2-13b-chat2026.02 | — | 57.69 | — | — | |
| AdaCADModel=LLaMA2-13B2026.06 | — | 37.88 | 80.95 | 42.16 | |
| AdaCADModel=LLaMA3-70B2026.06 | — | 43.12 | 86.82 | 48.95 | |
| CADModel=Qwen2.5-7b-instruct2026.02 | — | 79.37 | — | — | |
| CADModel=LLaMA3-8b-instruct2026.02 | — | 80.1 | — | — | |
| CADModel=Mistral-7b2026.02 | — | 65.72 | — | — | |
| CADModel=LLaMA2-13b-chat2026.02 | — | 74.59 | — | — | |
| CADModel=LLaMA2-13B2026.06 | — | 30.12 | 78.34 | 33.05 | |
| CADModel=LLaMA3-70B2026.06 | — | 33.45 | 83.92 | 36.88 | |
| COIECDModel=Qwen2.5-7b-instruct2026.02 | — | 78.56 | — | — | |
| COIECDModel=LLaMA3-8b-instruct2026.02 | — | 70.39 | — | — | |
| COIECDModel=Mistral-7b2026.02 | — | 42.96 | — | — | |
| COIECDModel=LLaMA2-13b-chat2026.02 | — | 54.41 | — | — | |
| CoRectModel=Qwen2.5-7b-instruct2026.02 | — | 80.15 | — | — | |
| CoRectModel=LLaMA3-8b-instruct2026.02 | — | 79.32 | — | — | |
| CoRectModel=Mistral-7b2026.02 | — | 70.37 | — | — | |
| CoRectModel=LLaMA2-13b-chat2026.02 | — | 76.52 | — | — | |
| FIDESModel=LLaMA2-13B2026.06 | — | 44.52 | 84.82 | 48.23 | |
| FIDESModel=LLaMA3-70B2026.06 | — | 51.35 | 92.45 | 61.82 | |
| GreedyModel=Qwen2.5-7b-instruct2026.02 | — | 70.54 | — | — | |
| GreedyModel=LLaMA3-8b-instruct2026.02 | — | 60.67 | — | — | |
| GreedyModel=Mistral-7b2026.02 | — | 44.51 | — | — | |
| GreedyModel=LLaMA2-13b-chat2026.02 | — | 49.29 | — | — | |
| Standard RAGModel=LLaMA2-13B2026.06 | — | 33.54 | 69.12 | 36.82 | |
| Standard RAGModel=LLaMA3-70B2026.06 | — | 38.92 | 73.54 | 42.15 |