Question Answering on SQuAD (Accuracy, CR, OR)
99.6AccuracyLayer-LR
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Layer-LRModel=Qwen2.5-7B, Mode=FTG (First-token generation), Context=Out-of-domain2026.02 | 99.6 | — | — | |
| Layer-LRModel=Llama-3.1-8B, Mode=FTG (First-token generation), Context=Out-of-domain2026.02 | 97.7 | — | — | |
| Layer-LRModel=Mistral-7B-v0.1, Mode=FTG (First-token generation), Context=Out-of-domain2026.02 | 91.8 | — | — | |
| DCRDModel=Llama3-8B2026.05 | 88.9 | — | — | |
| GreedyModel=Llama3-8B2026.05 | 87.2 | — | — | |
| ADACADModel=Llama3-8B2026.05 | 87.1 | — | — | |
| COIECDModel=Llama3-8B2026.05 | 86.9 | — | — | |
| DCRDModel=Llama2-7B2026.05 | 83.2 | — | — | |
| DCRDModel=Llama2-13B2026.05 | 79.5 | — | — | |
| COIECDModel=Llama2-13B2026.05 | 78.9 | — | — | |
| ADACADModel=Llama2-13B2026.05 | 76.5 | — | — | |
| COIECDModel=Llama2-7B2026.05 | 76 | — | — | |
| DGAOBackbone=Llama-3.2-3B2026.05 | 75.44 | 89.6 | 17.57 | |
| Pos-BeginningBackbone=Llama-3.2-3B2026.05 | 75.21 | 92.35 | 15.85 | |
| ADACADModel=Mistral-7B2026.05 | 75.2 | — | — | |
| Pos-EndBackbone=Llama-3.2-3B2026.05 | 74.99 | 90.49 | 17.65 | |
| ADACADModel=Llama2-7B2026.05 | 74.6 | — | — | |
| DEmOBackbone=Llama-3.2-3B2026.05 | 74.34 | 89.25 | 18.25 | |
| GreedyModel=Llama2-13B2026.05 | 73.9 | — | — | |
| GlobalEBackbone=Llama-3.2-3B2026.05 | 73.81 | 89.15 | 18.3 | |
| SFTBackbone=Llama-3.2-3B2026.05 | 73.73 | 88.98 | 18.54 | |
| SPHSBackbone=Llama-3.2-3B2026.05 | 73.59 | 89.1 | 18.71 | |
| PromptBackbone=Llama-3.2-3B2026.05 | 73.37 | 89.14 | 18.9 | |
| CADModel=Llama2-13B2026.05 | 72.7 | — | — | |
| CADModel=Llama3-8B2026.05 | 72.6 | — | — | |
| GreedyModel=Llama2-7B2026.05 | 71.6 | — | — | |
| DCRDModel=Mistral-7B2026.05 | 69.1 | — | — | |
| CADModel=Llama2-7B2026.05 | 67.7 | — | — | |
| GreedyModel=Mistral-7B2026.05 | 67.5 | — | — | |
| PINEBackbone=Llama-3.2-3B2026.05 | 66.91 | 81.75 | 18.42 | |
| COIECDModel=Mistral-7B2026.05 | 48.1 | — | — | |
| CADModel=Mistral-7B2026.05 | 25.2 | — | — |