Extractive Question Answering on SynthWiki@28K (test)
71.5Performance Score (Seed 42)Attn Sort k = 5
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Attn Sort k = 5Passes=5×(sort+gen), Backbone Model=YaRN-Llama-2-7b-64k, k (number of sorting iterations)=52026.06 | 71.5 | 73.5 | 67 | 70.67 | 3.33 | |
| Debiased k = 1Passes=1 sort + 1 gen, Backbone Model=YaRN-Llama-2-7b-64k, k (number of sorting iterations)=1, Debiasing Configuration=full-sort with divisive debiasing (α = 0.005, B = 40, mean)2026.06 | 57.5 | 53.5 | 56.5 | 55.83 | 2.08 | |
| Attn Sort k = 1Passes=1 sort + 1 gen, Backbone Model=YaRN-Llama-2-7b-64k, k (number of sorting iterations)=12026.06 | 47 | 45 | 49.5 | 47.17 | 2.25 | |
| No SortingPasses=1 gen, Backbone Model=YaRN-Llama-2-7b-64k2026.06 | 36 | 33.5 | 38 | 35.83 | 2.25 |