Question Answering on NaturalQA (test)
67.4AccuracyMemDefrag
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MemDefragTop-K=2, Time step (n)=202026.07 | 67.4 | — | |
| MemDefragTop-K=1, Time step (n)=202026.07 | 67.2 | — | |
| MemDefragTop-K=2, Time step (n)=102026.07 | 66.8 | — | |
| MemDefragTop-K=1, Time step (n)=102026.07 | 66 | — | |
| MemDefragTop-K=1, Time step (n)=302026.07 | 63.2 | — | |
| MemDefragTop-K=3, Time step (n)=102026.07 | 62.8 | — | |
| MemDefragTop-K=2, Time step (n)=302026.07 | 62.2 | — | |
| MemDefragTop-K=3, Time step (n)=202026.07 | 62 | — | |
| MemDefragTop-K=3, Time step (n)=302026.07 | 59.2 | — | |
| MemDefragTop-K=4, Time step (n)=202026.07 | 59.2 | — | |
| MemDefragTop-K=4, Time step (n)=102026.07 | 59 | — | |
| MemDefragTop-K=1, Time step (n)=402026.07 | 53.2 | — | |
| MemDefragTop-K=4, Time step (n)=302026.07 | 52.6 | — | |
| MemDefragTop-K=2, Time step (n)=402026.07 | 50 | — | |
| MemDefragTop-K=3, Time step (n)=402026.07 | 44.6 | — | |
| MemDefragTop-K=1, Time step (n)=502026.07 | 43 | — | |
| MemDefragTop-K=4, Time step (n)=402026.07 | 41.6 | — | |
| MemDefragTop-K=2, Time step (n)=502026.07 | 37.8 | — | |
| MemDefragTop-K=3, Time step (n)=502026.07 | 36.2 | — | |
| MemDefragTop-K=4, Time step (n)=502026.07 | 33.6 | — | |
| M+Time step (n)=102026.07 | 26.2 | — | |
| Promptingshots=5-shot2025.10 | 23.4 | 42.5 | |
| MemoryLLMTime step (n)=102026.07 | 23.4 | — | |
| DPO2025.10 | 22.3 | 56.2 | |
| AFHmode=Absolute2025.10 | 21.7 | 43.3 | |
| M+Time step (n)=302026.07 | 21 | — | |
| MASH w/ OTC-ST2025.10 | 20.9 | 57.4 | |
| M+Time step (n)=202026.07 | 19.6 | — | |
| MASH w/ EXP2025.10 | 18.9 | 53.6 | |
| MemoryLLMTime step (n)=302026.07 | 18.4 | — | |
| MemoryLLMTime step (n)=202026.07 | 18.2 | — | |
| MemoryLLMTime step (n)=402026.07 | 18 | — | |
| M+Time step (n)=502026.07 | 17.6 | — | |
| MemoryLLMTime step (n)=502026.07 | 17.4 | — | |
| M+Time step (n)=402026.07 | 17.2 | — | |
| AFHmode=Multisample2025.10 | 14.7 | 54.8 | |
| MASH w/ OTC2025.10 | 0.1 | 53.6 | |
| OTC2025.10 | 0 | — | |
| Ternary2025.10 | 0 | — |