Question Answering on WikiHop (test)
74AccuracyFine-Tuning
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| Fine-Tuning% Tuned=100, Backbone=Longformer-base2023.05 | 74 | — | — | |
| Coref-GRU2018.09 | 59.3 | — | — | |
| MHPGM+NOICNOIC=true2018.09 | 57.9 | — | — | |
| MHPGMNOIC=false2018.09 | 57.5 | — | — | |
| LATENTGRAPHMEMBackbone=Qwen3-8B2026.01 | 48.81 | — | 40.37 | |
| PREMemBackbone=Qwen3-8B2026.01 | 45.87 | — | 31.55 | |
| BiDAF2018.09 | 42.9 | — | — | |
| Prefix-Propagation% Tuned=0.05, Backbone=Longformer-base2023.05 | 42.2 | — | — | |
| MemGenBackbone=SmolLM3-3B2026.01 | 41.35 | — | 42.59 | |
| Reasoner-onlyBackbone=Qwen3-8B2026.01 | 40.62 | — | 31.55 | |
| MemGenBackbone=Qwen3-8B2026.01 | 40.5 | — | 40.3 | |
| LATENTGRAPHMEMBackbone=SmolLM3-3B2026.01 | 40.03 | — | 35.71 | |
| Prefix-Tuning% Tuned=0.1, Backbone=Longformer-base2023.05 | 38.9 | — | — | |
| PREMemBackbone=SmolLM3-3B2026.01 | 38.5 | — | 25.77 | |
| LATENTGRAPHMEMBackbone=Qwen2.5-1.5B-Instruct2026.01 | 36.13 | — | 25.03 | |
| Reasoner-onlyBackbone=Qwen2.5-1.5B-Instruct2026.01 | 34.63 | — | 23.57 | |
| Reasoner-onlyBackbone=SmolLM3-3B2026.01 | 34.5 | — | 24.61 | |
| MemGenBackbone=Qwen2.5-1.5B-Instruct2026.01 | 32.37 | — | 32.54 | |
| THEANINEBackbone=SmolLM3-3B2026.01 | 19.75 | — | 13.36 | |
| A-MemBackbone=SmolLM3-3B2026.01 | 17.13 | — | 28.45 | |
| THEANINEBackbone=Qwen3-8B2026.01 | 16.15 | — | 10.48 | |
| A-MemBackbone=Qwen2.5-1.5B-Instruct2026.01 | 12.12 | — | 20.53 | |
| ROBERTa PT% Tuned=0.1, Backbone=ROBERTa-base2023.05 | 11.7 | — | — | |
| Mem0Backbone=Qwen2.5-1.5B-Instruct2026.01 | 11.63 | — | 10.15 | |
| THEANINEBackbone=Qwen2.5-1.5B-Instruct2026.01 | 10.25 | — | 6.64 | |
| PREMemBackbone=Qwen2.5-1.5B-Instruct2026.01 | 10.13 | — | 10.78 | |
| A-MemBackbone=Qwen3-8B2026.01 | 9.38 | — | 17.89 | |
| RAGBackbone=Qwen3-8B2026.01 | 5.12 | — | 20.04 | |
| Mem0Backbone=Qwen3-8B2026.01 | 4.13 | — | 3.46 | |
| Mem0Backbone=SmolLM3-3B2026.01 | 2.5 | — | 2.26 | |
| RAGBackbone=SmolLM3-3B2026.01 | 2.25 | — | 16.52 | |
| RAGBackbone=Qwen2.5-1.5B-Instruct2026.01 | 1.5 | — | 8.05 | |
| BIGBIRD-ETCSetup=Fine-tuning2020.07 | — | 82.3 | — | |
| LongformerSetup=Fine-tuning2020.07 | — | 81.9 | — | |
| MRC-GCNSetup=Fine-tuning2020.07 | — | 78.3 | — | |
| MultiHopSetup=Fine-tuning2020.07 | — | 76.5 | — |