Knowledge composition selection on 2WikiMultihopQA
100Precision @ K=2BASE
Evaluation Results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BASEType=MAX2025.08 | 100 | 89.68 | 93.12 | 73.55 | 94.85 | 81.19 | |
| KCSBase Model=BERT-base2025.08 | 84.79 | 75.18 | 78.39 | 66.21 | 84.66 | 72.75 | |
| CLSBase Model=BERT-base2025.08 | 79.9 | 69.68 | 73.09 | 65.06 | 82.55 | 71.2 | |
| KCSBase Model=BERT-large2025.08 | 75.34 | 66.65 | 69.55 | 56.54 | 74.44 | 63.11 | |
| KCSBase Model=RoBERTa-large2025.08 | 74.21 | 65.74 | 68.56 | 55.58 | 73.75 | 62.3 | |
| CLSBase Model=BERT-large2025.08 | 73.25 | 64.25 | 67.25 | 61.5 | 79 | 67.74 | |
| RETRIEVALBase Model=BM25, Execution=+step by step2025.08 | 53.9 | 47.96 | 49.94 | 41.29 | 54.44 | 46.12 | |
| SENTENCE GRAPHStrategy=CommonEntity2025.08 | 51.2 | 45.24 | 47.23 | 37.09 | 48.68 | 41.33 | |
| RETRIEVALBase Model=bge-large-v1.5, Execution=+step by step2025.08 | 50.91 | 43.58 | 46.03 | 40.67 | 52.24 | 44.8 | |
| LLMBase Model=GPT-4, prompt_mode=zero-shot2025.08 | 49.74 | 43.59 | 45.64 | 40.1 | 52.56 | 44.65 | |
| RETRIEVALBase Model=BM252025.08 | 48.1 | 43.42 | 44.98 | 35.1 | 47.38 | 39.71 | |
| LLMBase Model=DeepSeek-V3, prompt_mode=zero-shot2025.08 | 45.77 | 40.15 | 42.02 | 37.96 | 50.09 | 42.43 | |
| CLSBase Model=RoBERTa-large2025.08 | 44.75 | 36.88 | 39.5 | 37.83 | 46.25 | 40.6 | |
| RETRIEVALBase Model=bge-large-v1.52025.08 | 44.02 | 39.12 | 40.75 | 33.47 | 44.73 | 37.66 | |
| RETRIEVALBase Model=text-ada-0022025.08 | 42.29 | 37.41 | 39.03 | 31.65 | 42.14 | 35.54 | |
| SENTENCE GRAPHStrategy=Similarity2025.08 | 41.62 | 37.87 | 39.12 | 27.88 | 38.07 | 31.74 | |
| RETRIEVALBase Model=bge-m32025.08 | 41.06 | 36.53 | 38.04 | 30.55 | 41.02 | 34.47 | |
| RETRIEVALBase Model=bge-small2025.08 | 40.04 | 35.5 | 37.02 | 30.69 | 41.01 | 34.53 | |
| LLMBase Model=Qwen2.5-7B, prompt_mode=zero-shot2025.08 | 25.05 | 21.58 | 22.74 | 19.61 | 25.2 | 21.61 | |
| LLMBase Model=Qwen2.5-14B, prompt_mode=zero-shot2025.08 | 19.24 | 16.09 | 17.14 | 16.68 | 20.84 | 18.11 | |
| LLMBase Model=Llama3.2-3B, prompt_mode=zero-shot2025.08 | 13.73 | 10.91 | 11.85 | 11.71 | 14 | 12.42 | |
| LLMBase Model=Llama3.1-8B, prompt_mode=zero-shot2025.08 | 10.96 | 8.84 | 9.55 | 10.61 | 12.85 | 11.33 | |
| BASEType=Random2025.08 | 8.6 | 6.95 | 7.5 | 9.01 | 10.96 | 9.64 |