Knowledge Composition Selection on HotpotQA
100Precision @2BASE
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| BASEType=MAX2025.08 | 100 | 87.01 | 91.85 | 78 | 97.52 | 85.33 | |
| KCSBase Model=BERT-base2025.08 | 64.18 | 55.46 | 58.7 | 50.12 | 64.33 | 55.52 | |
| KCSBase Model=BERT-large2025.08 | 63.29 | 54.71 | 57.9 | 49.09 | 62.97 | 54.37 | |
| CLSBase Model=BERT-base2025.08 | 62.7 | 54.05 | 57.26 | 48.62 | 62.42 | 53.87 | |
| CLSBase Model=BERT-large2025.08 | 58 | 49.52 | 52.64 | 44.83 | 59.69 | 49.3 | |
| RETRIEVALBase Model=BM252025.08 | 54.44 | 46.7 | 49.57 | 40.79 | 52.34 | 45.18 | |
| RETRIEVALBase Model=BM25, Execution=+step by step2025.08 | 54.08 | 46.6 | 49.37 | 40.75 | 52.44 | 45.2 | |
| RETRIEVALBase Model=bge-large-v1.5, Execution=+step by step2025.08 | 53.78 | 46.03 | 48.9 | 42.52 | 54.29 | 46.98 | |
| RETRIEVALBase Model=bge-large-v1.52025.08 | 53.56 | 45.79 | 48.66 | 41.27 | 52.78 | 45.63 | |
| LLMBase Model=GPT-4, prompt_mode=zero-shot2025.08 | 52.1 | 43.73 | 46.83 | 45.1 | 56.76 | 49.47 | |
| LLMBase Model=DeepSeek-V3, prompt_mode=zero-shot2025.08 | 49.93 | 41.66 | 44.71 | 43.08 | 53.95 | 47.14 | |
| RETRIEVALBase Model=bge-m32025.08 | 49.29 | 41.97 | 44.68 | 37.98 | 48.35 | 41.9 | |
| RETRIEVALBase Model=bge-small2025.08 | 48.83 | 41.49 | 44.33 | 38.19 | 48.82 | 42.22 | |
| RETRIEVALBase Model=text-ada-0022025.08 | 47.49 | 40.19 | 42.88 | 37.05 | 46.91 | 40.76 | |
| SENTENCE GRAPHStrategy=CommonEntity2025.08 | 46.39 | 40.03 | 42.4 | 33.11 | 42.74 | 36.79 | |
| KCSBase Model=RoBERTa-large2025.08 | 45.42 | 39.32 | 41.59 | 37.04 | 47.7 | 41.11 | |
| SENTENCE GRAPHStrategy=Similarity2025.08 | 40.95 | 35.4 | 37.46 | 27.39 | 35.5 | 30.5 | |
| CLSBase Model=RoBERTa-large2025.08 | 23 | 19.52 | 20.83 | 20.33 | 25.93 | 22.47 | |
| LLMBase Model=Qwen2.5-7B, prompt_mode=zero-shot2025.08 | 19.26 | 16.12 | 17.28 | 15.9 | 19.92 | 17.4 | |
| LLMBase Model=Qwen2.5-14B, prompt_mode=zero-shot2025.08 | 18.1 | 15.18 | 16.26 | 14.77 | 18.49 | 16.16 | |
| LLMBase Model=Llama3.2-3B, prompt_mode=zero-shot2025.08 | 9.62 | 8.08 | 8.65 | 8.11 | 10.18 | 8.88 | |
| LLMBase Model=Llama3.1-8B, prompt_mode=zero-shot2025.08 | 7.21 | 5.95 | 6.41 | 6.81 | 8.45 | 7.42 | |
| BASEType=Random2025.08 | 6.1 | 5.02 | 5.41 | 6.13 | 7.6 | 6.67 |