Knowledge-intensive Claim Extraction and Information Retrieval on 75-case (test)
3.555Overall ScoreDisasterLex
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| DisasterLexBase Model=Qwen 3.6 Flash2026.05 | 3.555 | 3.554 | 3.679 | 2.962 | 4.564 | |
| DisasterLexBase Model=Gemini 3.1 Flash Lite Preview2026.05 | 3.542 | 3.676 | 3.607 | 2.952 | 4.449 | |
| DisasterLexBase Model=DeepSeek V3.22026.05 | 3.033 | 3.044 | 3.235 | 2.292 | 4.256 | |
| DisasterLexBase Model=Qwen3 8B2026.05 | 2.566 | 2.598 | 2.492 | 1.92 | 3.923 | |
| DisasterLexBase Model=Qwen3 32B2026.05 | 2.565 | 2.657 | 2.466 | 1.974 | 3.769 | |
| DisasterLexBase Model=Llama 3.1 8B2026.05 | 1.753 | 1.864 | 1.405 | 1.394 | 3.03 | |
| LightRAGBase Model=Qwen3 8B2026.05 | 1.75 | 1.044 | 2.484 | 0.679 | 3.744 | |
| DisasterLexBase Model=Llama 3.3 70B2026.05 | 1.65 | 1.662 | 1.151 | 1.346 | 2.974 | |
| LightRAGBase Model=Qwen 3.6 Flash2026.05 | 1.407 | 0.941 | 1.801 | 0.397 | 3.462 | |
| LightRAGBase Model=Qwen3 32B2026.05 | 1.388 | 0.632 | 2.246 | 0.538 | 2.821 | |
| LightRAGBase Model=Gemini 3.1 Flash Lite Preview2026.05 | 1.286 | 0.676 | 1.965 | 0.388 | 2.885 | |
| LightRAGBase Model=Llama 3.1 8B2026.05 | 1.268 | 0.824 | 1.945 | 0.66 | 2.077 | |
| HippoRAG 2Base Model=Llama 3.3 70B2026.05 | 1.2 | 0.338 | 2.026 | 0.324 | 2.872 | |
| HippoRAG 2Base Model=Llama 3.1 8B2026.05 | 1.182 | 0.559 | 1.759 | 0.26 | 3 | |
| LightRAGBase Model=DeepSeek V3.22026.05 | 1.123 | 0.588 | 1.534 | 0.221 | 3.026 | |
| LightRAGBase Model=Llama 3.3 70B2026.05 | 1.076 | 0.809 | 1.706 | 0.538 | 1.577 | |
| HippoRAG 2Base Model=Qwen3 32B2026.05 | 1.04 | 0.176 | 1.439 | 0.34 | 2.987 | |
| HippoRAG 2Base Model=Qwen3 8B2026.05 | 0.907 | 0.059 | 1.389 | 0.269 | 2.59 | |
| HippoRAG 2Base Model=DeepSeek V3.22026.05 | 0.651 | 0.088 | 0.71 | 0.09 | 2.423 | |
| ReFoRCEBase Model=Gemini 3.1 Flash Lite Preview2026.05 | 0.62 | 0.157 | 0.474 | 0.455 | 1.769 | |
| HippoRAG 2Base Model=Qwen 3.6 Flash2026.05 | 0.591 | 0.147 | 0.412 | 0.038 | 2.538 | |
| ReFoRCEBase Model=Qwen3 32B2026.05 | 0.516 | 0.314 | 0.237 | 0.545 | 1.128 | |
| ReFoRCEBase Model=DeepSeek V3.22026.05 | 0.503 | 0.069 | 0.456 | 0.349 | 1.444 | |
| HippoRAG 2Base Model=Gemini 3.1 Flash Lite Preview2026.05 | 0.502 | 0 | 0.377 | 0.058 | 2.231 | |
| ReFoRCEBase Model=Llama 3.3 70B2026.05 | 0.499 | 0.196 | 0.351 | 0.285 | 1.538 | |
| CHESSBase Model=Qwen 3.6 Flash2026.05 | 0.462 | 0.588 | 0.325 | 0.442 | 0.538 | |
| CHESSBase Model=DeepSeek V3.22026.05 | 0.434 | 0.235 | 0.184 | 0.26 | 1.41 | |
| CHESSBase Model=Gemini 3.1 Flash Lite Preview2026.05 | 0.417 | 0.275 | 0.456 | 0.279 | 0.821 | |
| ReFoRCEBase Model=Qwen 3.6 Flash2026.05 | 0.354 | 0 | 0.409 | 0.08 | 1.282 | |
| ReFoRCEBase Model=Qwen3 8B, Note=n = 2 seeds2026.05 | 0.33 | 0.294 | 0.289 | 0.317 | 0.462 | |
| CHESSBase Model=Qwen3 32B2026.05 | 0.242 | 0.078 | 0.123 | 0.147 | 0.821 | |
| CHESSBase Model=Llama 3.1 8B2026.05 | 0.238 | 0.069 | 0.36 | 0.032 | 0.692 | |
| ReFoRCEBase Model=Llama 3.1 8B2026.05 | 0.212 | 0 | 0.088 | 0.022 | 1.051 | |
| CHESSBase Model=Llama 3.3 70B2026.05 | 0.158 | 0.039 | 0.061 | 0.103 | 0.564 | |
| CHESSBase Model=Qwen3 8B2026.05 | 0.069 | 0 | 0.07 | 0.096 | 0.103 |