Story Question Answering on DetectiveQA
82.3AccuracySitEmb-v1.5-Qwen3 (QA)
Evaluation Results
| Method | Links | |
|---|---|---|
| SitEmb-v1.5-Qwen3 (QA)Number of retrieved chunks=top-102025.08 | 82.3 | |
| Qwen3 (QA)Number of retrieved chunks=top-102025.08 | 81.8 | |
| SitEmb-v1.5-Qwen3 (QA)Number of retrieved chunks=top-52025.08 | 78.7 | |
| SitEmb-v1.5-Qwen3 (QA+SA)Number of retrieved chunks=top-102025.08 | 78.3 | |
| Qwen3 (QA)Number of retrieved chunks=top-52025.08 | 78.2 | |
| SitEmb-v1.5-Qwen3 (QA+SA)Number of retrieved chunks=top-52025.08 | 74.2 | |
| Qwen3 (out-of-box)Number of retrieved chunks=top-102025.08 | 73.2 | |
| SitEmb-v1.5-Qwen3 (QA)Number of retrieved chunks=top-32025.08 | 73.2 | |
| Qwen3 (QA)Number of retrieved chunks=top-32025.08 | 70.5 | |
| Qwen3 (out-of-box)Number of retrieved chunks=top-52025.08 | 68.7 | |
| SitEmb-v1.5-Qwen3 (QA+SA)Number of retrieved chunks=top-32025.08 | 66.5 | |
| Qwen3 (out-of-box)Number of retrieved chunks=top-32025.08 | 62.5 |