Book Plot Retrieval on Book Plot Retrieval (test)
63.03Recall@10SitEmb-v1.5-Qwen3 (QA+SA)
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| SitEmb-v1.5-Qwen3 (QA+SA)Setting=Situated, Size=8B2025.08 | 63.03 | 72.83 | 82.7 | |
| Qwen3 (QA+SA)Setting=Chunk-only, Size=8B2025.08 | 60.36 | 68.87 | 80.45 | |
| SitEmb-v1.5-Qwen3 (QA)Setting=Situated, Size=8B2025.08 | 53.57 | 63.56 | 78.64 | |
| voyage-context-3Setting=Late Chunking, Size=unk2025.08 | 51.39 | 60.89 | 73.7 | |
| Qwen3 (QA)Setting=Chunk-only, Size=8B2025.08 | 51.02 | 60.78 | 73.89 | |
| SitEmb-v1-M3 (SA)Setting=Situated, Size=1B2025.08 | 45.15 | 55.66 | 69.25 | |
| SitEmb-v1-M3 (SA) - ResidualSetting=Situated, Size=0.5B2025.08 | 43.85 | 54.98 | 68.93 | |
| Qwen3 (out-of-box)Setting=Chunk-only, Size=8B2025.08 | 43.57 | 52.94 | 69.48 | |
| M3 (SA) + ResidualSetting=Chunk-only, Size=1B2025.08 | 43.43 | 51.74 | 65.51 | |
| M3 (SA)Setting=Chunk-only, Size=0.5B2025.08 | 42.87 | 52.91 | 66.01 | |
| M3 (out-of-box)Setting=Chunk-only, Size=0.5B2025.08 | 32.92 | 41.46 | 55.85 |