Question Answering on DetectiveQA
67.25AccuracyQRRanker
Evaluation Results
| Method | Links | |
|---|---|---|
| QRRankerMethod Category=QRRanker Series, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 67.25 | |
| QRHeads-4BMethod Category=QRRanker Series, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 64.75 | |
| Qwen3-Reranker-4B (trained)Method Category=Reranking Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 64.52 | |
| SFT-Embedding-8BMethod Category=Embedding Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 62.85 | |
| Qwen3-Reranker-4B (vanilla)Method Category=Reranking Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 60.93 | |
| Qwen3-Embedding-8BMethod Category=Embedding Methods, Generator=Qwen3-8B, Retrieval Context=R@3 retrieved chunks2026.02 | 57.35 | |
| Top-downBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 40.27 | |
| Bottom-upBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 39.6 | |
| TOABackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 38.76 | |
| Top-downBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 35.91 | |
| Bottom-upBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 34.06 | |
| Zero-shotBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 33.72 | |
| TOABackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 32.72 | |
| QwenLong-L1.5Backbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 32.21 | |
| Zero-shotBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 30.87 | |
| MemAgentBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 30.03 | |
| MemAgentBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 27.35 |