Question Answering on LiteraryQA (test)
69.19AccuracyTop-down
Evaluation results
| Method | Links | |
|---|---|---|
| Top-downBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 69.19 | |
| TOABackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 68.14 | |
| Bottom-upBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 67.72 | |
| MemAgentBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 61.21 | |
| Zero-shotBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 60.24 | |
| Top-downBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 59.51 | |
| Bottom-upBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 58.64 | |
| TOABackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 58.25 | |
| QwenLong-L1.5Backbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 57.76 | |
| Zero-shotBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 56.48 | |
| MemAgentBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 52.65 |