Question Answering on NovelQA (test)
60.27AccuracyTop-down
Evaluation results
| Method | Links | |
|---|---|---|
| Top-downBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 60.27 | |
| Bottom-upBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 59.39 | |
| TOABackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 58.06 | |
| MemAgentBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 56.32 | |
| Zero-shotBackbone=Qwen3-235B-A22B-Instruct-2507, Quantization=FP82026.06 | 51.81 | |
| Top-downBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 51.45 | |
| Bottom-upBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 50.99 | |
| Zero-shotBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 48.78 | |
| TOABackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 45.71 | |
| QwenLong-L1.5Backbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 45.34 | |
| MemAgentBackbone=Qwen3-30B-A3B-Instruct-2507, Quantization=FP82026.06 | 40.06 |