Commonsense Reasoning on CSQA LongGenBench (test)
86.1Accuracy (n=15)Full Attention
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Full AttentionModel (Backbone)=QWEN2.5-32B-INSTRUCT, Attention Mechanism=Full Attention2025.08 | 86.1 | 87 | 87.1 | 86.7 | |
| RetroAttentionModel (Backbone)=QWEN2.5-32B-INSTRUCT, Attention Mechanism=RetroAttention2025.08 | 85.3 | 75 | 78.7 | 79.7 | |
| Full AttentionModel (Backbone)=QWEN2.5-14B-INSTRUCT, Attention Mechanism=Full Attention2025.08 | 83.7 | 83.5 | 83.5 | 83.6 | |
| QuestModel (Backbone)=QWEN2.5-32B-INSTRUCT, Attention Mechanism=Quest2025.08 | 82.3 | 69.3 | 74.4 | 75.4 | |
| RetroAttentionModel (Backbone)=QWEN2.5-14B-INSTRUCT, Attention Mechanism=RetroAttention2025.08 | 73.3 | 59.5 | 63.6 | 65.5 | |
| QuestModel (Backbone)=QWEN2.5-14B-INSTRUCT, Attention Mechanism=Quest2025.08 | 69.4 | 46.8 | 58.1 | 58.1 |