MQAR
Benchmarks
TaskDatasetSOTA resultTrendResults
MQAR Shuffle v1
99.99Accuracy
14
MQAR K2V2-Robustness v1
99.99Accuracy
7
MQAR K2V2 v1
99.99Accuracy
7
MQAR Std-Last v1
93.75Accuracy
5
MQAR npairs=24 (held-out)
1MQAR (held-out) Seed 42 Score
4
MQAR seq=128, kv=32
57.9Mean Accuracy
3
MQAR seq=128, kv=16
67Mean Accuracy
3
MQAR seq=128, kv=8
66.8Mean Accuracy
3
MQAR seq=128, kv=4
67Mean Accuracy
3
MQAR Hard
17.3Accuracy (MQAR Hard)
2
MQAR Medium
100Accuracy
2
MQAR Easy
100Accuracy
2