Mathematical Reasoning on MATH 500 (mean wall-clock time)
40.2Mean Wall-Clock Timeh2o
Evaluation Results
| Method | Links | |
|---|---|---|
| h2oKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 40.2 | |
| h2oKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 71.4 | |
| h2oKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 91.4 | |
| kv-keyKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 113.4 | |
| kv-valKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 113.5 | |
| band-adaptiveKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 113.8 | |
| lag-kv-keyKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 116.3 | |
| lag-kvKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 119.5 | |
| EPIKVKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 125 | |
| hs-varianceKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 128.8 | |
| kv-valKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 152.5 | |
| band-adaptiveKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 155.3 | |
| kv-keyKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 156 | |
| lag-kv-keyKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 158.3 | |
| lag-kvKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 163.4 | |
| noneKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 180 | |
| EPIKVKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 181.9 | |
| hs-varianceKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 181.9 | |
| hybrid-seg-hsKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 193.8 | |
| attn×hsKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 211.5 | |
| hybrid-seg-hsKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 222.5 | |
| band-adaptiveKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 246.3 | |
| raasKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 246.6 | |
| kv-keyKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 248.7 | |
| kv-valKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 249.4 | |
| lag-kv-keyKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 251 | |
| lag-kvKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 252 | |
| lag-kvKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 267.5 | |
| lag-kv-keyKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 269.1 | |
| kv-valKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 271.5 | |
| band-adaptiveKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 272.4 | |
| kv-keyKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 273.5 | |
| EPIKVKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 276.9 | |
| hs-varianceKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 282.2 | |
| attn×hsKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 290.5 | |
| hybrid-seg-hsKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 309.2 | |
| raasKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 317.8 | |
| EPIKVKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 319.9 | |
| hs-varianceKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 342.1 | |
| thinKVKV-cache budget=4096, Hardware=RTX A6000, GPU count per job=12026.06 | 344.6 | |
| thinKVKV-cache budget=2048, Hardware=RTX A6000, GPU count per job=12026.06 | 399.4 | |
| attn×hsKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 414.4 | |
| thinKVKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 416.5 | |
| hybrid-seg-hsKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 420.9 | |
| raasKV-cache budget=1024, Hardware=RTX A6000, GPU count per job=12026.06 | 515.8 | |
| thinKVKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 520.2 | |
| attn×hsKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 572.1 | |
| h2oKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 628 | |
| raasKV-cache budget=512, Hardware=RTX A6000, GPU count per job=12026.06 | 873.3 |