Question Answering on RULER QA-16k (test)
512Token CountInfLLM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| InfLLMToken Budget=5122026.03 | 512 | 21.7 | |
| UT-ACAToken Budget=5122026.03 | 385 | 21.65 | |
| InfLLMToken Budget=2562026.03 | 256 | 18.75 | |
| UT-ACAToken Budget=2562026.03 | 203 | 17.9 | |
| InfLLMToken Budget=1282026.03 | 128 | 14.05 | |
| UT-ACAToken Budget=1282026.03 | 105 | 14.65 | |
| InfLLMToken Budget=642026.03 | 64 | 9.25 | |
| UT-ACAToken Budget=642026.03 | 48 | 10.09 |