Question Answering on StrategyQA (Accuracy, PEff, η, kt)
61.8AccuracyPoLR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| PoLRLp (Prefix length)=4096, Model=DSQ7B2026.01 | 61.8 | 0.656 | 0.061 | 13.1 | |
| PoLRLp (Prefix length)=1024, Model=DSQ7B2026.01 | 61.1 | 0.655 | 0.063 | 12.1 | |
| PoLRLp (Prefix length)=2048, Model=DSQ7B2026.01 | 60.5 | 0.651 | 0.058 | 13 | |
| PoLRLp (Prefix length)=2, Model=DSQ7B2026.01 | 60 | 0.185 | 0.11 | 1.3 | |
| SCLp (Prefix length)=2, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=4, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=8, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=16, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| PoLRLp (Prefix length)=16, Model=DSQ7B2026.01 | 59.8 | 0.361 | 0.192 | 1.7 | |
| SCLp (Prefix length)=32, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=64, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=128, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=256, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=512, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| PoLRLp (Prefix length)=512, Model=DSQ7B2026.01 | 59.8 | 0.657 | 0.15 | 12.2 | |
| SCLp (Prefix length)=1024, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=2048, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| SCLp (Prefix length)=4096, Model=DSQ7B2026.01 | 59.8 | — | — | — | |
| PoLRLp (Prefix length)=256, Model=DSQ7B2026.01 | 59.6 | 0.662 | 0.315 | 9.4 | |
| PoLRLp (Prefix length)=4, Model=DSQ7B2026.01 | 59 | 0.275 | 0.107 | 1.4 | |
| PoLRLp (Prefix length)=8, Model=DSQ7B2026.01 | 59 | 0.313 | 0.146 | 1.5 | |
| PoLRLp (Prefix length)=32, Model=DSQ7B2026.01 | 57.9 | 0.47 | 0.29 | 3.7 | |
| PoLRLp (Prefix length)=64, Model=DSQ7B2026.01 | 57.9 | 0.565 | 0.383 | 7.4 | |
| PoLRLp (Prefix length)=128, Model=DSQ7B2026.01 | 57.6 | 0.646 | 0.43 | 11.2 |