Question Answering on HotpotQA τ = 96k
86.9E2E Wall Time (s)gpt-oss-20B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| gpt-oss-20BVariant=Sequential, Block size=—2026.05 | 86.9 | 1 | 145.2 | 1 | 1,493 | 1 | 11,127 | 1 | 5.34 | 1 | |
| gpt-oss-20BVariant=Parallel, Block size=16k2026.05 | 100.7 | 0.86 | 147.1 | 1.01 | 3,098 | 2.07 | 11,704 | 1.05 | 5.59 | 0.95 | |
| gpt-oss-20BVariant=Parallel, Block size=8k2026.05 | 101.8 | 0.85 | 168.1 | 1.16 | 5,401 | 3.62 | 11,710 | 1.05 | 3.39 | 1.58 | |
| Llama-3.1-8BVariant=Parallel, Block size=8k2026.05 | 104.6 | 1.44 | 62.8 | 1.38 | 1,151 | 0.65 | 5,424 | 1.07 | 17.89 | 0.8 | |
| gpt-oss-20BVariant=Parallel, Block size=4k2026.05 | 105.1 | 0.83 | 205.3 | 1.41 | 10,017 | 6.71 | 11,554 | 1.04 | 2.12 | 2.52 | |
| Llama-3.1-8BVariant=Parallel, Block size=4k2026.05 | 105.8 | 1.43 | 71 | 1.56 | 2,199 | 1.24 | 5,313 | 1.05 | 10.43 | 1.37 | |
| gpt-oss-20BVariant=Parallel, Block size=2k2026.05 | 108.3 | 0.8 | 253.8 | 1.75 | 16,180 | 10.84 | 11,303 | 1.02 | 1.53 | 3.48 | |
| Llama-3.1-8BVariant=Parallel, Block size=2k2026.05 | 119.4 | 1.26 | 83.8 | 1.84 | 4,575 | 2.58 | 5,433 | 1.07 | 6.57 | 2.18 | |
| Llama-3.1-8BVariant=Parallel, Block size=16k2026.05 | 131.7 | 1.14 | 45.2 | 0.99 | 681 | 0.38 | 5,272 | 1.04 | 32.96 | 0.43 | |
| Llama-3.1-8BVariant=Sequential, Block size=—2026.05 | 150.8 | 1 | 45.5 | 1 | 1,776 | 1 | 5,082 | 1 | 14.31 | 1 | |
| gpt-oss-120BVariant=Sequential, Block size=—2026.05 | 214.5 | 1 | 51 | 1 | 1,462 | 1 | 9,480 | 1 | 7.28 | 1 | |
| gpt-oss-120BVariant=Parallel, Block size=4k2026.05 | 217.7 | 0.99 | 90 | 1.76 | 9,894 | 6.77 | 9,701 | 1.02 | 3.58 | 2.04 | |
| gpt-oss-120BVariant=Parallel, Block size=2k2026.05 | 233.1 | 0.92 | 109 | 2.14 | 16,486 | 11.28 | 8,913 | 0.94 | 2.51 | 2.91 | |
| Llama-3.3-70BVariant=Parallel, Block size=4k2026.05 | 266.6 | 1.4 | 52.5 | 1.35 | 8,360 | 0.97 | 5,645 | 0.94 | 10.62 | 2.13 | |
| Llama-3.3-70BVariant=Parallel, Block size=8k2026.05 | 268.5 | 1.39 | 46.4 | 1.19 | 6,823 | 0.79 | 5,647 | 0.94 | 13.29 | 1.7 | |
| gpt-oss-120BVariant=Parallel, Block size=8k2026.05 | 270.3 | 0.79 | 59.7 | 1.17 | 6,994 | 4.78 | 9,136 | 0.96 | 5.22 | 1.39 | |
| gpt-oss-120BVariant=Parallel, Block size=16k2026.05 | 276 | 0.78 | 49.8 | 0.98 | 4,116 | 2.81 | 9,615 | 1.01 | 7.64 | 0.95 | |
| Llama-3.3-70BVariant=Parallel, Block size=16k2026.05 | 289.7 | 1.29 | 39.6 | 1.01 | 5,317 | 0.62 | 6,145 | 1.03 | 19.79 | 1.14 | |
| Llama-3.3-70BVariant=Parallel, Block size=2k2026.05 | 292.4 | 1.28 | 64.1 | 1.64 | 12,303 | 1.43 | 6,427 | 1.07 | 8.09 | 2.8 | |
| Llama-3.3-70BVariant=Sequential, Block size=—2026.05 | 373.5 | 1 | 39 | 1 | 8,582 | 1 | 5,994 | 1 | 22.62 | 1 |