Question Answering on Natural Questions (AL, TPS)
232.58TPSPRISM
Evaluation results
| Method | Links | ||
|---|---|---|---|
| PRISMModel=LLaMA-2, Temp=T = 02026.02 | 232.58 | 4.24 | |
| HASSModel=LLaMA-2, Temp=T = 02026.02 | 217.04 | 4.08 | |
| EAGLE-2Model=LLaMA-2, Temp=T = 02026.02 | 201.01 | 3.84 | |
| PRISMModel=LLaMA-2, Temp=T = 12026.02 | 195.25 | 3.96 | |
| HASSModel=LLaMA-2, Temp=T = 12026.02 | 179.97 | 3.78 | |
| EAGLE-2Model=LLaMA-2, Temp=T = 12026.02 | 173.72 | 3.64 | |
| PRISMModel=LLaMA-3, Temp=T = 02026.02 | 168.44 | 3.58 | |
| HASSModel=LLaMA-3, Temp=T = 02026.02 | 155.38 | 3.41 | |
| EAGLE-2Model=LLaMA-3, Temp=T = 02026.02 | 142.68 | 3.24 | |
| StandardModel=LLaMA-2, Temp=T = 02026.02 | 136.31 | 2.84 | |
| StandardModel=LLaMA-2, Temp=T = 12026.02 | 134.66 | 2.79 | |
| StandardModel=LLaMA-3, Temp=T = 02026.02 | 130.6 | 4.48 | |
| PRISMModel=LLaMA-3, Temp=T = 12026.02 | 117.48 | 3.21 | |
| StandardModel=LLaMA-3, Temp=T = 12026.02 | 115.29 | 3.93 | |
| HASSModel=LLaMA-3, Temp=T = 12026.02 | 112.4 | 3.13 | |
| EAGLE-2Model=LLaMA-3, Temp=T = 12026.02 | 107 | 0.98 | |
| VanillaModel=LLaMA-2, Temp=T = 02026.02 | 104.52 | — | |
| VanillaModel=LLaMA-2, Temp=T = 12026.02 | 103.38 | — | |
| VanillaModel=LLaMA-3, Temp=T = 12026.02 | 90.97 | — | |
| VanillaModel=LLaMA-3, Temp=T = 02026.02 | 90.88 | — |