Question Answering on NQ (Execution Time Overhead)
0.064Absolute Execution Time Overhead (s)Perplexity
Evaluation results
| Method | Links | |
|---|---|---|
| PerplexityModel=Llama-3-8B2026.03 | 0.064 | |
| PerplexityModel=Mistral-7B2026.03 | 0.075 | |
| PerplexityModel=Qwen2.5-3B2026.03 | 0.108 | |
| PerplexityModel=Llama-2-7B2026.03 | 0.129 | |
| PerplexityModel=Qwen2.5-14B2026.03 | 0.148 | |
| PerplexityModel=Qwen2.5-7B2026.03 | 0.173 | |
| STCModel=Llama-2-7B2026.03 | 3.028 | |
| STCModel=Mistral-7B2026.03 | 3.25 | |
| STCModel=Llama-3-8B2026.03 | 5.767 | |
| STCModel=Qwen2.5-3B2026.03 | 7.081 | |
| STCModel=Qwen2.5-7B2026.03 | 7.235 | |
| STCModel=Qwen2.5-14B2026.03 | 12.127 | |
| P(True)Model=Qwen2.5-3B2026.03 | 25.471 | |
| P(True)Model=Llama-3-8B2026.03 | 41.258 | |
| P(True)Model=Llama-2-7B2026.03 | 41.669 | |
| P(True)Model=Qwen2.5-7B2026.03 | 47.472 | |
| tokenSARModel=Llama-3-8B2026.03 | 49.245 | |
| P(True)Model=Mistral-7B2026.03 | 50.507 | |
| tokenSARModel=Llama-2-7B2026.03 | 50.865 | |
| tokenSARModel=Qwen2.5-3B2026.03 | 51.428 | |
| tokenSARModel=Qwen2.5-14B2026.03 | 59.225 | |
| P(True)Model=Qwen2.5-14B2026.03 | 61.176 | |
| tokenSARModel=Mistral-7B2026.03 | 68.746 | |
| tokenSARModel=Qwen2.5-7B2026.03 | 71.28 | |
| CCPModel=Llama-2-7B2026.03 | 355.393 | |
| CCPModel=Qwen2.5-3B2026.03 | 368.5 | |
| CCPModel=Llama-3-8B2026.03 | 375.405 | |
| CCPModel=Qwen2.5-14B2026.03 | 403.897 | |
| CCPModel=Mistral-7B2026.03 | 437.571 | |
| CCPModel=Qwen2.5-7B2026.03 | 458.807 | |
| ConUModel=Qwen2.5-3B2026.03 | 713.243 | |
| DegModel=Qwen2.5-3B2026.03 | 723.76 | |
| EccModel=Qwen2.5-3B2026.03 | 724.153 | |
| EigVModel=Qwen2.5-3B2026.03 | 724.462 | |
| Eigen ScoreModel=Qwen2.5-3B2026.03 | 731.121 | |
| DegModel=Qwen2.5-7B2026.03 | 743.213 | |
| EigVModel=Qwen2.5-7B2026.03 | 743.579 | |
| EccModel=Qwen2.5-7B2026.03 | 743.618 | |
| Eigen ScoreModel=Qwen2.5-7B2026.03 | 746.425 | |
| ConUModel=Qwen2.5-7B2026.03 | 747.901 | |
| Predictive EntropyModel=Qwen2.5-3B2026.03 | 824.862 | |
| LN EntropyModel=Qwen2.5-3B2026.03 | 824.943 | |
| Predictive EntropyModel=Qwen2.5-7B2026.03 | 863.792 | |
| LN EntropyModel=Qwen2.5-7B2026.03 | 863.88 | |
| sentenceSARModel=Qwen2.5-3B2026.03 | 884.191 | |
| Semantic EntropyModel=Qwen2.5-3B2026.03 | 894.15 | |
| DegModel=Llama-3-8B2026.03 | 907.995 | |
| EigVModel=Llama-3-8B2026.03 | 908.361 | |
| EccModel=Llama-3-8B2026.03 | 908.374 | |
| ConUModel=Llama-3-8B2026.03 | 910.635 | |
| Semantic EntropyModel=Qwen2.5-7B2026.03 | 929.426 | |
| sentenceSARModel=Qwen2.5-7B2026.03 | 934.741 | |
| Eigen ScoreModel=Llama-3-8B2026.03 | 951.202 | |
| Predictive EntropyModel=Llama-3-8B2026.03 | 1,006.703 | |
| LN EntropyModel=Llama-3-8B2026.03 | 1,006.781 | |
| DegModel=Mistral-7B2026.03 | 1,023.109 | |
| EigVModel=Mistral-7B2026.03 | 1,023.446 | |
| EccModel=Mistral-7B2026.03 | 1,023.511 | |
| Predictive EntropyModel=Mistral-7B2026.03 | 1,028.696 | |
| LN EntropyModel=Mistral-7B2026.03 | 1,028.779 | |
| ConUModel=Mistral-7B2026.03 | 1,030.129 | |
| Semantic EntropyModel=Llama-3-8B2026.03 | 1,070.526 | |
| sentenceSARModel=Llama-3-8B2026.03 | 1,073.833 | |
| Eigen ScoreModel=Mistral-7B2026.03 | 1,077.065 | |
| Semantic EntropyModel=Mistral-7B2026.03 | 1,089.431 | |
| sentenceSARModel=Mistral-7B2026.03 | 1,097.11 | |
| SARModel=Qwen2.5-3B2026.03 | 1,106.083 | |
| SARModel=Qwen2.5-7B2026.03 | 1,179.924 | |
| SARModel=Llama-3-8B2026.03 | 1,262.202 | |
| Predictive EntropyModel=Llama-2-7B2026.03 | 1,373.943 | |
| LN EntropyModel=Llama-2-7B2026.03 | 1,374.02 | |
| SARModel=Mistral-7B2026.03 | 1,378.736 | |
| DegModel=Llama-2-7B2026.03 | 1,383.066 | |
| EigVModel=Llama-2-7B2026.03 | 1,383.42 | |
| EccModel=Llama-2-7B2026.03 | 1,383.779 | |
| ConUModel=Llama-2-7B2026.03 | 1,387.134 | |
| Eigen ScoreModel=Llama-2-7B2026.03 | 1,407.099 | |
| Semantic EntropyModel=Llama-2-7B2026.03 | 1,420.822 | |
| sentenceSARModel=Llama-2-7B2026.03 | 1,425.529 | |
| SARModel=Llama-2-7B2026.03 | 1,608.375 | |
| DegModel=Qwen2.5-14B2026.03 | 1,647.769 | |
| EccModel=Qwen2.5-14B2026.03 | 1,648.152 | |
| EigVModel=Qwen2.5-14B2026.03 | 1,648.646 | |
| ConUModel=Qwen2.5-14B2026.03 | 1,655.025 | |
| Predictive EntropyModel=Qwen2.5-14B2026.03 | 1,717.797 | |
| LN EntropyModel=Qwen2.5-14B2026.03 | 1,717.879 | |
| Eigen ScoreModel=Qwen2.5-14B2026.03 | 1,735.322 | |
| Semantic EntropyModel=Qwen2.5-14B2026.03 | 1,775.456 | |
| sentenceSARModel=Qwen2.5-14B2026.03 | 1,783.411 | |
| SARModel=Qwen2.5-14B2026.03 | 2,015.596 |