Question Answering on TQA (Execution Time Overhead)
0.173Absolute Execution Time Overhead (s)Perplexity
Evaluation results
| Method | Links | |
|---|---|---|
| PerplexityModel=Qwen2.5-14B2026.03 | 0.173 | |
| PerplexityModel=Llama-2-7B2026.03 | 0.175 | |
| PerplexityModel=Mistral-7B2026.03 | 0.215 | |
| PerplexityModel=Qwen2.5-7B2026.03 | 0.226 | |
| PerplexityModel=Llama-3-8B2026.03 | 0.243 | |
| PerplexityModel=Qwen2.5-3B2026.03 | 0.299 | |
| STCModel=Mistral-7B2026.03 | 6.138 | |
| STCModel=Llama-2-7B2026.03 | 6.156 | |
| STCModel=Llama-3-8B2026.03 | 12.81 | |
| STCModel=Qwen2.5-7B2026.03 | 17.417 | |
| STCModel=Qwen2.5-14B2026.03 | 20.265 | |
| STCModel=Qwen2.5-3B2026.03 | 21.703 | |
| tokenSARModel=Llama-3-8B2026.03 | 33.505 | |
| tokenSARModel=Qwen2.5-14B2026.03 | 39.718 | |
| tokenSARModel=Llama-2-7B2026.03 | 42.683 | |
| P(True)Model=Qwen2.5-3B2026.03 | 42.733 | |
| tokenSARModel=Mistral-7B2026.03 | 44.933 | |
| tokenSARModel=Qwen2.5-7B2026.03 | 45.261 | |
| tokenSARModel=Qwen2.5-3B2026.03 | 57.409 | |
| P(True)Model=Llama-3-8B2026.03 | 64.794 | |
| P(True)Model=Qwen2.5-7B2026.03 | 67.394 | |
| P(True)Model=Llama-2-7B2026.03 | 77.504 | |
| P(True)Model=Mistral-7B2026.03 | 84.098 | |
| P(True)Model=Qwen2.5-14B2026.03 | 110.757 | |
| CCPModel=Qwen2.5-14B2026.03 | 400.007 | |
| CCPModel=Llama-3-8B2026.03 | 413.634 | |
| CCPModel=Mistral-7B2026.03 | 449.906 | |
| CCPModel=Qwen2.5-7B2026.03 | 451.137 | |
| CCPModel=Llama-2-7B2026.03 | 451.485 | |
| CCPModel=Qwen2.5-3B2026.03 | 530.589 | |
| DegModel=Qwen2.5-7B2026.03 | 694.2 | |
| EigVModel=Qwen2.5-7B2026.03 | 695.372 | |
| EccModel=Qwen2.5-7B2026.03 | 696.236 | |
| Eigen ScoreModel=Qwen2.5-7B2026.03 | 719.904 | |
| DegModel=Mistral-7B2026.03 | 737.141 | |
| EigVModel=Mistral-7B2026.03 | 738.037 | |
| EccModel=Mistral-7B2026.03 | 738.256 | |
| ConUModel=Qwen2.5-7B2026.03 | 738.87 | |
| Predictive EntropyModel=Mistral-7B2026.03 | 766.178 | |
| LN EntropyModel=Mistral-7B2026.03 | 766.41 | |
| DegModel=Llama-3-8B2026.03 | 767.505 | |
| EigVModel=Llama-3-8B2026.03 | 768.285 | |
| EccModel=Llama-3-8B2026.03 | 768.468 | |
| Eigen ScoreModel=Mistral-7B2026.03 | 779.427 | |
| ConUModel=Mistral-7B2026.03 | 792.341 | |
| Predictive EntropyModel=Qwen2.5-7B2026.03 | 799.185 | |
| LN EntropyModel=Qwen2.5-7B2026.03 | 799.403 | |
| Eigen ScoreModel=Llama-3-8B2026.03 | 800.555 | |
| Semantic EntropyModel=Mistral-7B2026.03 | 801.556 | |
| ConUModel=Llama-3-8B2026.03 | 823.381 | |
| Semantic EntropyModel=Qwen2.5-7B2026.03 | 850.439 | |
| Predictive EntropyModel=Llama-3-8B2026.03 | 858.933 | |
| LN EntropyModel=Llama-3-8B2026.03 | 859.173 | |
| sentenceSARModel=Mistral-7B2026.03 | 859.242 | |
| sentenceSARModel=Qwen2.5-7B2026.03 | 897.532 | |
| Semantic EntropyModel=Llama-3-8B2026.03 | 899.641 | |
| sentenceSARModel=Llama-3-8B2026.03 | 958.088 | |
| DegModel=Llama-2-7B2026.03 | 999.541 | |
| EigVModel=Llama-2-7B2026.03 | 1,000.469 | |
| EccModel=Llama-2-7B2026.03 | 1,000.627 | |
| Predictive EntropyModel=Llama-2-7B2026.03 | 1,001.054 | |
| LN EntropyModel=Llama-2-7B2026.03 | 1,001.284 | |
| DegModel=Qwen2.5-3B2026.03 | 1,014.938 | |
| EigVModel=Qwen2.5-3B2026.03 | 1,015.869 | |
| EccModel=Qwen2.5-3B2026.03 | 1,016.296 | |
| Eigen ScoreModel=Llama-2-7B2026.03 | 1,039.766 | |
| ConUModel=Qwen2.5-3B2026.03 | 1,042.695 | |
| ConUModel=Llama-2-7B2026.03 | 1,043.482 | |
| Semantic EntropyModel=Llama-2-7B2026.03 | 1,046.829 | |
| Eigen ScoreModel=Qwen2.5-3B2026.03 | 1,063.181 | |
| SARModel=Mistral-7B2026.03 | 1,086.359 | |
| sentenceSARModel=Llama-2-7B2026.03 | 1,093.151 | |
| SARModel=Qwen2.5-7B2026.03 | 1,111.684 | |
| SARModel=Llama-3-8B2026.03 | 1,152.862 | |
| Predictive EntropyModel=Qwen2.5-3B2026.03 | 1,185.428 | |
| LN EntropyModel=Qwen2.5-3B2026.03 | 1,185.655 | |
| Semantic EntropyModel=Qwen2.5-3B2026.03 | 1,269.717 | |
| sentenceSARModel=Qwen2.5-3B2026.03 | 1,299.645 | |
| DegModel=Qwen2.5-14B2026.03 | 1,328.461 | |
| EigVModel=Qwen2.5-14B2026.03 | 1,329.348 | |
| EccModel=Qwen2.5-14B2026.03 | 1,329.495 | |
| SARModel=Llama-2-7B2026.03 | 1,330.471 | |
| ConUModel=Qwen2.5-14B2026.03 | 1,381.948 | |
| Predictive EntropyModel=Qwen2.5-14B2026.03 | 1,414.291 | |
| LN EntropyModel=Qwen2.5-14B2026.03 | 1,414.523 | |
| Eigen ScoreModel=Qwen2.5-14B2026.03 | 1,426.132 | |
| Semantic EntropyModel=Qwen2.5-14B2026.03 | 1,449.723 | |
| sentenceSARModel=Qwen2.5-14B2026.03 | 1,505.557 | |
| SARModel=Qwen2.5-3B2026.03 | 1,581.633 | |
| SARModel=Qwen2.5-14B2026.03 | 1,705.063 |