Question Answering on WQ (absolute execution time overhead (seconds))
0.039Absolute Execution Time Overhead (s)Perplexity
Evaluation results
| Method | Links | |
|---|---|---|
| PerplexityModel=Llama-3-8B2026.03 | 0.039 | |
| PerplexityModel=Mistral-7B2026.03 | 0.058 | |
| PerplexityModel=Qwen2.5-3B2026.03 | 0.09 | |
| PerplexityModel=Llama-2-7B2026.03 | 0.099 | |
| PerplexityModel=Qwen2.5-7B2026.03 | 0.109 | |
| PerplexityModel=Qwen2.5-14B2026.03 | 0.128 | |
| STCModel=Llama-2-7B2026.03 | 3.106 | |
| STCModel=Mistral-7B2026.03 | 3.265 | |
| STCModel=Qwen2.5-14B2026.03 | 4.228 | |
| STCModel=Llama-3-8B2026.03 | 4.408 | |
| STCModel=Qwen2.5-3B2026.03 | 5.967 | |
| STCModel=Qwen2.5-7B2026.03 | 6.65 | |
| P(True)Model=Qwen2.5-3B2026.03 | 24.845 | |
| tokenSARModel=Qwen2.5-14B2026.03 | 33.112 | |
| P(True)Model=Qwen2.5-14B2026.03 | 34.523 | |
| tokenSARModel=Llama-3-8B2026.03 | 42.5 | |
| P(True)Model=Qwen2.5-7B2026.03 | 43 | |
| P(True)Model=Llama-3-8B2026.03 | 44.397 | |
| P(True)Model=Llama-2-7B2026.03 | 45.502 | |
| P(True)Model=Mistral-7B2026.03 | 47.685 | |
| tokenSARModel=Qwen2.5-3B2026.03 | 77.041 | |
| tokenSARModel=Qwen2.5-7B2026.03 | 79.275 | |
| tokenSARModel=Llama-2-7B2026.03 | 79.666 | |
| tokenSARModel=Mistral-7B2026.03 | 83.959 | |
| CCPModel=Qwen2.5-14B2026.03 | 215.823 | |
| CCPModel=Llama-3-8B2026.03 | 363.803 | |
| CCPModel=Mistral-7B2026.03 | 492.417 | |
| CCPModel=Qwen2.5-7B2026.03 | 505.802 | |
| CCPModel=Llama-2-7B2026.03 | 530.564 | |
| CCPModel=Qwen2.5-3B2026.03 | 541.178 | |
| DegModel=Llama-3-8B2026.03 | 609.385 | |
| EigVModel=Llama-3-8B2026.03 | 609.569 | |
| EccModel=Llama-3-8B2026.03 | 609.604 | |
| ConUModel=Llama-3-8B2026.03 | 618.553 | |
| Eigen ScoreModel=Llama-3-8B2026.03 | 628.671 | |
| Predictive EntropyModel=Llama-3-8B2026.03 | 693.584 | |
| LN EntropyModel=Llama-3-8B2026.03 | 693.63 | |
| Semantic EntropyModel=Llama-3-8B2026.03 | 741.462 | |
| sentenceSARModel=Llama-3-8B2026.03 | 751.045 | |
| DegModel=Qwen2.5-7B2026.03 | 842.112 | |
| EigVModel=Qwen2.5-7B2026.03 | 842.272 | |
| EccModel=Qwen2.5-7B2026.03 | 842.305 | |
| ConUModel=Qwen2.5-7B2026.03 | 844.796 | |
| Predictive EntropyModel=Mistral-7B2026.03 | 880.289 | |
| LN EntropyModel=Mistral-7B2026.03 | 880.338 | |
| DegModel=Mistral-7B2026.03 | 884.519 | |
| EigVModel=Mistral-7B2026.03 | 884.717 | |
| EccModel=Mistral-7B2026.03 | 884.749 | |
| Eigen ScoreModel=Qwen2.5-7B2026.03 | 885.305 | |
| ConUModel=Mistral-7B2026.03 | 888.34 | |
| SARModel=Llama-3-8B2026.03 | 911.192 | |
| Semantic EntropyModel=Mistral-7B2026.03 | 934.841 | |
| Eigen ScoreModel=Mistral-7B2026.03 | 937.786 | |
| sentenceSARModel=Mistral-7B2026.03 | 939.009 | |
| DegModel=Qwen2.5-3B2026.03 | 985.459 | |
| EigVModel=Qwen2.5-3B2026.03 | 985.672 | |
| EccModel=Qwen2.5-3B2026.03 | 985.68 | |
| ConUModel=Qwen2.5-3B2026.03 | 985.693 | |
| Predictive EntropyModel=Qwen2.5-7B2026.03 | 1,033.338 | |
| LN EntropyModel=Qwen2.5-7B2026.03 | 1,033.386 | |
| Eigen ScoreModel=Qwen2.5-3B2026.03 | 1,046.944 | |
| Semantic EntropyModel=Qwen2.5-7B2026.03 | 1,095.089 | |
| sentenceSARModel=Qwen2.5-7B2026.03 | 1,098.173 | |
| Predictive EntropyModel=Llama-2-7B2026.03 | 1,177.434 | |
| LN EntropyModel=Llama-2-7B2026.03 | 1,177.483 | |
| DegModel=Llama-2-7B2026.03 | 1,195.29 | |
| EigVModel=Llama-2-7B2026.03 | 1,195.448 | |
| EccModel=Llama-2-7B2026.03 | 1,195.505 | |
| ConUModel=Llama-2-7B2026.03 | 1,203.438 | |
| SARModel=Mistral-7B2026.03 | 1,218.985 | |
| Predictive EntropyModel=Qwen2.5-3B2026.03 | 1,226.562 | |
| LN EntropyModel=Qwen2.5-3B2026.03 | 1,226.606 | |
| Semantic EntropyModel=Llama-2-7B2026.03 | 1,228.517 | |
| Eigen ScoreModel=Llama-2-7B2026.03 | 1,230.437 | |
| sentenceSARModel=Llama-2-7B2026.03 | 1,237.058 | |
| DegModel=Qwen2.5-14B2026.03 | 1,255.907 | |
| EigVModel=Qwen2.5-14B2026.03 | 1,256.075 | |
| EccModel=Qwen2.5-14B2026.03 | 1,256.113 | |
| ConUModel=Qwen2.5-14B2026.03 | 1,266.87 | |
| Semantic EntropyModel=Qwen2.5-3B2026.03 | 1,295.211 | |
| sentenceSARModel=Qwen2.5-3B2026.03 | 1,295.818 | |
| Predictive EntropyModel=Qwen2.5-14B2026.03 | 1,303.567 | |
| LN EntropyModel=Qwen2.5-14B2026.03 | 1,303.611 | |
| Semantic EntropyModel=Qwen2.5-14B2026.03 | 1,340.643 | |
| Eigen ScoreModel=Qwen2.5-14B2026.03 | 1,350.655 | |
| sentenceSARModel=Qwen2.5-14B2026.03 | 1,352.041 | |
| SARModel=Qwen2.5-7B2026.03 | 1,417.933 | |
| SARModel=Llama-2-7B2026.03 | 1,473.864 | |
| SARModel=Qwen2.5-14B2026.03 | 1,531.791 | |
| SARModel=Qwen2.5-3B2026.03 | 1,628.814 |