Language Model Performance Evaluation on LLM Evaluation Benchmark
92.27PerformanceOracleTopP
Evaluation Results
| Method | Links | |
|---|---|---|
| OracleTopPModel=Qwen3-30B-A3B-Instruct-2507, Density=2%2025.10 | 92.27 | |
| OracleTopPModel=Qwen3-30B-A3B-Instruct-2507, Density=5%2025.10 | 91.75 | |
| PQCacheModel=Qwen3-30B-A3B-Instruct-2507, Density=10%2025.10 | 91.52 | |
| OracleTopKModel=Qwen3-30B-A3B-Instruct-2507, Density=10%2025.10 | 91.41 | |
| OracleTopPModel=Qwen3-30B-A3B-Instruct-2507, Density=10%2025.10 | 91.33 | |
| OracleTopPModel=Qwen3-30B-A3B-Instruct-2507, Density=20%2025.10 | 91.33 | |
| PQCacheModel=Qwen3-30B-A3B-Instruct-2507, Density=20%2025.10 | 91.17 | |
| OracleTopKModel=Qwen3-30B-A3B-Instruct-2507, Density=5%2025.10 | 91.11 | |
| OracleTopKModel=Qwen3-30B-A3B-Instruct-2507, Density=20%2025.10 | 91.08 | |
| denseModel=Qwen3-30B-A3B-Instruct-2507, Density=100%2025.10 | 91.02 | |
| PQCacheModel=Qwen3-30B-A3B-Instruct-2507, Density=5%2025.10 | 90.94 | |
| PQCacheModel=Qwen3-30B-A3B-Instruct-2507, Density=2%2025.10 | 90.91 | |
| vAttention(OracleTopK)Model=Qwen3-30B-A3B-Instruct-2507, Density=2%2025.10 | 90.89 | |
| vAttention(OracleTopK)Model=Qwen3-30B-A3B-Instruct-2507, Density=5%2025.10 | 90.89 | |
| vAttention(OracleTopK)Model=Qwen3-30B-A3B-Instruct-2507, Density=10%2025.10 | 90.8 | |
| vAttention(OracleTopK)Model=Qwen3-30B-A3B-Instruct-2507, Density=20%2025.10 | 90.63 | |
| OracleTopKModel=Qwen3-30B-A3B-Instruct-2507, Density=2%2025.10 | 90.58 | |
| OracleTopPModel=Qwen3-4B-Instruct-2507, Density=2%2025.10 | 88.83 | |
| denseModel=Qwen3-4B-Instruct-2507, Density=100%2025.10 | 88.67 | |
| OracleTopPModel=Qwen3-4B-Instruct-2507, Density=10%2025.10 | 88.33 | |
| OracleTopKModel=Qwen3-4B-Instruct-2507, Density=20%2025.10 | 88.22 | |
| vAttention(OracleTopK)Model=Qwen3-4B-Instruct-2507, Density=20%2025.10 | 88.22 | |
| vAttention(OracleTopK)Model=Qwen3-4B-Instruct-2507, Density=5%2025.10 | 88.17 | |
| vAttention(OracleTopK)Model=Llama-3.1-8B-Instruct, Density=20%2025.10 | 88.11 | |
| OracleTopPModel=Qwen3-4B-Instruct-2507, Density=5%2025.10 | 88.06 | |
| denseModel=Llama-3.1-8B-Instruct, Density=100%2025.10 | 87.89 | |
| OracleTopPModel=Qwen3-4B-Instruct-2507, Density=20%2025.10 | 87.72 | |
| PQCacheModel=Qwen3-4B-Instruct-2507, Density=10%2025.10 | 87.67 | |
| PQCacheModel=Qwen3-4B-Instruct-2507, Density=20%2025.10 | 87.67 | |
| OracleTopPModel=Llama-3.1-8B-Instruct, Density=20%2025.10 | 87.62 | |
| vAttention(OracleTopK)Model=Qwen3-4B-Instruct-2507, Density=2%2025.10 | 87.61 | |
| OracleTopKModel=Qwen3-4B-Instruct-2507, Density=10%2025.10 | 87.61 | |
| PQCacheModel=Qwen3-4B-Instruct-2507, Density=5%2025.10 | 87.56 | |
| vAttention(OracleTopK)Model=Llama-3.1-8B-Instruct, Density=10%2025.10 | 87.5 | |
| OracleTopPModel=Llama-3.1-8B-Instruct, Density=10%2025.10 | 87.45 | |
| OracleTopKModel=Qwen3-4B-Instruct-2507, Density=5%2025.10 | 87.44 | |
| vAttention(OracleTopK)Model=Llama-3.1-8B-Instruct, Density=2%2025.10 | 87.18 | |
| OracleTopPModel=Llama-3.1-8B-Instruct, Density=5%2025.10 | 87.01 | |
| PQCacheModel=Llama-3.1-8B-Instruct, Density=20%2025.10 | 86.98 | |
| vAttention(OracleTopK)Model=Qwen3-4B-Instruct-2507, Density=10%2025.10 | 86.94 | |
| OracleTopKModel=Llama-3.1-8B-Instruct, Density=20%2025.10 | 86.9 | |
| vAttention(OracleTopK)Model=Llama-3.1-8B-Instruct, Density=5%2025.10 | 86.72 | |
| OracleTopKModel=Llama-3.1-8B-Instruct, Density=10%2025.10 | 86.37 | |
| OracleTopKModel=Qwen3-4B-Instruct-2507, Density=2%2025.10 | 86.33 | |
| PQCacheModel=Llama-3.1-8B-Instruct, Density=10%2025.10 | 86.29 | |
| OracleTopPModel=Llama-3.1-8B-Instruct, Density=2%2025.10 | 86.07 | |
| PQCacheModel=Qwen3-4B-Instruct-2507, Density=2%2025.10 | 85.8 | |
| OracleTopKModel=Llama-3.1-8B-Instruct, Density=5%2025.10 | 83.83 | |
| DoubleSparsityModel=Llama-3.1-8B-Instruct, Density=20%2025.10 | 83.4 | |
| PQCacheModel=Llama-3.1-8B-Instruct, Density=5%2025.10 | 83.14 | |
| MagicPigModel=Qwen3-4B-Instruct-2507, Density=20%2025.10 | 76.37 | |
| DoubleSparsityModel=Llama-3.1-8B-Instruct, Density=10%2025.10 | 74.75 | |
| OracleTopKModel=Llama-3.1-8B-Instruct, Density=2%2025.10 | 74.1 | |
| MagicPigModel=Qwen3-30B-A3B-Instruct-2507, Density=20%2025.10 | 69.25 | |
| PQCacheModel=Llama-3.1-8B-Instruct, Density=2%2025.10 | 68.95 | |
| denseModel=Llama-3.2-3B-Instruct, Density=100%2025.10 | 66.25 | |
| vAttention(OracleTopK)Model=Llama-3.2-3B-Instruct, Density=20%2025.10 | 65.95 | |
| vAttention(OracleTopK)Model=Llama-3.2-3B-Instruct, Density=5%2025.10 | 65.45 | |
| vAttention(OracleTopK)Model=Llama-3.2-3B-Instruct, Density=10%2025.10 | 65.09 | |
| OracleTopPModel=Llama-3.2-3B-Instruct, Density=20%2025.10 | 62.15 | |
| DoubleSparsityModel=Qwen3-30B-A3B-Instruct-2507, Density=20%2025.10 | 59.57 | |
| OracleTopPModel=Llama-3.2-3B-Instruct, Density=10%2025.10 | 59.34 | |
| vAttention(OracleTopK)Model=Llama-3.2-3B-Instruct, Density=2%2025.10 | 59.25 | |
| PQCacheModel=Llama-3.2-3B-Instruct, Density=20%2025.10 | 59.25 | |
| OracleTopKModel=Llama-3.2-3B-Instruct, Density=20%2025.10 | 59 | |
| OracleTopPModel=Llama-3.2-3B-Instruct, Density=5%2025.10 | 56.41 | |
| OracleTopKModel=Llama-3.2-3B-Instruct, Density=10%2025.10 | 53.33 | |
| DoubleSparsityModel=Llama-3.1-8B-Instruct, Density=5%2025.10 | 52.83 | |
| PQCacheModel=Llama-3.2-3B-Instruct, Density=10%2025.10 | 51.72 | |
| OracleTopPModel=Llama-3.2-3B-Instruct, Density=2%2025.10 | 51.19 | |
| DoubleSparsityModel=Qwen3-4B-Instruct-2507, Density=20%2025.10 | 47.5 | |
| OracleTopKModel=Llama-3.2-3B-Instruct, Density=5%2025.10 | 47.32 | |
| PQCacheModel=Llama-3.2-3B-Instruct, Density=5%2025.10 | 46.32 | |
| MagicPigModel=Llama-3.1-8B-Instruct, Density=20%2025.10 | 44.08 | |
| OracleTopKModel=Llama-3.2-3B-Instruct, Density=2%2025.10 | 41.57 | |
| DoubleSparsityModel=Llama-3.2-3B-Instruct, Density=20%2025.10 | 40.26 | |
| MagicPigModel=Qwen3-4B-Instruct-2507, Density=10%2025.10 | 39.63 | |
| PQCacheModel=Llama-3.2-3B-Instruct, Density=2%2025.10 | 39.3 | |
| MagicPigModel=Qwen3-30B-A3B-Instruct-2507, Density=10%2025.10 | 38.43 | |
| vAttention(OracleTopK)Model=Llama-3.2-1B-Instruct, Density=10%2025.10 | 37.83 | |
| vAttention(OracleTopK)Model=Llama-3.2-1B-Instruct, Density=5%2025.10 | 37.8 | |
| vAttention(OracleTopK)Model=Llama-3.2-1B-Instruct, Density=20%2025.10 | 37.51 | |
| denseModel=Llama-3.2-1B-Instruct, Density=100%2025.10 | 37.47 | |
| vAttention(OracleTopK)Model=Llama-3.2-1B-Instruct, Density=2%2025.10 | 36.78 | |
| DoubleSparsityModel=Qwen3-30B-A3B-Instruct-2507, Density=10%2025.10 | 36.2 | |
| MagicPigModel=Llama-3.2-3B-Instruct, Density=20%2025.10 | 36.19 | |
| PQCacheModel=Llama-3.2-1B-Instruct, Density=10%2025.10 | 36.18 | |
| OracleTopPModel=Llama-3.2-1B-Instruct, Density=20%2025.10 | 36 | |
| PQCacheModel=Llama-3.2-1B-Instruct, Density=20%2025.10 | 35.97 | |
| OracleTopKModel=Llama-3.2-1B-Instruct, Density=20%2025.10 | 35.89 | |
| OracleTopPModel=Llama-3.2-1B-Instruct, Density=10%2025.10 | 35.81 | |
| OracleTopKModel=Llama-3.2-1B-Instruct, Density=10%2025.10 | 35.22 | |
| DoubleSparsityModel=Llama-3.1-8B-Instruct, Density=2%2025.10 | 34.9 | |
| OracleTopPModel=Llama-3.2-1B-Instruct, Density=5%2025.10 | 34.74 | |
| OracleTopKModel=Llama-3.2-1B-Instruct, Density=5%2025.10 | 32.66 | |
| MagicPigModel=Qwen3-30B-A3B-Instruct-2507, Density=5%2025.10 | 32.5 | |
| OracleTopPModel=Llama-3.2-1B-Instruct, Density=2%2025.10 | 31.3 | |
| DoubleSparsityModel=Llama-3.2-3B-Instruct, Density=10%2025.10 | 30.77 | |
| PQCacheModel=Llama-3.2-1B-Instruct, Density=5%2025.10 | 30.47 | |
| MagicPigModel=Llama-3.1-8B-Instruct, Density=10%2025.10 | 30.2 |