Knowledge Retrieval on TriviaQA
62.11Exact Match (EM)W/O (FP16)
Evaluation Results
| Method | Links | ||||
|---|---|---|---|---|---|
| W/O (FP16)Backbone=Gemma-2-9B2025.11 | 62.11 | — | 68.92 | 17.21 | |
| SliM-LLMBackbone=Gemma-2-9B2025.11 | 61.23 | — | 68.87 | 8.31 | |
| TAQ-KLBackbone=Gemma-2-9B2025.11 | 61.04 | — | 68.58 | 6.76 | |
| TAQ-ISBackbone=Gemma-2-9B2025.11 | 59.67 | — | 68.08 | 6.76 | |
| TAQ-OBackbone=Gemma-2-9B2025.11 | 59.33 | — | 67.67 | 9.02 | |
| GPTQBackbone=Gemma-2-9B2025.11 | 52.64 | — | 61.04 | 5.75 | |
| AWQBackbone=Gemma-2-9B2025.11 | 51.51 | — | 60.14 | 5.74 | |
| W/O (FP16)Backbone=Qwen2.5-3B2025.11 | 17.24 | — | 24.69 | 5.75 | |
| W/O (FP16)Backbone=Qwen2.5-7B2025.11 | 16.94 | — | 25.62 | 14.19 | |
| GPTQBackbone=Qwen2.5-7B2025.11 | 15.97 | — | 26.57 | 5.19 | |
| SliM-LLMBackbone=Qwen2.5-7B2025.11 | 15.77 | — | 26.2 | 5.83 | |
| AWQBackbone=Qwen2-7B2025.11 | 14.79 | — | 25.16 | 5.19 | |
| TAQ-KLBackbone=Qwen2.5-7B2025.11 | 14.06 | — | 23.83 | 5.95 | |
| TAQ-ISBackbone=Qwen2.5-7B2025.11 | 12.55 | — | 23.95 | 5.95 | |
| TAQ-OBackbone=Qwen2.5-7B2025.11 | 12.26 | — | 23.23 | 9.05 | |
| AWQBackbone=Qwen2.5-3B2025.11 | 9.72 | — | 17.66 | 2.5 | |
| TAQ-OBackbone=Qwen2.5-3B2025.11 | 8.45 | — | 20.34 | 3.2 | |
| GPTQBackbone=Qwen2.5-3B2025.11 | 7.23 | — | 15.54 | 1.93 | |
| AWQBackbone=Qwen2.5-7B2025.11 | 6.98 | — | 18.15 | 5.19 | |
| GPTQBackbone=Qwen2-7B2025.11 | 5.71 | — | 17.1 | 5.19 | |
| TAQ-ISBackbone=Qwen2.5-3B2025.11 | 3.91 | — | 17.14 | 2.25 | |
| SliM-LLMBackbone=Qwen2.5-3B2025.11 | 3.22 | — | 17.48 | 2.77 | |
| SliM-LLMBackbone=Qwen2-7B2025.11 | 2.44 | — | 21.09 | 5.83 | |
| TAQ-ISBackbone=Qwen2-7B2025.11 | 2.25 | — | 20.79 | 5.95 | |
| TAQ-KLBackbone=Qwen2-7B2025.11 | 2.25 | — | 21.17 | 5.95 | |
| TAQ-OBackbone=Qwen2-7B2025.11 | 1.86 | — | 20.58 | 9.05 | |
| TAQ-KLBackbone=Qwen2.5-3B2025.11 | 1.32 | — | 13.69 | 2.25 | |
| W/O (FP16)Backbone=Qwen2-7B2025.11 | 0.73 | — | 12.94 | 14.19 | |
| MTPLoop strategy=1 to 8, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 3.353 | — | — | |
| MTPLoop strategy=4 to 8, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 3.324 | — | — | |
| MTP-DLoop strategy=1, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 1.114 | — | — | |
| MTP-DLoop strategy=4, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 2.782 | — | — | |
| MTP-DLoop strategy=1 to 8, Training protocol=Training Free, Training data size=70B tokens2026.03 | — | 2.263 | — | — | |
| MTP-DLoop strategy=1 to 8, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 3.455 | — | — | |
| MTP-DLoop strategy=4 to 8, Training protocol=Training Free, Training data size=70B tokens2026.03 | — | 3.192 | — | — | |
| MTP-DLoop strategy=4 to 8, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 3.985 | — | — | |
| MTP-DLoop strategy=4 to 8, Training protocol=Continued pre-training, Training data size=350B tokens2026.03 | — | 3.972 | — | — | |
| MTP-DLoop strategy=1 to 16, Training protocol=Training Free, Training data size=70B tokens2026.03 | — | 2.687 | — | — | |
| MTP-DLoop strategy=1 to 16, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 3.157 | — | — | |
| MTP-DLoop strategy=4 to 16, Training protocol=Training Free, Training data size=70B tokens2026.03 | — | 3.741 | — | — | |
| MTP-DLoop strategy=4 to 16, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 4.765 | — | — | |
| MTP-DLoop strategy=4 to 8 to 16, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 4.323 | — | — | |
| MTP-D ensembleLoop strategy=4 to 8, Training protocol=Continued pre-training, Training data size=70B tokens2026.03 | — | 3.959 | — | — |