Multiple Choice Question Answering on FrenchBench
82.2AccuracyTokens to Words
Evaluation Results
| Method | Links | |
|---|---|---|
| Tokens to WordsBackbone=Qwen3-8B2025.05 | 82.2 | |
| Original tokenizationBackbone=Qwen3-8B2025.05 | 81.7 | |
| Original tokenizationModel=Qwen3-8B2025.05 | 81.7 | |
| Token DistillationBackbone=Qwen3-8B2025.05 | 81.5 | |
| Token DistillationModel=Qwen3-8B2025.05 | 81.5 | |
| NTPBackbone=Qwen3-8B2025.05 | 81.2 | |
| NTPModel=Qwen3-8B2025.05 | 81.2 | |
| Original tokenizationBackbone=Avg.2025.05 | 73.2 | |
| Original tokenizationModel=Avg.2025.05 | 73.2 | |
| Token DistillationBackbone=Llama3-8B-i2025.05 | 72.9 | |
| Token DistillationBackbone=Avg.2025.05 | 72.9 | |
| Token DistillationModel=Llama3-8B-i2025.05 | 72.9 | |
| Token DistillationModel=Avg.2025.05 | 72.9 | |
| Original tokenizationBackbone=Llama3-8B-i2025.05 | 72.1 | |
| Original tokenizationModel=Llama3-8B-i2025.05 | 72.1 | |
| NTPBackbone=Avg.2025.05 | 70.8 | |
| NTPModel=Avg.2025.05 | 70.8 | |
| NTPBackbone=Llama3-8B-i2025.05 | 70.1 | |
| NTPModel=Llama3-8B-i2025.05 | 70.1 | |
| MeanBackbone=Qwen3-8B2025.05 | 69.6 | |
| MeanModel=Qwen3-8B2025.05 | 69.6 | |
| Original tokenizationBackbone=Mistral-7B2025.05 | 69.5 | |
| Original tokenizationModel=Mistral-7B2025.05 | 69.5 | |
| Original tokenizationBackbone=Llama3-8B2025.05 | 69.4 | |
| Original tokenizationModel=Llama3-8B2025.05 | 69.4 | |
| Tokens to WordsBackbone=Avg.2025.05 | 69.1 | |
| Token DistillationBackbone=Llama3-8B2025.05 | 68.9 | |
| Token DistillationModel=Llama3-8B2025.05 | 68.9 | |
| Token DistillationBackbone=Mistral-7B2025.05 | 68.5 | |
| Token DistillationModel=Mistral-7B2025.05 | 68.5 | |
| NTPBackbone=Llama3-8B2025.05 | 67 | |
| NTPModel=Llama3-8B2025.05 | 67 | |
| Tokens to WordsBackbone=Llama3-8B-i2025.05 | 66.6 | |
| ZeTTBackbone=Mistral-7B2025.05 | 66.1 | |
| ZeTTModel=Mistral-7B2025.05 | 66.1 | |
| ZeTTBackbone=Llama3-8B-i2025.05 | 65.2 | |
| ZeTTModel=Llama3-8B-i2025.05 | 65.2 | |
| Tokens to WordsBackbone=Mistral-7B2025.05 | 64.8 | |
| NTPBackbone=Mistral-7B2025.05 | 64.7 | |
| NTPModel=Mistral-7B2025.05 | 64.7 | |
| Tokens to WordsBackbone=Llama3-8B2025.05 | 62.6 | |
| RandomBackbone=Qwen3-8B2025.05 | 62.1 | |
| RandomModel=Qwen3-8B2025.05 | 62.1 | |
| MeanBackbone=Llama3-8B-i2025.05 | 61.7 | |
| MeanModel=Llama3-8B-i2025.05 | 61.7 | |
| MeanBackbone=Avg.2025.05 | 61.5 | |
| MeanModel=Avg.2025.05 | 61.5 | |
| ZeTTBackbone=Llama3-8B2025.05 | 61.3 | |
| ZeTTModel=Llama3-8B2025.05 | 61.3 | |
| MeanBackbone=Llama3-8B2025.05 | 58.4 | |
| MeanModel=Llama3-8B2025.05 | 58.4 | |
| MeanBackbone=Mistral-7B2025.05 | 56.3 | |
| MeanModel=Mistral-7B2025.05 | 56.3 | |
| RandomBackbone=Avg.2025.05 | 52.7 | |
| RandomModel=Avg.2025.05 | 52.7 | |
| RandomBackbone=Llama3-8B-i2025.05 | 51.5 | |
| RandomModel=Llama3-8B-i2025.05 | 51.5 | |
| RandomBackbone=Mistral-7B2025.05 | 51 | |
| RandomModel=Mistral-7B2025.05 | 51 | |
| RandomBackbone=Llama3-8B2025.05 | 46.1 | |
| RandomModel=Llama3-8B2025.05 | 46.1 |