Multilingual Evaluation on TAXI1500
75.7HLFRMACRO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MACROModel=Gemma3-12B, Language=en2026.05 | 75.7 | 0.657 | |
| DG-CFModel=Gemma3-12B, Language=en2026.05 | 73.6 | 0.731 | |
| MACROModel=Gemma3-12B, Language=de2026.05 | 69.4 | 0.631 | |
| MACROModel=Gemma3-12B, Language=ru2026.05 | 68.5 | 0.693 | |
| DG-CFModel=Gemma3-12B, Language=de2026.05 | 67.8 | 0.686 | |
| MACROModel=Gemma3-12B, Language=vi2026.05 | 67.6 | 0.655 | |
| DG-CFModel=Gemma3-12B, Language=ar2026.05 | 67.2 | 0.699 | |
| DG-CFModel=Gemma3-12B, Language=vi2026.05 | 65 | 0.707 | |
| DG-CFModel=Gemma3-12B, Language=ru2026.05 | 64.3 | 0.743 | |
| MACROModel=Gemma3-12B, Language=sw2026.05 | 64 | 0.634 | |
| MACROModel=Gemma3-12B, Language=zh2026.05 | 63.1 | 0.973 | |
| MACROModel=Gemma3-12B, Language=ar2026.05 | 63.1 | 0.67 | |
| DG-CFModel=Gemma3-12B, Language=zh2026.05 | 60.5 | 0.85 | |
| DG-CFModel=Gemma3-12B, Language=sw2026.05 | 57.8 | 0.653 | |
| MACROModel=Qwen3-8B, Language=en2026.05 | 53.2 | 0.468 | |
| MACROModel=Gemma3-4B, Language=en2026.05 | 51.4 | 0.606 | |
| MACROModel=Qwen3-8B, Language=vi2026.05 | 51.4 | 0.601 | |
| MACROModel=Qwen3-4B, Language=en2026.05 | 50.5 | 0.439 | |
| MACROModel=Qwen3-8B, Language=de2026.05 | 50.5 | 0.544 | |
| DG-CFModel=Qwen3-4B, Language=en2026.05 | 48.5 | 0.48 | |
| MACROModel=Gemma3-4B, Language=de2026.05 | 45 | 0.551 | |
| MACROModel=Qwen3-8B, Language=ar2026.05 | 44.1 | 0.638 | |
| DG-CFModel=Gemma3-4B, Language=en2026.05 | 43.4 | 0.67 | |
| MACROModel=Gemma3-4B, Language=ru2026.05 | 43.2 | 0.666 | |
| MACROModel=Qwen3-4B, Language=ru2026.05 | 41.4 | 0.518 | |
| DG-CFModel=Gemma3-4B, Language=de2026.05 | 40.5 | 0.58 | |
| DG-CFModel=Gemma3-4B, Language=zh2026.05 | 40.5 | 0.807 | |
| DG-CFModel=Qwen3-8B, Language=vi2026.05 | 40.5 | 0.621 | |
| MACROModel=Qwen3-8B, Language=ru2026.05 | 39.6 | 0.591 | |
| DG-CFModel=Gemma3-4B, Language=ru2026.05 | 39.4 | 0.65 | |
| MACROModel=Qwen3-4B, Language=zh2026.05 | 38.7 | 0.769 | |
| DG-CFModel=Qwen3-8B, Language=en2026.05 | 38.7 | 0.474 | |
| DG-CFModel=Qwen3-8B, Language=ar2026.05 | 38.7 | 0.622 | |
| MACROModel=Qwen3-4B, Language=ar2026.05 | 37.8 | 0.556 | |
| MACROModel=Qwen3-8B, Language=zh2026.05 | 37.8 | 0.865 | |
| DG-CFModel=Gemma3-4B, Language=vi2026.05 | 37.7 | 0.631 | |
| MACROModel=Gemma3-4B, Language=zh2026.05 | 36.9 | 0.961 | |
| MACROModel=Gemma3-4B, Language=vi2026.05 | 36.9 | 0.61 | |
| MACROModel=Gemma3-4B, Language=sw2026.05 | 36 | 0.609 | |
| DG-CFModel=Qwen3-8B, Language=zh2026.05 | 36 | 0.689 | |
| DG-CFModel=Gemma3-4B, Language=sw2026.05 | 35.7 | 0.647 | |
| DG-CFModel=Qwen3-4B, Language=vi2026.05 | 35.6 | 0.485 | |
| MACROModel=Gemma3-4B, Language=ar2026.05 | 35.1 | 0.659 | |
| MACROModel=Qwen3-4B, Language=vi2026.05 | 35.1 | 0.522 | |
| DG-CFModel=Gemma3-4B, Language=ar2026.05 | 34.2 | 0.716 | |
| DG-CFModel=Qwen3-4B, Language=ar2026.05 | 34.2 | 0.615 | |
| DG-CFModel=Qwen3-8B, Language=de2026.05 | 34.2 | 0.562 | |
| MACROModel=Qwen3-4B, Language=de2026.05 | 33.3 | 0.439 | |
| DG-CFModel=Qwen3-4B, Language=ru2026.05 | 31.5 | 0.526 | |
| DG-CFModel=Qwen3-4B, Language=zh2026.05 | 30.9 | 0.589 | |
| DG-CFModel=Qwen3-4B, Language=de2026.05 | 28.4 | 0.444 | |
| DG-CFModel=Qwen3-8B, Language=ru2026.05 | 27 | 0.566 | |
| DG-CFModel=Qwen3-4B, Language=sw2026.05 | 10.7 | 0.312 | |
| MACROModel=Qwen3-4B, Language=sw2026.05 | 9 | 0.32 | |
| MACROModel=Qwen3-8B, Language=sw2026.05 | 8.1 | 1.25 | |
| DG-CFModel=Qwen3-8B, Language=sw2026.05 | 0.9 | 0.693 |