Machine Translation on FLORES-200 (xx→en)
97.68XCOMETClaude-3.5-Sonnet
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Claude-3.5-Sonnet2025.05 | 97.68 | 89.41 | |
| Gemini-1.5-Pro2025.05 | 97.32 | 88.15 | |
| Qwen2.5-72B-Instruct2025.05 | 97.28 | 88.9 | |
| Qwen3-32BReasoning/Thinking Mode=true2025.05 | 97.18 | 88.88 | |
| SAMASBase LLM=GPT-5, Routing Method=SAMAS (Dynamic)2026.02 | 97.1 | 90.43 | |
| Gemma2-27B-it2025.05 | 96.72 | 88.64 | |
| GPT-5Base LLM=GPT-5, Routing Method=Single-LLM2026.02 | 96.7 | 88.61 | |
| SSR-Zero-7B2025.05 | 96.7 | 87.97 | |
| TACTICRouting Method=Multi-Agent2026.02 | 96.69 | 90.15 | |
| SSR-X-Zero-7BReasoning/Thinking Mode=true2025.05 | 96.62 | 88.84 | |
| Qwen3-8BReasoning/Thinking Mode=true2025.05 | 96.55 | 88.61 | |
| GPT-4o2025.05 | 96.5 | 88.94 | |
| GemmaX2-28-9B-v0.12025.05 | 96.33 | 87.85 | |
| Gemma2-9B-it2025.05 | 96.25 | 88.32 | |
| SAMASBase LLM=GPT-5, Routing Method=Faulkner-esque Style2026.02 | 96.06 | 88.08 | |
| SAMASBase LLM=Qwen3-30B-A3B, Routing Method=SAMAS (Dynamic)2026.02 | 95.96 | 90.15 | |
| SAMASBase LLM=Qwen3-235B-A22B, Routing Method=SAMAS (Dynamic)2026.02 | 95.86 | 90.84 | |
| SAMASBase LLM=Qwen3-30B-A3B, Routing Method=Faulkner-esque Style2026.02 | 95.82 | 88.26 | |
| TowerInstruct-13B-v0.12025.05 | 95.8 | 87.3 | |
| Qwen3-32B2025.05 | 95.75 | 87.68 | |
| GPT-4.1Base LLM=GPT-4.1, Routing Method=Single-LLM2026.02 | 95.71 | 88.2 | |
| SAMASBase LLM=Qwen-mt-plu, Routing Method=SAMAS (Dynamic)2026.02 | 95.7 | 88.19 | |
| SAMASBase LLM=Qwen-mt-plu, Routing Method=Faulkner-esque Style2026.02 | 95.65 | 88.17 | |
| SSR-Zero-3B2025.05 | 95.64 | 87.23 | |
| Qwen2.5-32B-Instruct2025.05 | 95.5 | 87.13 | |
| SAMASBase LLM=Qwen3-30B-A3B, Routing Method=Hemingway-esque Style2026.02 | 95.37 | 88 | |
| SAMASBase LLM=Qwen3-235B-A22B, Routing Method=Faulkner-esque Style2026.02 | 95.22 | 89.39 | |
| DeepSeek-V3Base LLM=DeepSeek-V3, Routing Method=Single-LLM2026.02 | 95.14 | 88.45 | |
| DeepSeek-R1Base LLM=DeepSeek-R1, Routing Method=Single-LLM2026.02 | 95.13 | 88.36 | |
| SAMASBase LLM=Qwen-mt-plu, Routing Method=Hemingway-esque Style2026.02 | 95.12 | 88.18 | |
| Gemini-2.5-proBase LLM=Gemini-2.5-pro, Routing Method=Single-LLM2026.02 | 95.02 | 87.82 | |
| TowerInstruct-7B-v0.22025.05 | 94.88 | 86.95 | |
| Qwen-mt-pluBase LLM=Qwen-mt-plu, Routing Method=Single-LLM2026.02 | 94.43 | 87.43 | |
| TransAgentsRouting Method=Multi-Agent2026.02 | 94.31 | 87.28 | |
| MAATSRouting Method=Multi-Agent2026.02 | 94.09 | 87.21 | |
| Qwen2.5-7B-Instruct2025.05 | 94.06 | 86.28 | |
| Qwen3-235B-A22BBase LLM=Qwen3-235B-A22B, Routing Method=Single-LLM2026.02 | 94.05 | 87.17 | |
| SAMASBase LLM=Qwen3-235B-A22B, Routing Method=Hemingway-esque Style2026.02 | 94.01 | 89.8 | |
| SAMASBase LLM=GPT-5, Routing Method=Hemingway-esque Style2026.02 | 93.96 | 89.23 | |
| Qwen3-8B2025.05 | 93.89 | 85.82 | |
| Qwen2.5-3B-Instruct2025.05 | 93.41 | 85.46 | |
| Qwen3-30B-A3BBase LLM=Qwen3-30B-A3B, Routing Method=Single-LLM2026.02 | 91.55 | 85.12 | |
| HiMATERouting Method=Multi-Agent2026.02 | 91.06 | 85 | |
| QwQ-32BReasoning/Thinking Mode=true2025.05 | 89.21 | 80.88 | |
| Qwen2.5-7B2025.05 | 85.54 | 73.62 | |
| Qwen2.5-3B2025.05 | 65.94 | 51.8 |