Machine Translation (BLEU) on Flores
40.91BLEU ScoreBefore RL
Evaluation Results
| Method | Links | |
|---|---|---|
| Before RLModel=Qwen3-30B-A3B2026.05 | 40.91 | |
| Vanilla RLModel=Qwen3-30B-A3B2026.05 | 40.87 | |
| RL+SAEModel=Qwen3-30B-A3B2026.05 | 40.84 | |
| RL+SAEModel=Qwen3-8B2026.05 | 38.74 | |
| Vanilla RLModel=Qwen3-8B2026.05 | 38.32 | |
| Before RLModel=Qwen3-8B2026.05 | 38.18 | |
| SFTModel=Gemma-2-9B, Dataset=Russian 110k2025.07 | 31.82 | |
| SFT+GRPOModel=Gemma-2-9B, Dataset=Russian 110k2025.07 | 31.55 | |
| SASFTModel=Gemma-2-9B, Dataset=Russian 110k2025.07 | 31.31 | |
| RL+SAEModel=Qwen3-1.7B2026.05 | 31.06 | |
| SFT+GRPOModel=Gemma-2-9B2025.07 | 30.71 | |
| SFTModel=Gemma-2-9B2025.07 | 30.59 | |
| SFT+PenaltyModel=Gemma-2-9B, Dataset=Russian 110k2025.07 | 30.32 | |
| SFT+GRPOModel=Qwen3-8B-Base2025.07 | 30.14 | |
| SFT+GRPOModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 30.14 | |
| SFT+PenaltyModel=Qwen3-8B-Base2025.07 | 30.1 | |
| SFT+PenaltyModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 30.1 | |
| SFT+PenaltyModel=Gemma-2-9B2025.07 | 30.09 | |
| SFTModel=Qwen3-8B-Base2025.07 | 29.99 | |
| SFTModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 29.99 | |
| SASFTModel=Qwen3-8B-Base2025.07 | 29.97 | |
| SASFTModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 29.97 | |
| SFT+GRPOModel=Gemma-2-9B2025.07 | 29.68 | |
| Vanilla RLModel=Qwen3-1.7B2026.05 | 29.44 | |
| SASFTModel=Gemma-2-9B2025.07 | 29.41 | |
| SFT+PenaltyBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.07 | 29.29 | |
| SFTModel=Gemma-2-9B2025.07 | 29.23 | |
| SFT+PenaltyModel=Gemma-2-9B2025.07 | 29.18 | |
| SFTBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.07 | 29.15 | |
| SASFTBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.07 | 28.87 | |
| SFT+GRPOBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.07 | 28.68 | |
| Before RLModel=Qwen3-1.7B2026.05 | 28.47 | |
| SASFTModel=Gemma-2-9B2025.07 | 28.19 | |
| SFT+GRPOBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 27.07 | |
| SASFTModel=Qwen3-8B-Base, Dataset=Russian 110k2025.07 | 26.95 | |
| SFT+PenaltyModel=Qwen3-8B-Base, Dataset=Russian 110k2025.07 | 26.89 | |
| SFT+GRPOModel=Qwen3-8B-Base, Dataset=Russian 110k2025.07 | 26.86 | |
| SFTBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 26.37 | |
| SFT+PenaltyModel=Qwen3-8B-Base2025.07 | 26.12 | |
| SFTModel=Qwen3-8B-Base, Dataset=Russian 110k2025.07 | 25.77 | |
| SFT+PenaltyModel=Llama-3.1-8B, Dataset=Russian 110k2025.07 | 25.49 | |
| SFT+PenaltyBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 24.83 | |
| SFT+GRPOModel=Qwen3-8B-Base2025.07 | 24.81 | |
| SFTModel=Llama-3.1-8B2025.07 | 24.73 | |
| SFT+PenaltyModel=Llama-3.1-8B2025.07 | 24.68 | |
| SASFTModel=Qwen3-1.7B-Base, Dataset=Russian 110k2025.07 | 24.56 | |
| SFT+PenaltyModel=Llama-3.1-8B2025.07 | 24.36 | |
| SASFTModel=Llama-3.1-8B2025.07 | 24.05 | |
| SFT+GRPOModel=Qwen3-1.7B-Base, Dataset=Russian 110k2025.07 | 23.84 | |
| SFT+GRPOBackbone=Llama-3.1-8B, Training dataset setting=Korean 210k2025.07 | 23.79 | |
| SASFTModel=Llama-3.1-8B2025.07 | 23.73 | |
| SFTModel=Qwen3-1.7B-Base2025.07 | 23.7 | |
| SFTModel=Qwen3-1.7B-Base, Training Dataset=Chinese 110k2026.05 | 23.7 | |
| SFTBackbone=Llama-3.1-8B, Training dataset setting=Korean 210k2025.07 | 23.68 | |
| SFT+PenaltyModel=Qwen3-1.7B-Base, Dataset=Russian 110k2025.07 | 23.68 | |
| SASFTModel=Qwen3-1.7B-Base2025.07 | 23.67 | |
| SASFTModel=Qwen3-1.7B-Base, Training Dataset=Chinese 110k2026.05 | 23.67 | |
| SFT+PenaltyModel=Qwen3-1.7B-Base2025.07 | 23.55 | |
| SFT+PenaltyModel=Qwen3-1.7B-Base, Training Dataset=Chinese 110k2026.05 | 23.55 | |
| SFT+GRPOModel=Gemma-2-2B, Dataset=Russian 110k2025.07 | 23.54 | |
| SASFTModel=Llama-3.1-8B, Dataset=Russian 110k2025.07 | 23.52 | |
| SFTModel=Qwen3-1.7B-Base, Dataset=Russian 110k2025.07 | 23.46 | |
| SFT+GRPOModel=Qwen3-1.7B-Base2025.07 | 23.45 | |
| SFT+GRPOModel=Qwen3-1.7B-Base, Training Dataset=Chinese 110k2026.05 | 23.45 | |
| SFT+PenaltyBackbone=Llama-3.1-8B, Training dataset setting=Korean 210k2025.07 | 23.44 | |
| SASFTModel=Gemma-2-2B, Dataset=Russian 110k2025.07 | 23.31 | |
| SASFTBackbone=Llama-3.1-8B, Training dataset setting=Korean 210k2025.07 | 23.25 | |
| SFTModel=Gemma-2-2B, Dataset=Russian 110k2025.07 | 23.25 | |
| SFT+GRPOModel=Qwen3-1.7B-Base2025.07 | 23.04 | |
| SASFTModel=Qwen3-1.7B-Base2025.07 | 22.97 | |
| SFT+PenaltyModel=Gemma-2-2B2025.07 | 22.87 | |
| SFTModel=Qwen3-8B-Base2025.07 | 22.86 | |
| SFTModel=Llama-3.1-8B2025.07 | 22.81 | |
| SFT+GRPOModel=Gemma-2-2B2025.07 | 22.8 | |
| SFT+PenaltyModel=Qwen3-1.7B-Base2025.07 | 22.58 | |
| SFTModel=Gemma-2-2B2025.07 | 22.56 | |
| SASFTBackbone=Qwen3-1.7B-Base, Training dataset setting=Korean 210k2025.07 | 22.55 | |
| SFT+PenaltyBackbone=Qwen3-1.7B-Base, Training dataset setting=Korean 210k2025.07 | 22.5 | |
| SFTModel=Qwen3-1.7B-Base2025.07 | 22.49 | |
| SFT+GRPOBackbone=Qwen3-1.7B-Base, Training dataset setting=Korean 210k2025.07 | 22.44 | |
| SFTBackbone=Qwen3-1.7B-Base, Training dataset setting=Korean 210k2025.07 | 22.4 | |
| SFT+GRPOModel=Llama-3.1-8B2025.07 | 22.34 | |
| SASFTModel=Gemma-2-2B2025.07 | 22.28 | |
| SFT+PenaltyModel=Gemma-2-2B, Dataset=Russian 110k2025.07 | 22.18 | |
| SFT+GRPOModel=Llama-3.1-8B, Dataset=Russian 110k2025.07 | 21.82 | |
| SASFTBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 21.8 | |
| SFTModel=Llama-3.1-8B, Dataset=Russian 110k2025.07 | 21.45 | |
| SFT+GRPOModel=Llama-3.1-8B2025.07 | 20.75 | |
| SFT+GRPOBackbone=Gemma-2-2B, Training dataset setting=Korean 210k2025.07 | 19.35 | |
| SFTBackbone=Gemma-2-2B, Training dataset setting=Korean 210k2025.07 | 19.31 | |
| SASFTModel=Qwen3-8B-Base2025.07 | 18.96 | |
| SFT+GRPOModel=Gemma-2-2B2025.07 | 18.49 | |
| SFT+PenaltyModel=Gemma-2-2B2025.07 | 18.48 | |
| SFTModel=Gemma-2-2B2025.07 | 18.39 | |
| SASFTBackbone=Gemma-2-2B, Training dataset setting=Korean 210k2025.07 | 18.34 | |
| SASFTModel=Gemma-2-2B2025.07 | 17.96 | |
| SFT+PenaltyBackbone=Gemma-2-2B, Training dataset setting=Korean 210k2025.07 | 15.45 |