Manim Animation Generation on ManimBench 1.0 (test)
85.7Visual SimilarityQwen 3 Coder 30B (A3B) (@3)
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Qwen 3 Coder 30B (A3B) (@3)Inference Strategy=RITL-DOC, Fine-tuning Stage=GRPO, Sampling=@32026.04 | 85.7 | 54.2 | 94 | |
| Qwen 3 Coder 30B (A3B)Inference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 83 | 53.7 | 92 | |
| OpenAI GPT-4.1 (@3)Inference Strategy=RITL-DOC, Fine-tuning Stage=Base, Sampling=@32026.04 | 82.8 | 56.1 | 94 | |
| Qwen 3 Coder 30B (A3B) (@3)Inference Strategy=RITL-DOC, Fine-tuning Stage=SFT, Sampling=@32026.04 | 82.6 | 53.7 | 92 | |
| OpenAI GPT-4.1Inference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 81.9 | 56.6 | 92 | |
| Qwen 3 Coder 30B (A3B)Inference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 80.8 | 54.4 | 89 | |
| Qwen 3 Coder 30B (A3B) (@3)Inference Strategy=RITL-DOC, Fine-tuning Stage=Base, Sampling=@32026.04 | 79.2 | 53.1 | 89 | |
| Qwen 3 Coder 30B (A3B)Inference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 77.7 | 53.5 | 87 | |
| Qwen 3 Coder Next 80BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 73.9 | 54.1 | 85 | |
| Qwen 3 14BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 71 | 55.9 | 79 | |
| SeedCoder 8BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 69.2 | 57.5 | 77 | |
| SeedCoder 8BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 69.2 | 57.4 | 77 | |
| Qwen 3 14BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 68.7 | 55.7 | 76 | |
| Mistral Small 3.2 24BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 68.4 | 55.3 | 77 | |
| Qwen 2.5 Coder 14BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 67.1 | 56.9 | 74 | |
| Qwen 2.5 Coder 14BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 66.4 | 56.9 | 73 | |
| Qwen 2.5 Coder 14BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 66 | 56.6 | 73 | |
| SeedCoder 8BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 65.6 | 57.3 | 73 | |
| Qwen 2.5 Coder 7BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 64.9 | 55.1 | 71 | |
| Mistral Small 3.2 24BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 64.5 | 56.6 | 70 | |
| Qwen 3 14BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 63.6 | 56.5 | 71 | |
| Qwen 2.5 Coder 7BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 63.5 | 55 | 69 | |
| Mistral Small 3.2 24BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 63.2 | 57.3 | 69 | |
| Qwen 2.5 Coder 7BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 60.6 | 54.8 | 66 | |
| Ministral 3 14BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 59.7 | 55.9 | 65 | |
| Ministral 3 14BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 59.6 | 56.2 | 65 | |
| Ministral 3 14BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 56.8 | 52.5 | 64 | |
| Ministral 3 8BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 55.4 | 57.2 | 60 | |
| Ministral 3 8BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 54.6 | 56.9 | 59 | |
| Qwen 3 8BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 51.4 | 56.5 | 58 | |
| Ministral 3 8BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 50.2 | 52.3 | 58 | |
| Qwen 3 8BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 47.4 | 55.7 | 52 | |
| Qwen 3 8BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 41.5 | 55.4 | 47 | |
| Meta LLaMA 3.1 8BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 41.2 | 54.3 | 46 | |
| Meta LLaMA 3.1 8BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 40.2 | 54.3 | 45 | |
| Qwen 2.5 Coder 3BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 40.1 | 54.5 | 44 | |
| Meta LLaMA 3.1 8BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 38.9 | 54.5 | 44 | |
| Qwen 2.5 Coder 3BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 36.2 | 54.4 | 40 | |
| Qwen 3 4BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 31.9 | 54.3 | 35 | |
| Ministral 3 3BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 31.4 | 56 | 35 | |
| Qwen 2.5 Coder 3BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 30.6 | 54.7 | 34 | |
| Ministral 3 3BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 30.5 | 55.4 | 34 | |
| Qwen 3 4BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 30.2 | 54.1 | 33 | |
| Qwen 3 4BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 25.2 | 55 | 28 | |
| Qwen 2.5 Coder 1.5BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 24.7 | 50.8 | 28 | |
| Qwen 2.5 Coder 1.5BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 23 | 51.6 | 26 | |
| Ministral 3 3BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 20.7 | 47.6 | 24 | |
| LLaMA 3.2 3BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 20.5 | 51.2 | 23 | |
| Qwen 2.5 Coder 1.5BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 20.4 | 51.7 | 23 | |
| LLaMA 3.2 3BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 14.3 | 51.5 | 16 | |
| LLaMA 3.2 3BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 12.6 | 52.3 | 14 | |
| LLaMA 3.2 1BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 10 | 48.7 | 11 | |
| LLaMA 3.2 1BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 10 | 45.8 | 11 | |
| LLaMA 3.2 1BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 9.2 | 47.7 | 10 | |
| Qwen 2.5 0.5BInference Strategy=RITL-DOC, Fine-tuning Stage=Base2026.04 | 8.8 | 49.5 | 10 | |
| Qwen 2.5 0.5BInference Strategy=RITL-DOC, Fine-tuning Stage=GRPO2026.04 | 8.8 | 50 | 10 | |
| Qwen 2.5 0.5BInference Strategy=RITL-DOC, Fine-tuning Stage=SFT2026.04 | 7.2 | 48.4 | 8 |