Educational Video Generation on 200-task benchmark
599Success RateAverage
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| AverageEvaluation Protocol=Trace success (ours), Prompt size=599/600 AES/TEA videos2026.05 | 599 | — | — | — | — | — | — | — | — | |
| DeepSeek-V3.1Evaluation Protocol=Trace success (ours), Prompt size=200 AES/TEA videos2026.05 | 200 | — | — | — | — | — | — | — | — | |
| GLM-4.6Evaluation Protocol=Trace success (ours), Prompt size=200 AES/TEA videos2026.05 | 200 | — | — | — | — | — | — | — | — | |
| Qwen3-235BEvaluation Protocol=Trace success (ours), Prompt size=199 AES/TEA videos2026.05 | 199 | — | — | — | — | — | — | — | — | |
| Manim-Direct (No RAG)Evaluation Protocol=Video rendering (e2e), Prompt size=165 videos, ∼ 4k tokens2026.05 | 165 | — | — | — | — | — | — | — | — | |
| Manim-Direct (RAG-enhanced)Evaluation Protocol=Video rendering (e2e), Prompt size=158 videos, ∼ 12k tokens2026.05 | 158 | — | — | — | — | — | — | — | — | |
| Code2Video#Eval=2002026.05 | — | 68.1 | 24 | 13.75 | 13.07 | 15.22 | 15.41 | 17.46 | 74.9 | |
| manim_direct#Eval=1562026.05 | — | 9.22 | 17.4 | 17.35 | 16.48 | 17.93 | 17.9 | 17.87 | 87.52 | |
| manim_direct_novta#Eval=1272026.05 | — | 8.91 | 7.5 | 17.79 | 17.41 | 18.11 | 18.28 | 18.11 | 89.7 | |
| VTA + RSL#Eval=2002026.05 | — | 7.72 | 31.6 | 17.7 | 17.02 | 18.36 | 18.42 | 18.67 | 90.17 | |
| VTA Only#Eval=5992026.05 | — | 8.59 | 20.3 | 17.36 | 16.67 | 18.01 | 18.03 | 18.37 | 88.45 |