Pedagogical Dimension Evaluation on GRADE BEA 2025 (test)
77Strict F1 (MI)G3-27B GenVer
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| G3-27B GenVerAugmentation Strategy=Gen+Verify, Model Backbone=Gemma3-27B, Authoring Group=Ours2026.05 | 77 | 51 | 57 | 60 | |
| G3-12B NoAugAugmentation Strategy=NoAug, Model Backbone=Gemma3-12B, Authoring Group=Ours2026.05 | 75 | 50 | 58 | 69 | |
| G3-12B GenAugmentation Strategy=Gen, Model Backbone=Gemma3-12B, Authoring Group=Ours2026.05 | 74 | 53 | 55 | 58 | |
| GLM SFTEvaluation Protocol=SFT, Authoring Group=bea-jh2026.05 | 72 | 48 | 59 | 54 | |
| Mathstral EnsEvaluation Protocol=Ensemble, Authoring Group=MSA2026.05 | 72 | 57 | 58 | 69 | |
| GPT4o AugEvaluation Protocol=Augmentation, Authoring Group=TutorMind2026.05 | 70 | — | — | — | |
| Gemini 2.5 Pro ICLEvaluation Protocol=ICL, Authoring Group=BLCU2026.05 | 62 | 68 | 67 | 56 | |
| GLM GRPOEvaluation Protocol=GRPO, Authoring Group=bea-jh2026.05 | 56 | 57 | 57 | 66 | |
| DeepSeek ICLEvaluation Protocol=ICL, Authoring Group=BLCU2026.05 | 51 | 51 | 54 | 47 | |
| Grok ICLEvaluation Protocol=ICL, Authoring Group=BLCU2026.05 | 48 | 39 | 43 | 42 | |
| GPT4o ICLEvaluation Protocol=ICL, Authoring Group=BLCU2026.05 | 46 | 45 | 49 | 44 | |
| Claude ICLEvaluation Protocol=ICL, Authoring Group=BLCU2026.05 | 45 | 49 | 63 | 43 | |
| o3-mini ICLEvaluation Protocol=ICL, Authoring Group=BLCU2026.05 | 43 | 47 | 49 | 45 |