Turn-level student simulation on Math tutoring dataset (test)
64.4ActsProfileRL
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| ProfileRLSimulator Base Model=Llama-3.1-8B-Instruct2026.05 | 64.4 | 0.516 | 14.9 | 0.7 | 29.6 | |
| History SFTSimulator Base Model=Llama-3.1-8B-Instruct2026.05 | 60.2 | 0.479 | 11.1 | 0.694 | 28.7 | |
| DPOSimulator Base Model=Llama-3.1-8B-Instruct2026.05 | 59.3 | 0.474 | 10.8 | 0.68 | 25.9 | |
| SFTSimulator Base Model=Llama-3.1-8B-Instruct2026.05 | 58.6 | 0.465 | 10.2 | 0.68 | 26.1 | |
| History ICLSimulator Base Model=GPT-5.42026.05 | 49.5 | 0.466 | 9.4 | 0.531 | 17.6 | |
| Cognitive ProfileSimulator Base Model=GPT-5.42026.05 | 47.5 | 0.461 | 10.5 | 0.503 | 15 | |
| OCEAN PersonaSimulator Base Model=GPT-5.42026.05 | 44.3 | 0.456 | 7.9 | 0.465 | 12 | |
| Knowledge ProfileSimulator Base Model=GPT-5.42026.05 | 42.7 | 0.438 | 8.6 | 0.446 | 10.4 |