CodeWorkout
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
CodeWorkout
72.32Accuracy
49
CodeWorkout (5-fold averaged)
0.593CodeBLEU
49
CodeWorkout (test)
41.4Exp.% (med)
7
CodeWorkout Java
0.816AUC
6
CodeWorkout (test)
82.4A1 Performance Score
4
CodeWorkout Java
58CodeBLEU
3
CodeWorkout Selected KC set (test)
63.1AUC
3
CodeWorkout Generated KC set (test)
0.676AUC
3
CodeWorkout Human KC set (test)
66.5AUC
3
CodeWorkout Selected KC set (test)
0.069RMSE
3
CodeWorkout Generated KC set (test)
0.07RMSE
3
CodeWorkout Human KC set (test)
0.077RMSE
3