MCEval
Benchmarks
TaskDatasetSOTA resultTrendResults
MCEval mLAMA 8K (test)
79.5Accuracy
14
MCEval HaluEval 8K (test)
82.2Accuracy
14
MCEval CSQA 8K (test)
84.6Accuracy
14
MCEval PAWS 8K (test)
88.9Accuracy
14
MCEval Agnews 8K (test)
88.3Accuracy
14
MCEval NER 8K (test)
0.877Accuracy
14
MCEval 1.0 (test)
87.8Real Style Score
12
McEval Multi-line
40JavaScript Pass@1
10
McEval Single-line
78.8JavaScript Pass@1
10