Rubric-based scoring on NCTE 1-7 scoring scale (transcript segments)
0.96MAEConstant Baseline
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Constant BaselineInference protocol=Median prediction for MAE and mean prediction for MSE2026.04 | 0.96 | 1.35 | |
| DeBERTaV3 largeFine-tuning status=Fine-Tuned2026.04 | 0.96 | 1.31 | |
| BERT largeFine-tuning status=Fine-Tuned2026.04 | 0.97 | 1.34 | |
| RoBERTa baseFine-tuning status=Fine-Tuned2026.04 | 0.97 | 1.34 | |
| RoBERTa largeFine-tuning status=Fine-Tuned2026.04 | 0.97 | 1.37 | |
| ALBERT baseFine-tuning status=Fine-Tuned2026.04 | 0.98 | 1.34 | |
| BERT baseFine-tuning status=Fine-Tuned2026.04 | 0.98 | 1.36 | |
| ALBERT largeFine-tuning status=Fine-Tuned2026.04 | 0.99 | 1.45 | |
| DeBERTaV3 baseFine-tuning status=Fine-Tuned2026.04 | 1 | 1.56 | |
| Mistral Small InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.02 | 1.78 | |
| Qwen2 4B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.18 | 2.29 | |
| Mixtral 8x22B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.21 | 2.41 | |
| Mixtral 8x7B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.39 | 2.85 | |
| Qwen2 30B A3B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.56 | 3.59 | |
| Llama 3.1 8B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.63 | 3.98 | |
| Qwen2 235B A22B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.67 | 4.16 | |
| Mistral Small 24B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.75 | 4.28 | |
| Llama 3.1 70B InstructFine-tuning status=Non-Fine-Tuned, Inference protocol=Prompted inference2026.04 | 1.98 | 5.32 | |
| RoBERTa largeFine-tuning status=Non-Fine-Tuned2026.04 | 3.27 | 12.01 | |
| DeBERTaV3 largeFine-tuning status=Non-Fine-Tuned2026.04 | 3.66 | 14.65 | |
| DeBERTaV3 baseFine-tuning status=Non-Fine-Tuned2026.04 | 4.08 | 17.97 | |
| ALBERT largeFine-tuning status=Non-Fine-Tuned2026.04 | 4.22 | 19.1 | |
| RoBERTa baseFine-tuning status=Non-Fine-Tuned2026.04 | 4.33 | 20.08 | |
| ALBERT baseFine-tuning status=Non-Fine-Tuned2026.04 | 4.34 | 20.12 | |
| BERT largeFine-tuning status=Non-Fine-Tuned2026.04 | 4.45 | 21.07 | |
| BERT baseFine-tuning status=Non-Fine-Tuned2026.04 | 4.58 | 22.29 |