Validation timing detection on Human-validated Japanese
70.75Overall Macro-F1Human
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Human2026.06 | 70.75 | 65.57 | 92.67 | 76.8 | |
| ModernBERT2026.06 | 61.8 | 59.46 | 88 | 70.97 | |
| MEGUMI2026.06 | 57 | 56.86 | 58 | 57.43 | |
| XLM-RoBERTa2026.06 | 55.93 | 55.56 | 60 | 57.69 | |
| BERT2026.06 | 53.7 | 53.45 | 62 | 57.41 | |
| GPT 4o NanoEvaluation Protocol=Zero-shot2026.06 | 53.45 | 53.6 | 47.6 | 50.4 | |
| Random Baseline2026.06 | 47 | 47.17 | 50 | 48.54 | |
| GPT 4o NanoEvaluation Protocol=CoT2026.06 | 45.74 | 52.65 | 95.6 | 67.9 | |
| GPT 4o NanoEvaluation Protocol=3-shot2026.06 | 44.93 | 52.29 | 92 | 66.67 | |
| Llama 3.1 8bEvaluation Protocol=3-shot2026.06 | 38.83 | 51.57 | 98.8 | 67.77 | |
| Llama 3.1 8bEvaluation Protocol=LoRA2026.06 | 37.63 | 52.02 | 98.5 | 68.08 | |
| Llama 3.1 8bEvaluation Protocol=Zero-shot2026.06 | 35.36 | 50.62 | 98.4 | 66.85 |