Punctuation Restoration on IWSLT English (test)
84F1 (Comma)Llama-3.2-1B Scoring
Evaluation Results
| Method | Links | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Llama-3.2-1B ScoringInference Setting=Scoring, Evaluation Protocol=Fine-tuned, K (Lookahead context budget)=22026.04 | 84 | 98.6 | 93.5 | 93.7 | 98.7 | 98.6 | 98.7 | 83.6 | 84.4 | 98.9 | 98.3 | 94.9 | 92.2 | 94 | 93.4 | |
| Llama-3.2-1B ScoringInference Setting=Scoring, Evaluation Protocol=Fine-tuned, K (Lookahead context budget)=12026.04 | 80.4 | 98.7 | 93.5 | 92.7 | 98.4 | 98.4 | 98.4 | 79.9 | 80.8 | 98.9 | 98.6 | 95.6 | 91.4 | 93.2 | 92.3 | |
| Llama-3.2-1B ScoringInference Setting=Scoring, Evaluation Protocol=No-fine-tuning, K (Lookahead context budget)=22026.04 | 79 | 93.2 | 87.1 | 89.3 | 98.5 | 97.3 | 97.9 | 73.1 | 86 | 95.3 | 91.2 | 85.5 | 88.8 | 88.1 | 90.8 | |
| Llama-3.2-1B ScoringInference Setting=Scoring, Evaluation Protocol=No-fine-tuning, K (Lookahead context budget)=12026.04 | 75.3 | 94.6 | 86.8 | 88.6 | 98.4 | 97.1 | 97.7 | 69.9 | 81.5 | 94.9 | 94.2 | 84.8 | 88.8 | 87 | 90.4 | |
| ELECTRA-SmallEvaluation Protocol=Fine-tuned, K (Lookahead context budget)=22026.04 | 75.2 | 99.2 | 92.5 | 91.3 | 97.7 | 98.5 | 98.1 | 79.7 | 71.3 | 99.2 | 99.2 | 93.3 | 91.7 | 92.5 | 90.2 | |
| WtPFINETUNEEvaluation Protocol=Fully fine-tuned, Number of Layers=122023.05 | 69.8 | 82.7 | 77.9 | 63.4 | — | — | — | — | — | — | — | — | — | — | — | |
| CANINE-SEvaluation Protocol=Fine-tuned2023.05 | 66.6 | 80.2 | 72.3 | 57.7 | — | — | — | — | — | — | — | — | — | — | — | |
| WtPPUNCTEvaluation Protocol=Logistic regression on punctuation logits, Number of Layers=122023.05 | 61.3 | 76 | 73.3 | 60.3 | — | — | — | — | — | — | — | — | — | — | — | |
| Llama-3.2-1B-InstructInference Setting=Prompt, Evaluation Protocol=No-fine-tuning2026.04 | 45.7 | 69.9 | 62.2 | 56.6 | 97.5 | 80.2 | 88 | 39.7 | 53.7 | 99.8 | 53.8 | 79.6 | 50.5 | 54.1 | 59.6 |