Machine Reading Comprehension on CMRC
96.71F1 ScoreC-DPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| C-DPOSize=70B, Evaluation Protocol=zero-shot2024.09 | 96.71 | — | |
| C-SFTSize=70B, Evaluation Protocol=zero-shot2024.09 | 96.52 | — | |
| Token-Opt Trans.Training Setting=CMRC-Trained, Evaluation Protocol=Source Language Fine-tuning2025.09 | 93.6 | 88.4 | |
| C-DPOSize=8B, Evaluation Protocol=zero-shot2024.09 | 93.48 | — | |
| Basic Trans.Training Setting=CMRC-Trained, Evaluation Protocol=Source Language Fine-tuning2025.09 | 92.7 | 87.7 | |
| C-SFTSize=8B, Evaluation Protocol=zero-shot2024.09 | 92.26 | — | |
| Llama-3 InstructSize=70B, Evaluation Protocol=zero-shot2024.09 | 91.21 | — | |
| Expanded VocabTraining Setting=CMRC-Trained, Evaluation Protocol=Source Language Fine-tuning2025.09 | 91.1 | 81.3 | |
| Hybrid Trans.Training Setting=CMRC-Trained, Evaluation Protocol=Source Language Fine-tuning2025.09 | 90.2 | 83.1 | |
| Base Llama2Training Setting=CMRC-Trained, Evaluation Protocol=Source Language Fine-tuning2025.09 | 89.5 | 77.2 | |
| UROMANTraining Setting=CMRC-Trained, Evaluation Protocol=Source Language Fine-tuning2025.09 | 88.3 | 79.6 | |
| Llama-3 InstructSize=8B, Evaluation Protocol=zero-shot2024.09 | 87.69 | — |