Definition Generation on Oxford
30.87BLEULM-LEXICON-MOE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LM-LEXICON-MOEParameter Count=4x8B, Mode=BoN-Oracle2026.02 | 30.87 | 43.24 | — | |
| LM-LEXICON-DENSEParameter Count=8B, Mode=BoN-Oracle2026.02 | 30.07 | 42.78 | — | |
| GEMINI-1.5-PROIn-Context Learning=Retrieval-ICL, shots=322026.02 | 28.1 | 36.98 | — | |
| LM-LEXICON-DENSEParameter Count=8B, Mode=BoN-ORM2026.02 | 26.74 | 35.18 | — | |
| Huang et al. (2021)Test=Unknown2023.05 | 26.52 | — | — | |
| LM-LEXICON-DENSEParameter Count=8B, Mode=Zero-shot2026.02 | 26.09 | 34.55 | — | |
| Rerank-T5Supervised training=true2026.02 | 25.56 | 28 | — | |
| GEMINI-1.5-PROIn-Context Learning=Random-ICL, shots=322026.02 | 25.51 | 35.97 | — | |
| LM-LEXICON-MOEParameter Count=4x8B, Mode=BoN-ORM2026.02 | 24.18 | 33.79 | — | |
| MDM-T5Supervised training=true2026.02 | 24.16 | 27.68 | — | |
| SimpDefinerSupervised training=true2026.02 | 23.48 | 29.59 | — | |
| LM-LEXICON-MOEParameter Count=4x8B, Mode=Zero-shot2026.02 | 23.35 | 32.94 | — | |
| LlamaDictSupervised training=true2026.02 | 22.77 | 36.46 | — | |
| Contrast-T5Supervised training=true2026.02 | 22.51 | 28.18 | — | |
| GPT-4-TURBOIn-Context Learning=Random-ICL, shots=322026.02 | 21.93 | 30.82 | — | |
| Flan-T5-DefSupervised training=true2026.02 | 21.34 | 32.39 | — | |
| GPT-4-TURBOIn-Context Learning=Retrieval-ICL, shots=322026.02 | 20.44 | 34.35 | — | |
| CLAUDE-3-OPUSIn-Context Learning=Random-ICL, shots=322026.02 | 19.99 | 34.21 | — | |
| Flan-T5 XLTest=Soft domain shift2023.05 | 18.69 | 38.72 | 89.75 | |
| Flan-T5 XLTest=In-distribution2023.05 | 16.61 | 36.27 | 89.4 | |
| CLAUDE-3-OPUSIn-Context Learning=Retrieval-ICL, shots=322026.02 | 15.51 | 25.99 | — | |
| Flan-T5 XLTest=Hard domain shift2023.05 | 8.37 | 25.06 | 87.56 | |
| Flan-T5 XLTest=Zero-shot (task shift)2023.05 | 2.88 | 16.2 | 86.52 |