Definition Modeling on Urban
36.16BLEULM-LEXICON-DENSE
Evaluation results
| Method | Links | ||
|---|---|---|---|
| LM-LEXICON-DENSEParameter Count=8B, Mode=BoN-Oracle2026.02 | 36.16 | 38.87 | |
| LM-LEXICON-MOEParameter Count=4x8B, Mode=BoN-Oracle2026.02 | 35.23 | 35.69 | |
| LM-LEXICON-MOEParameter Count=4x8B, Mode=Zero-shot2026.02 | 31.26 | 33.81 | |
| LM-LEXICON-MOEParameter Count=4x8B, Mode=BoN-ORM2026.02 | 31.08 | 33.26 | |
| LM-LEXICON-DENSEParameter Count=8B, Mode=BoN-ORM2026.02 | 26.73 | 28.54 | |
| LM-LEXICON-DENSEParameter Count=8B, Mode=Zero-shot2026.02 | 26.09 | 28.35 | |
| GPT-4-TURBOIn-Context Learning=Retrieval-ICL, shots=322026.02 | 22.53 | 26.53 | |
| Rerank-T5Supervised training=true2026.02 | 17.77 | 18.25 | |
| Contrast-T5Supervised training=true2026.02 | 17.53 | 16.34 | |
| MDM-T5Supervised training=true2026.02 | 17.53 | 17.18 | |
| LlamaDictSupervised training=true2026.02 | 15.7 | 14.51 | |
| SimpDefinerSupervised training=true2026.02 | 13.54 | 15.37 | |
| GPT-4-TURBOIn-Context Learning=Random-ICL, shots=322026.02 | 11.08 | 12.19 | |
| GEMINI-1.5-PROIn-Context Learning=Retrieval-ICL, shots=322026.02 | 8.85 | 9.18 | |
| GEMINI-1.5-PROIn-Context Learning=Random-ICL, shots=322026.02 | 8.44 | 9.59 | |
| CLAUDE-3-OPUSIn-Context Learning=Retrieval-ICL, shots=322026.02 | 5.93 | 7.19 | |
| Flan-T5-DefSupervised training=true2026.02 | 5.33 | 10.61 | |
| CLAUDE-3-OPUSIn-Context Learning=Random-ICL, shots=322026.02 | 1.59 | 3.08 |