Learner Dictionary Definition Generation on D3J (test)
93.5TruthfulnessGPT 5.1
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT 5.1prompting_strategy=Zero-shot2026.01 | 93.5 | 86.1 | 97.6 | 57.1 | 96.3 | |
| GPT 5.1prompting_strategy=Few-shot, runs=32026.01 | 93.2 | 87.8 | 96.7 | 65.5 | 95.6 | |
| Claude Sonnet 4.5prompting_strategy=Few-shot, runs=32026.01 | 91.5 | 90.8 | 91.9 | 88 | 92 | |
| Claude Sonnet 4.5prompting_strategy=Zero-shot2026.01 | 89.5 | 87.2 | 94.6 | 73.4 | 91.2 | |
| Wiktionarysource=Human-written reference2026.01 | 69.7 | — | 90.8 | 76.3 | — | |
| Llama 3.3 Swallowprompting_strategy=Zero-shot2026.01 | 68.9 | 65.9 | 56 | 66.9 | 71.9 | |
| Qwen3prompting_strategy=Zero-shot2026.01 | 62.4 | 57.9 | 74.2 | 42.7 | 52.3 | |
| Qwen3prompting_strategy=Few-shot, runs=32026.01 | 61.8 | 60.7 | 67 | 49 | 65.2 | |
| Llama 3.3 Swallowprompting_strategy=Few-shot, runs=32026.01 | 60.8 | 63.8 | 51.9 | 72.2 | 70.5 |