Character Description Generation on CroSS (test)
28.49PRISMA ScoreGPT-4.1-mini
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| GPT-4.1-miniContext strategy=Full context, Trace=—, Desc=ZS2026.04 | 28.49 | 16.08 | 72.67 | 28.49 | 16.8 | |
| Qwen-3-8b (guided-QA)Context strategy=Lead-128k + guided-QA, Trace=GRPO, Desc=ZS2026.04 | 23.78 | 14.42 | 68.09 | 36.18 | 17.49 | |
| Qwen-3-8b (guided-QA)Context strategy=Lead-128k + guided-QA, Trace=ZS, Desc=ZS2026.04 | 23.72 | 14.62 | 68.57 | 36.84 | 17.47 | |
| Qwen-3-8bContext strategy=Coref-32k, Trace=—, Desc=ZS2026.04 | 23.13 | 13.03 | 50.01 | 35.9 | 17.44 | |
| Qwen-3-8b (guided-QA)Context strategy=Coref-32k + guided-QA, Trace=GRPO, Desc=ZS2026.04 | 22.63 | 13.08 | 49.7 | 35.38 | 17.33 | |
| Qwen-3-8bContext strategy=Coref-32k + reasoning, Trace=built-in, Desc=ZS2026.04 | 22.26 | 12.52 | 46.84 | 33.5 | 16.02 | |
| Qwen-3-8bContext strategy=Lead-128k (w/ YaRN), Trace=—, Desc=ZS2026.04 | 22.22 | 13.7 | 66.39 | 35.48 | 17.21 | |
| Qwen-3-8b (guided-QA)Context strategy=Coref-32k + guided-QA, Trace=ZS, Desc=ZS2026.04 | 21.88 | 13.11 | 49.63 | 35.46 | 17.24 | |
| Qwen-3-8bContext strategy=Lead-128k + reasoning, Trace=built-in, Desc=ZS2026.04 | 18.08 | 10.97 | 66.98 | 31.18 | 15.75 | |
| GPT-4.1-miniContext strategy=No context, Trace=—, Desc=ZS2026.04 | 10.01 | 3.98 | — | 21.46 | 16.1 | |
| Qwen-3-8bContext strategy=No Context, Trace=—, Desc=ZS2026.04 | 7.69 | 2.92 | — | 19.83 | 16.4 |