Short-context Reading Comprehension on ROPES
77.63F1 ScoreBase+ctx
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Base+ctxLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 77.63 | 77.63 | |
| Base+ctxLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 71.9 | 71.9 | |
| D2LckptLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 65.9 | 65.85 | |
| Doc2AtomLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 58.08 | 58.08 | |
| D2LatomLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 53.93 | 53.93 | |
| Doc2AtomLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 53.52 | 53.52 | |
| D2LrawLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 53.06 | 52.98 | |
| D2LrawLLM Backbone=Gemma-2-2B-It, Evaluation Protocol=In-domain2026.06 | 52.94 | 52.94 | |
| D2LatomLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 51.95 | 51.95 | |
| D2LckptLLM Backbone=Qwen3-4B-Instruct, Evaluation Protocol=In-domain2026.06 | 15.26 | 15.25 |