Event Extraction on MUC-4 and MultiMUC (test)
58.32English ScoreTHINKTWICE
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| THINKTWICEBackbone=Qwen think, Selector=oracle2026.01 | 58.32 | 45.52 | 46.74 | 22.63 | 45.14 | 46.73 | 44.18 | |
| THINKTWICEBackbone=Llama R1, Selector=oracle2026.01 | 56.6 | 12.79 | 4.66 | 11.82 | 2.96 | 19.44 | 18.04 | |
| THINKTWICEBackbone=Qwen think, Selector=Reward2026.01 | 42.13 | 28.01 | 28.96 | 9.12 | 27.55 | 29.37 | 27.52 | |
| THINKTWICEBackbone=Llama R1, Selector=Reward2026.01 | 41.99 | 5.78 | 1.87 | 4.8 | 1.8 | 12.16 | 11.4 | |
| THINKTWICEBackbone=Qwen think, Selector=F12026.01 | 41.4 | 23.73 | 23.68 | 8.54 | 20.68 | 22.27 | 23.38 | |
| THINKTWICEBackbone=Qwen think, Selector=Majority2026.01 | 36.6 | 16.81 | 15.78 | 5.54 | 14.29 | 13.66 | 17.11 | |
| Qwen thinkSelector=✗, Decoding=Greedy2026.01 | 35.97 | 17.6 | 16.17 | 7.79 | 15.33 | 16.14 | 18.17 | |
| IterX T5-enc largeSelector=✗2026.01 | 35.2 | 21.46 | 20.66 | 23.91 | 23.77 | 21.93 | 24.49 | |
| THINKTWICEBackbone=Llama R1, Selector=F12026.01 | 35 | 4.83 | 1.67 | 5.02 | 0.18 | 7.05 | 8.96 | |
| Llama R1Selector=✗, Decoding=Greedy2026.01 | 28.52 | 3.57 | 1.3 | 3.07 | 0.51 | 4.21 | 6.86 | |
| THINKTWICEBackbone=Llama R1, Selector=Majority2026.01 | 25.06 | 2.25 | 1.2 | 2.21 | 0 | 3.22 | 5.66 |