Document-level Information Extraction on MultiMUC (averaged across languages)
33.08F1 ScoreTHINKTWICE Qwen 3 (oracle)
Evaluation results
| Method | Links | |
|---|---|---|
| THINKTWICE Qwen 3 (oracle)Selector=oracle, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 33.08 | |
| THINKTWICE Llama R1 (oracle)Selector=oracle, Backbone=Llama R1, Shots=Zero-shot2026.01 | 29.66 | |
| THINKTWICE Qwen 3Selector=F1 Voting, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 15.04 | |
| THINKTWICE Qwen 3Selector=Majority, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 14.83 | |
| THINKTWICE Llama R1Selector=F1 Voting, Backbone=Llama R1, Shots=Zero-shot2026.01 | 13.22 | |
| Greedy Qwen 3Selector=X, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 12.98 | |
| ChatGPT 3.5Selector=X, Backbone=GPT-3.5, Shots=3-shot2026.01 | 12.93 | |
| THINKTWICE Llama R1Selector=Majority, Backbone=Llama R1, Shots=Zero-shot2026.01 | 12.78 | |
| Greedy Llama R1Selector=X, Backbone=Llama R1, Shots=Zero-shot2026.01 | 11.46 |