Document-level Information Extraction on MUC
46.48F1 ScoreTHINKTWICE Qwen 3 (oracle)
Evaluation results
| Method | Links | |
|---|---|---|
| THINKTWICE Qwen 3 (oracle)Selector=oracle, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 46.48 | |
| THINKTWICE Llama R1 (oracle)Selector=oracle, Backbone=Llama R1, Shots=Zero-shot2026.01 | 42.32 | |
| THINKTWICE Qwen 3Selector=Majority, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 26.18 | |
| THINKTWICE Qwen 3Selector=F1 Voting, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 24.82 | |
| Greedy Qwen 3Selector=X, Backbone=Qwen 3, Shots=Zero-shot2026.01 | 22.99 | |
| ChatGPT 3.5Selector=X, Backbone=GPT-3.5, Shots=3-shot2026.01 | 22.41 | |
| THINKTWICE Llama R1Selector=Majority, Backbone=Llama R1, Shots=Zero-shot2026.01 | 21.96 | |
| THINKTWICE Llama R1Selector=F1 Voting, Backbone=Llama R1, Shots=Zero-shot2026.01 | 21.23 | |
| Greedy Llama R1Selector=X, Backbone=Llama R1, Shots=Zero-shot2026.01 | 18.68 |