Closed-ended reasoning on GPQA Diamond (test)
65.9AccuracyMajority Voting (MV)
Evaluation results
| Method | Links | |
|---|---|---|
| Majority Voting (MV)K=8, Backbone=Qwen3-14B, Strategy=Shortest-K Merge2025.12 | 65.9 | |
| Majority Voting (MV)K=4, Backbone=Qwen3-14B, Strategy=Shortest-K Merge2025.12 | 65.2 | |
| THINKMERGE (DirectMerge (A))K=8, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 64.1 | |
| Majority Voting (MV)K=8, Backbone=Qwen3-14B, Strategy=Early-Ready2025.12 | 64.1 | |
| THINKMERGE (Ours (C))K=8, Backbone=Qwen3-14B, Strategy=Early-Ready2025.12 | 64.1 | |
| THINKMERGE (DirectMerge (A))K=4, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 64 | |
| Majority Voting (MV)K=4, Backbone=Qwen3-14B, Strategy=Early-Ready2025.12 | 64 | |
| Majority Voting (MV)K=8, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 63.9 | |
| THINKMERGE (Ours (D))K=4, Backbone=Qwen3-14B, Strategy=Shortest-K Merge2025.12 | 63.8 | |
| THINKMERGE (Ours (D))K=2, Backbone=Qwen3-14B, Strategy=Shortest-K Merge2025.12 | 63.7 | |
| THINKMERGE (Ours (D))K=8, Backbone=Qwen3-14B, Strategy=Shortest-K Merge2025.12 | 63.7 | |
| Majority Voting (MV)K=2, Backbone=Qwen3-14B, Strategy=Shortest-K Merge2025.12 | 63.4 | |
| Majority Voting (MV)K=4, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 63 | |
| THINKMERGE (Trimming (B))K=8, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 63 | |
| THINKMERGE (Trimming (B))K=4, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 62.7 | |
| THINKMERGE (Ours (C))K=4, Backbone=Qwen3-14B, Strategy=Early-Ready2025.12 | 62.4 | |
| THINKMERGE (Ours (C))K=2, Backbone=Qwen3-14B, Strategy=Early-Ready2025.12 | 62.1 | |
| THINKMERGE (DirectMerge (A))K=2, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 61.6 | |
| THINKMERGE (Trimming (B))K=2, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 61.4 | |
| Majority Voting (MV)K=2, Backbone=Qwen3-14B, Strategy=All-Reduce2025.12 | 60.8 | |
| Majority Voting (MV)K=2, Backbone=Qwen3-14B, Strategy=Early-Ready2025.12 | 60.8 | |
| Majority Voting (MV)K=8, Backbone=Qwen3-4B, Strategy=Shortest-K Merge2025.12 | 55.5 | |
| THINKMERGE (Ours (D))K=8, Backbone=Qwen3-4B, Strategy=Shortest-K Merge2025.12 | 53.8 | |
| Majority Voting (MV)K=4, Backbone=Qwen3-4B, Strategy=Shortest-K Merge2025.12 | 53.7 | |
| Majority Voting (MV)K=8, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 53.3 | |
| Majority Voting (MV)K=8, Backbone=Qwen3-4B, Strategy=Early-Ready2025.12 | 53.3 | |
| Majority Voting (MV)K=8, Backbone=R1-Distill-Qwen-7B, Strategy=Shortest-K Merge2025.12 | 52.9 | |
| THINKMERGE (Ours (D))K=2, Backbone=Qwen3-4B, Strategy=Shortest-K Merge2025.12 | 52.6 | |
| Majority Voting (MV)K=8, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 52.5 | |
| Majority Voting (MV)K=8, Backbone=R1-Distill-Qwen-7B, Strategy=Early-Ready2025.12 | 52.5 | |
| THINKMERGE (Ours (C))K=4, Backbone=Qwen3-4B, Strategy=Early-Ready2025.12 | 52.4 | |
| THINKMERGE (DirectMerge (A))K=4, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 52.2 | |
| THINKMERGE (Trimming (B))K=4, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 52.2 | |
| Majority Voting (MV)K=2, Backbone=Qwen3-4B, Strategy=Shortest-K Merge2025.12 | 52 | |
| THINKMERGE (Ours (D))K=4, Backbone=Qwen3-4B, Strategy=Shortest-K Merge2025.12 | 51.8 | |
| THINKMERGE (DirectMerge (A))K=8, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 51.6 | |
| Majority Voting (MV)K=4, Backbone=Qwen3-4B, Strategy=Early-Ready2025.12 | 51.4 | |
| THINKMERGE (Trimming (B))K=8, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 51.3 | |
| THINKMERGE (Ours (C))K=8, Backbone=Qwen3-4B, Strategy=Early-Ready2025.12 | 51.3 | |
| Majority Voting (MV)K=4, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 51.2 | |
| THINKMERGE (DirectMerge (A))K=2, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 50.3 | |
| THINKMERGE (Ours (C))K=2, Backbone=Qwen3-4B, Strategy=Early-Ready2025.12 | 50.3 | |
| Majority Voting (MV)K=4, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 50.3 | |
| Majority Voting (MV)K=4, Backbone=R1-Distill-Qwen-7B, Strategy=Early-Ready2025.12 | 50.3 | |
| Majority Voting (MV)K=4, Backbone=R1-Distill-Qwen-7B, Strategy=Shortest-K Merge2025.12 | 50.2 | |
| THINKMERGE (Ours (C))K=8, Backbone=R1-Distill-Qwen-7B, Strategy=Early-Ready2025.12 | 50.2 | |
| THINKMERGE (Trimming (B))K=2, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 50 | |
| THINKMERGE (DirectMerge (A))K=8, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 50 | |
| THINKMERGE (Trimming (B))K=8, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 50 | |
| Majority Voting (MV)K=2, Backbone=Qwen3-4B, Strategy=Early-Ready2025.12 | 49.4 | |
| Majority Voting (MV)K=2, Backbone=Qwen3-4B, Strategy=All-Reduce2025.12 | 49.2 | |
| THINKMERGE (DirectMerge (A))K=2, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 49.2 | |
| THINKMERGE (Trimming (B))K=2, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 49.2 | |
| THINKMERGE (Ours (C))K=2, Backbone=R1-Distill-Qwen-7B, Strategy=Early-Ready2025.12 | 49 | |
| THINKMERGE (Trimming (B))K=4, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 49 | |
| THINKMERGE (DirectMerge (A))K=4, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 48.9 | |
| THINKMERGE (Ours (C))K=4, Backbone=R1-Distill-Qwen-7B, Strategy=Early-Ready2025.12 | 48.9 | |
| Majority Voting (MV)K=2, Backbone=R1-Distill-Qwen-7B, Strategy=Early-Ready2025.12 | 47.4 | |
| THINKMERGE (Ours (D))K=4, Backbone=R1-Distill-Qwen-7B, Strategy=Shortest-K Merge2025.12 | 47.4 | |
| THINKMERGE (Ours (D))K=8, Backbone=R1-Distill-Qwen-7B, Strategy=Shortest-K Merge2025.12 | 47.2 | |
| Majority Voting (MV)K=2, Backbone=R1-Distill-Qwen-7B, Strategy=Shortest-K Merge2025.12 | 47.1 | |
| Majority Voting (MV)K=2, Backbone=R1-Distill-Qwen-7B, Strategy=All-Reduce2025.12 | 44.9 | |
| THINKMERGE (Ours (D))K=2, Backbone=R1-Distill-Qwen-7B, Strategy=Shortest-K Merge2025.12 | 43.8 |