Code Understanding on CRUX (test)
69.3Pass@1ReM-MoA*
Evaluation Results
| Method | Links | |
|---|---|---|
| ReM-MoA*Width (N)=8, Depth (L)=3, Trend=↑↑, Distilled Reviewer=true2026.06 | 69.3 | |
| ReM-MoA*Width (N)=7, Depth (L)=3, Trend=↑↑, Distilled Reviewer=true2026.06 | 68.7 | |
| ReM-MoA*Width (N)=6, Depth (L)=3, Trend=↑↑, Distilled Reviewer=true2026.06 | 68 | |
| ReM-MoAWidth (N)=8, Depth (L)=3, Trend=↑↑2026.06 | 67.3 | |
| ReM-MoA*Width (N)=5, Depth (L)=3, Trend=↑↑, Distilled Reviewer=true2026.06 | 67 | |
| ReM-MoAWidth (N)=7, Depth (L)=3, Trend=↑↑2026.06 | 66.8 | |
| ReM-MoAWidth (N)=6, Depth (L)=3, Trend=↑↑2026.06 | 66.2 | |
| ReM-MoA*Width (N)=4, Depth (L)=3, Trend=↑↑, Distilled Reviewer=true2026.06 | 66 | |
| ReM-MoAWidth (N)=5, Depth (L)=3, Trend=↑↑2026.06 | 65.3 | |
| ReM-MoA*Width (N)=3, Depth (L)=3, Trend=↑↑, Distilled Reviewer=true2026.06 | 64.6 | |
| ReM-MoAWidth (N)=4, Depth (L)=3, Trend=↑↑2026.06 | 64.3 | |
| AttentionMoAWidth (N)=8, Depth (L)=3, Trend=↑↑2026.06 | 63.6 | |
| AttentionMoAWidth (N)=7, Depth (L)=3, Trend=↑↑2026.06 | 63.3 | |
| ReM-MoAWidth (N)=3, Depth (L)=3, Trend=↑↑2026.06 | 63 | |
| AttentionMoAWidth (N)=6, Depth (L)=3, Trend=↑↑2026.06 | 62.9 | |
| AttentionMoAWidth (N)=5, Depth (L)=3, Trend=↑↑2026.06 | 62.3 | |
| AttentionMoAWidth (N)=4, Depth (L)=3, Trend=↑↑2026.06 | 61.5 | |
| RMoAWidth (N)=8, Depth (L)=3, Trend=↑→2026.06 | 60.7 | |
| RMoAWidth (N)=7, Depth (L)=3, Trend=↑→2026.06 | 60.6 | |
| AttentionMoAWidth (N)=3, Depth (L)=3, Trend=↑↑2026.06 | 60.5 | |
| RMoAWidth (N)=6, Depth (L)=3, Trend=↑→2026.06 | 60.3 | |
| RMoAWidth (N)=5, Depth (L)=3, Trend=↑→2026.06 | 59.8 | |
| RMoAWidth (N)=4, Depth (L)=3, Trend=↑→2026.06 | 59.3 | |
| Standard MoAWidth (N)=6, Depth (L)=3, Trend=→2026.06 | 58.6 | |
| Standard MoAWidth (N)=7, Depth (L)=3, Trend=→2026.06 | 58.5 | |
| RMoAWidth (N)=3, Depth (L)=3, Trend=↑→2026.06 | 58.5 | |
| Standard MoAWidth (N)=5, Depth (L)=3, Trend=→2026.06 | 58.4 | |
| Standard MoAWidth (N)=8, Depth (L)=3, Trend=→2026.06 | 58.3 | |
| Standard MoAWidth (N)=4, Depth (L)=3, Trend=→2026.06 | 58 | |
| Standard MoAWidth (N)=3, Depth (L)=3, Trend=→2026.06 | 57.5 |