Commonsense Reasoning on COMM
43.73AccuracyVERSATILEFFN
Evaluation Results
| Method | Links | |
|---|---|---|
| VERSATILEFFNBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 43.73 | |
| 4-LOOPBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 42.86 | |
| 6-LOOPBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 41.93 | |
| MOEBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 41.2 | |
| VERSATILEFFNBase Model Params=720.81M, Protocol=Zero-shot2025.12 | 40.79 | |
| 2-LOOPBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 40.7 | |
| BASEBase Model Params=1.21B, Protocol=Zero-shot2025.12 | 40.05 | |
| 4-LOOPBase Model Params=720.81M, Protocol=Zero-shot2025.12 | 39.39 | |
| 6-LOOPBase Model Params=720.81M, Protocol=Zero-shot2025.12 | 38.98 | |
| 2-LOOPBase Model Params=720.81M, Protocol=Zero-shot2025.12 | 37.84 | |
| MOEBase Model Params=720.81M, Protocol=Zero-shot2025.12 | 37.1 | |
| VERSATILEFFNBase Model Params=354.71M, Protocol=Zero-shot2025.12 | 36.69 | |
| 4-LOOPBase Model Params=354.71M, Protocol=Zero-shot2025.12 | 36.53 | |
| 6-LOOPBase Model Params=354.71M, Protocol=Zero-shot2025.12 | 35.95 | |
| 2-LOOPBase Model Params=354.71M, Protocol=Zero-shot2025.12 | 35.46 | |
| BASEBase Model Params=720.81M, Protocol=Zero-shot2025.12 | 35.14 | |
| MOEBase Model Params=354.71M, Protocol=Zero-shot2025.12 | 34.64 | |
| BASEBase Model Params=354.71M, Protocol=Zero-shot2025.12 | 32.76 |