WhatsUp
Benchmarks
TaskDatasetSOTA resultTrendResults
WhatsUp
51.3Accuracy
21
WhatsUp
8.2GPT-4o Score
18
WhatsUp
99.2Accuracy
10
WhatsUp-B
99.9Binary Robust Accuracy
9
WhatsUp AURORA-BENCH
3.3GPT-4o Score
9
WhatsUp (test)
0.25Human Evaluation Score
4
WhatsUp B-FB 2x2 directional variants
66.67GroupScore
3
WhatsUp B-LR 2x2 directional variants
82.84GroupScore
3
WhatsUp A-OU 2x2 directional variants
99.03GroupScore
3
WhatsUp A-LR 2x2 directional variants
95.87GroupScore
3
WhatsUp B (1 × 4 groups)
49.94GroupScore
2
WhatsUp A 1 × 4 groups
56.8GroupScore
2