SMAC
Benchmarks
Task NameDataset NameSOTA ResultTrendResults
SMAC Medium
16.74Backward Transfer
8
SMAC (Expert)
30.8Backward Transfer
8
SMAC Expert Marine-Hard
99.4Performance at 3m
8
SMAC Stalker-Zealot Unseen (test)
78.1Mean Win Rate
8
SMAC 3s5z vs 3s6z
91.9Win Rate
8
SMAC MMM2
98.9Win Rate
8
SMAC 5m vs 6m
90.9Win Rate
8
SMAC Super Hard (test)
83.926h_vs_8z Win Rate
8
SMAC corridor Super Hard (test)
20Average Score
8
SMAC 27m_vs_30m Super Hard (test)
19.71Averaged Score
8
SMAC MMM2 Super Hard (test)
20.9Averaged Score
8
SMAC 3s5z_vs_3s6z Super Hard (test)
20.94Averaged Score
8
SMAC 6h_vs_8z Super Hard (test)
19.4Averaged Score
8
SMAC 3s5z_vs_3s6z (test)
20.94Test Win Rate
8
SMAC 3s5z (test)
97Test Win Rate
8
SMAC (3m → 1c3s5z)
3.8Interactions to 80% Reward (x10^5)
7
SMAC 3m → 10mvs11m
6.4Convergence Steps (×10^5)
7
SMAC (8m → 3m)
80,000Environment Interactions (x10^5)
7
SMAC 8m to 8mvs9m (transfer)
0.53Steps to 80% Reward (x10^6)
7
SMACv2 Terran Medium
7.27Forward Transfer
7
SMACv2 Zerg (Medium)
12.76Forward Transfer
7
SMAC Protoss Medium v2
13.79Forward Transfer
7
SMAC Stalker-zealot Medium
22.74Forward Transfer
7
SMAC Stalker-zealot Expert
27.55Forward Transfer
7
SMAC Marines (Medium)
10.97Forward Transfer
7