# Macro action selection with deep reinforcement learning in StarCraft

Source: [Macro action selection with deep reinforcement learning in StarCraft](<https://www.sota2.com/research/paper/68fa66811e209131cea10bb6>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-12-25T07:43:22.557Z

Authors: Sijia Xu, Hongyu Kuang, Zhi Zhuang, Renjie Hu, Yang Liu, Huyang Sun

Published: 2018-12-02T06:06:28.000Z

## Abstract

StarCraft (SC) is one of the most popular and successful Real Time Strategy (RTS) games. In recent years, SC is also widely accepted as a challenging testbed for AI research because of its enormous state space, partially observed information, multi-agent collaboration, and so on. With the help of annual AIIDE and CIG competitions, a growing number of SC bots are proposed and continuously improved. However, a large gap remains between the top-level bot and the professional human player. One vital reason is that current SC bots mainly rely on predefined rules to select macro actions during their games. These rules are not scalable and efficient enough to cope with the enormous yet partially observed state space in the game. In this paper, we propose a deep reinforcement learning (DRL) framework to improve the selection of macro actions. Our framework is based on the combination of the Ape-X DQN and the Long-Short-Term-Memory (LSTM). We use this framework to build our bot, named as LastOrder. Our evaluation, based on training against all bots from the AIIDE 2017 StarCraft AI competition set, shows that LastOrder achieves an 83% winning rate, outperforming 26 bots in total 28 entrants.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [StarCraft AI on AIIDE StarCraft AI competition without IO sync 2017](<https://www.sota2.com/research/sota/starcraft-ai-on-aiide-starcraft-ai-competition-without-io-sync-2017.md>): StarCraft AI; AIIDE StarCraft AI competition without IO sync 2017; Win Rate: 83.06.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1812.00336>)

[PDF](<https://arxiv.org/pdf/1812.00336.pdf>)

[Implementation](<https://github.com/Bilibili/LastOrder>)