# PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learning

Source: [PCGRLLM: Large Language Model-Driven Reward Design for Procedural Content Generation Reinforcement Learning](<https://www.sota2.com/research/paper/68fa76fd1e209131cea70456>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-27T05:00:21.000Z

Authors: In-Chang Baek, Sung-Hyun Kim, Sam Earle, Zehua Jiang, Jin-Ha Noh, Julian Togelius, Kyung-Joong Kim

Published: 2025-02-15T21:00:40.000Z

## Abstract

Reward design plays a pivotal role in the training of game AIs, requiring substantial domain-specific knowledge and human effort. In recent years, several studies have explored reward generation for training game agents and controlling robots using large language models (LLMs). In the content generation literature, there has been early work on generating reward functions for reinforcement learning agent generators. This work introduces PCGRLLM, an extended architecture based on earlier work, which employs a feedback mechanism and several reasoning-based prompt engineering techniques. We evaluate the proposed method on a story-to-reward generation task in a two-dimensional environment using two state-of-the-art LLMs across various reasoning-based prompting methods. Our experiments provide insightful evaluations that demonstrate the capabilities of LLMs essential for content generation tasks. The results demonstrate a substantial performance improvement over the previous structure, achieving performance comparable to that of humans. Our work demonstrates the potential to reduce human dependency in game AI development, while supporting and enhancing creative processes.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Dungeon Level Generation on Dungeon Scenario 3](<https://www.sota2.com/research/sota/dungeon-level-generation-on-dungeon-scenario-3.md>): Dungeon Level Generation; Dungeon Scenario 3; Normalized Mean Score: 0.873.

- [Dungeon Level Generation on Dungeon Scenario 4](<https://www.sota2.com/research/sota/dungeon-level-generation-on-dungeon-scenario-4.md>): Dungeon Level Generation; Dungeon Scenario 4; Normalized Mean Score: 57.3.

- [Dungeon Level Generation on Dungeon Scenario 1](<https://www.sota2.com/research/sota/dungeon-level-generation-on-dungeon-scenario-1.md>): Dungeon Level Generation; Dungeon Scenario 1; Normalized Mean Score: 0.309.

- [Dungeon Level Generation on Dungeon Scenario Mean](<https://www.sota2.com/research/sota/dungeon-level-generation-on-dungeon-scenario-mean.md>): Dungeon Level Generation; Dungeon Scenario Mean; Normalized Mean Score: 42.

- [Dungeon Level Generation on Dungeon Scenario 2](<https://www.sota2.com/research/sota/dungeon-level-generation-on-dungeon-scenario-2.md>): Dungeon Level Generation; Dungeon Scenario 2; Normalized Mean Score: 30.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2502.10906>)

[PDF](<https://arxiv.org/pdf/2502.10906.pdf>)
