# ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation

Source: [ChatGPT vs SBST: A Comparative Assessment of Unit Test Suite Generation](<https://www.sota2.com/research/paper/68fa6fe71e209131cea48054>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T01:29:17.533Z

Authors: Yutian Tang, Zhijie Liu, Zhichao Zhou, Xiapu Luo

Published: 2023-07-02T15:09:40.000Z

## Abstract

Recent advancements in large language models (LLMs) have demonstrated exceptional success in a wide range of general domain tasks, such as question answering and following instructions. Moreover, LLMs have shown potential in various software engineering applications. In this study, we present a systematic comparison of test suites generated by the ChatGPT LLM and the state-of-the-art SBST tool EvoSuite. Our comparison is based on several critical factors, including correctness, readability, code coverage, and bug detection capability. By highlighting the strengths and weaknesses of LLMs (specifically ChatGPT) in generating unit test cases compared to EvoSuite, this work provides valuable insights into the performance of LLMs in solving software engineering problems. Overall, our findings underscore the potential of LLMs in software engineering and pave the way for further research in this area.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.00588>)

[PDF](<https://arxiv.org/pdf/2307.00588.pdf>)
