# Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration

Source: [Unleashing the Emergent Cognitive Synergy in Large Language Models: A Task-Solving Agent through Multi-Persona Self-Collaboration](<https://www.sota2.com/research/paper/68fa72af1e209131cea57dc3>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-28T13:57:05.302Z

Authors: Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, Heng Ji

Published: 2023-07-11T14:45:19.000Z

## Abstract

Human intelligence thrives on cognitive synergy, where collaboration among different minds yield superior outcomes compared to isolated individuals. In this work, we propose Solo Performance Prompting (SPP), which transforms a single LLM into a cognitive synergist by engaging in multi-turn self-collaboration with multiple personas. A cognitive synergist is an intelligent agent that collaboratively combines multiple minds' strengths and knowledge to enhance problem-solving in complex tasks. By dynamically identifying and simulating different personas based on task inputs, SPP unleashes the potential of cognitive synergy in LLMs. Our in-depth analysis shows that assigning multiple fine-grained personas in LLMs improves problem-solving abilities compared to using a single or fixed number of personas. We evaluate SPP on three challenging tasks: Trivia Creative Writing, Codenames Collaborative, and Logic Grid Puzzle, encompassing both knowledge-intensive and reasoning-intensive types. Unlike previous works, such as Chain-of-Thought, that solely enhance the reasoning abilities in LLMs, experimental results demonstrate that SPP effectively reduces factual hallucination, and maintains strong reasoning capabilities. Additionally, comparative experiments show that cognitive synergy only emerges in GPT-4 and does not appear in less capable models, such as GPT-3.5-turbo and Llama2-13b-chat, which draws an interesting analogy to human development. Code, data, and prompts can be found at: https://github.com/MikeWangWZHL/Solo-Performance-Prompting.git.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Inference Cost on MATH](<https://www.sota2.com/research/sota/inference-cost-on-math.md>): Mathematical Reasoning; MATH; Inference Cost (USD): 0.33.

- [Question Answering on DROP (Inference Cost)](<https://www.sota2.com/research/sota/question-answering-on-drop-inference-cost.md>): Question Answering; DROP; Inference Cost (USD): 0.301.

- [Inference Cost on GSM8K](<https://www.sota2.com/research/sota/inference-cost-on-gsm8k.md>): Mathematical Reasoning; GSM8K; Inference Cost (USD): 0.429.

- [Question Answering on HotpotQA (Inference Cost)](<https://www.sota2.com/research/sota/question-answering-on-hotpotqa-inference-cost.md>): Question Answering; HotpotQA; Inference Cost (USD): 0.415.

- [Mathematical Reasoning on GSM8K (Solve Rate)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-gsm8k-solve-rate.md>): Mathematical Reasoning; GSM8K; Solve Rate: 92.8.

- [Mathematical Reasoning on MATH (Solve Rate %)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-solve-rate-percent.md>): Mathematical Reasoning; MATH; Solve Rate (MATH): 50.8.

- [Medical Reasoning on MedAgentsBench Hard Subsets](<https://www.sota2.com/research/sota/medical-reasoning-on-medagentsbench-hard-subsets.md>): Medical Reasoning; MedAgentsBench Hard Subsets; MEDQA: 0.45.

- [Inference Cost (USD) on MBPP](<https://www.sota2.com/research/sota/inference-cost-usd-on-mbpp.md>): Code Generation; MBPP; Inference Cost (USD): 0.226.

- [Aggregated Multi-task Evaluation on Average](<https://www.sota2.com/research/sota/aggregated-multi-task-evaluation-on-average.md>): Aggregated Multi-task Evaluation; Average; Performance Score: 63.87.

- [Coding on MBPP (Solve Rate)](<https://www.sota2.com/research/sota/coding-on-mbpp-solve-rate.md>): Coding; MBPP; Solve Rate: 72.72.

- [Coding on HumanEval (Solve Rate)](<https://www.sota2.com/research/sota/coding-on-humaneval-solve-rate.md>): Coding; HumanEval; Solve Rate: 0.9236.

- [Math on GSM8K (Solve Rate and Executability)](<https://www.sota2.com/research/sota/math-on-gsm8k-solve-rate-and-executability.md>): Math; GSM8K; Solve Rate: 91.18.

- [Math on MATH (Solve Rate, Executability)](<https://www.sota2.com/research/sota/math-on-math-solve-rate-executability.md>): Math; MATH; Solve Rate: 52.88.

- [Reading Comprehension on DROP (F1 Score)](<https://www.sota2.com/research/sota/reading-comprehension-on-drop-f1-score.md>): Reading Comprehension; DROP; F1 Score: 81.3.

- [Mathematical Reasoning on GSM8K (success rate)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-gsm8k-success-rate.md>): Mathematical Reasoning; GSM8K; Success Rate: 87.1.

- [Mathematical Reasoning on MATH (Performance and Cost Metrics)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-performance-and-cost-metrics.md>): Mathematical Reasoning; MATH; Accuracy: 57.41.

- [Reasoning on HotpotQA (Solve Rate and Executability)](<https://www.sota2.com/research/sota/reasoning-on-hotpotqa-solve-rate-and-executability.md>): Reasoning; HotpotQA; Solve Rate: 81.62.

- [Problem-Solving on MMLU (test)](<https://www.sota2.com/research/sota/problem-solving-on-mmlu-test.md>): Problem-Solving; MMLU (test); MMLU Problem Solving Accuracy: 65.94.

- [Science on GPQA (Solve Rate/Executability)](<https://www.sota2.com/research/sota/science-on-gpqa-solve-rate-executability.md>): Science; GPQA; Solve Rate: 33.72.

- [Question Answering on DROP (F1 score)](<https://www.sota2.com/research/sota/question-answering-on-drop-f1-score.md>): Question Answering; DROP; F1 Score: 78.9.

- [Reasoning on GPQA (80% test)](<https://www.sota2.com/research/sota/reasoning-on-gpqa-80-percent-test.md>): Reasoning; GPQA (80% test); Accuracy: 49.8.

- [Scientific Problem Solving on SciBench](<https://www.sota2.com/research/sota/scientific-problem-solving-on-scibench.md>): Scientific problem solving; SciBench; Pass@20: 28.22.

- [Reading Comprehension on DROP (test) (Accuracy)](<https://www.sota2.com/research/sota/reading-comprehension-on-drop-test-accuracy.md>): Reading Comprehension; DROP (test); Accuracy: 74.4.

- [Cross-domain LLM Evaluation on MMLU, GSM8K, MultiArith, SVAMP, AQuA, and HumanEval](<https://www.sota2.com/research/sota/cross-domain-llm-evaluation-on-mmlu-gsm8k-multiarith-svamp-aqua-and-humaneval.md>): Cross-domain LLM Evaluation; MMLU, GSM8K, MultiArith, SVAMP, AQuA, HumanEval; Average Score: 86.21.

- [Reading Comprehension on DROP (Performance and Cost)](<https://www.sota2.com/research/sota/reading-comprehension-on-drop-performance-and-cost.md>): Reading Comprehension; DROP; Performance Score: 76.13.

- [Code Generation on MBPP (Perf., Cost)](<https://www.sota2.com/research/sota/code-generation-on-mbpp-perf-cost.md>): Code Generation; MBPP; Performance (%): 70.38.

- [Mathematical Reasoning on AMC (Perf., Cost)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-amc-perf-cost.md>): Mathematical Reasoning; AMC; Accuracy: 53.74.

- [Code Generation on HumanEval (Pass@k, AUC, and Token Usage)](<https://www.sota2.com/research/sota/code-generation-on-humaneval-pass-at-k-auc-and-token-usage.md>): Code Generation; HumanEval; Pass@1: 89.1.

- [Accuracy on MATH](<https://www.sota2.com/research/sota/accuracy-on-math.md>): Mathematical Reasoning; MATH; Accuracy: 51.9.

- [Reading Comprehension on DROP (test)](<https://www.sota2.com/research/sota/reading-comprehension-on-drop-test.md>): Reading Comprehension; DROP (test); F1 Score: 74.38.

- [General Knowledge Evaluation on MMLU](<https://www.sota2.com/research/sota/general-knowledge-evaluation-on-mmlu.md>): General Knowledge Evaluation; MMLU; MMLU Accuracy: 77.69.

- [Mathematical Reasoning on GSM8K (test) (Accuracy, Reward)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-gsm8k-test-accuracy-reward.md>): Mathematical Reasoning; GSM8K (test); Accuracy: 87.5.

- [Science Reasoning on GPQA (test)](<https://www.sota2.com/research/sota/science-reasoning-on-gpqa-test.md>): Science Reasoning; GPQA (test); Accuracy: 31.82.

- [Reasoning on AIME 24 (Traces, Consistency, Accuracy)](<https://www.sota2.com/research/sota/reasoning-on-aime-24-traces-consistency-accuracy.md>): Reasoning; AIME 24; Accuracy: 26.39.

- [Code Generation on MBPP (tau, Speed, Score)](<https://www.sota2.com/research/sota/code-generation-on-mbpp-tau-speed-score.md>): Code Generation; MBPP; Overall Score: 70.4.

- [Reasoning on DROP](<https://www.sota2.com/research/sota/reasoning-on-drop.md>): Reasoning; DROP; Score: 81.62.

- [Arithmetic Reasoning on MultiArith (test)](<https://www.sota2.com/research/sota/arithmetic-reasoning-on-multiarith-test.md>): Arithmetic Reasoning; MultiArith (test); Accuracy: 97.49.

- [Code Generation on MBPP (pass@1, pass@80)](<https://www.sota2.com/research/sota/code-generation-on-mbpp-pass-at-1-pass-at-80.md>): Code Generation; MBPP; Pass@1: 73.3.

- [Mathematical Reasoning on GSM8K (Accuracy %)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-gsm8k-accuracy-percent.md>): Mathematical Reasoning; GSM8K; Accuracy (GSM8K): 92.8.

- [Multi-task Knowledge and Reasoning on MMLU-Pro](<https://www.sota2.com/research/sota/multi-task-knowledge-and-reasoning-on-mmlu-pro.md>): Multi-task Knowledge and Reasoning; MMLU-Pro; Average Score @1: 61.7.

- [Code Generation on MBPP (pass@1, Improvement Rate (%))](<https://www.sota2.com/research/sota/code-generation-on-mbpp-pass-at-1-improvement-rate-percent.md>): Code Generation; MBPP; Pass@1: 73.19.

- [Code Generation on HumanEval (test)](<https://www.sota2.com/research/sota/code-generation-on-humaneval-test.md>): Code Generation; HumanEval (test); Pass@1: 88.32.

- [Mathematical Reasoning on MATH (test) (Reward, Acc (%))](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-test-reward-acc-percent.md>): Mathematical Reasoning; MATH (test); Accuracy: 45.43.

- [Arithmetic Reasoning on MultiArith](<https://www.sota2.com/research/sota/arithmetic-reasoning-on-multiarith.md>): Arithmetic Reasoning; MultiArith; Accuracy: 97.49.

- [Code Generation on MBPP (test)](<https://www.sota2.com/research/sota/code-generation-on-mbpp-test.md>): Code Generation; MBPP (test); Pass@1: 73.19.

- [Mathematical Reasoning on AQuA (Accuracy)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-aqua-accuracy.md>): Math Reasoning; AQUA; Accuracy: 79.23.

- [Question Answering on GPQA (Accuracy)](<https://www.sota2.com/research/sota/question-answering-on-gpqa-accuracy.md>): Question Answering; GPQA; Accuracy: 35.1.

- [Code Generation on HumanEval](<https://www.sota2.com/research/sota/code-generation-on-humaneval.md>): Code Generation; HumanEval; Pass@1: 88.32.

- [Mathematical Reasoning on MATH (Accuracy and Token Usage)](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math-accuracy-and-token-usage.md>): Mathematical Reasoning; MATH; Accuracy: 45.43.

- [Mathematical Reasoning on MATH](<https://www.sota2.com/research/sota/mathematical-reasoning-on-math.md>): Mathematical Reasoning; MATH; Accuracy: 51.7.

- [Code Generation on HumanEval+](<https://www.sota2.com/research/sota/code-generation-on-humaneval-plus.md>): Code Generation; HumanEval+; Pass@1: not recorded.

- [Multi-hop Question Answering on HotpotQA (test)](<https://www.sota2.com/research/sota/multi-hop-question-answering-on-hotpotqa-test.md>): Multi-hop Question Answering; HotpotQA (test); F1: not recorded.

- [Multitask Language Understanding on MMLU (Accuracy only)](<https://www.sota2.com/research/sota/multitask-language-understanding-on-mmlu-accuracy-only.md>): Multitask Language Understanding; MMLU; Accuracy: not recorded.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2307.05300>)

[PDF](<https://arxiv.org/pdf/2307.05300.pdf>)
