Prompt Optimization Evaluation on HelpSteer2
0.5072HelpfulnessMA-SAPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MA-SAPOBackbone=GPT-4o, Framework=multi-agent framework2025.10 | 0.5072 | 0.6038 | 0.8527 | 0.5244 | 0.757 | 0.649 | |
| RAG (sparse, k=10)Backbone=GPT-4o, Framework=retrieval-augmented generation2025.10 | 0.4903 | 0.5745 | 0.8642 | 0.4161 | 0.6567 | 0.6003 | |
| MARSBackbone=GPT-4o, Framework=multi-agent framework2025.10 | 0.4791 | 0.5569 | 0.8268 | 0.4095 | 0.6234 | 0.5791 | |
| Role AssignmentBackbone=GPT-4o, Framework=single-pass prompting2025.10 | 0.44 | 0.5175 | 0.8221 | 0.4025 | 0.5992 | 0.5563 | |
| MARSBackbone=Llama3-8B, Framework=multi-agent framework2025.10 | 0.4296 | 0.5019 | 0.7994 | 0.3957 | 0.6181 | 0.5482 | |
| MADBackbone=GPT-4o, Framework=multi-agent framework2025.10 | 0.4167 | 0.5049 | 0.8067 | 0.4084 | 0.6708 | 0.5615 | |
| MA-SAPOBackbone=Llama3-8B, Framework=multi-agent framework2025.10 | 0.4005 | 0.4754 | 0.8294 | 0.4833 | 0.8441 | 0.6065 | |
| RAG (sparse, k=10)Backbone=Llama3-8B, Framework=retrieval-augmented generation2025.10 | 0.399 | 0.4711 | 0.7989 | 0.3722 | 0.5814 | 0.5245 | |
| MADBackbone=Llama3-8B, Framework=multi-agent framework2025.10 | 0.3971 | 0.4532 | 0.7898 | 0.3998 | 0.6439 | 0.5368 | |
| Direct GenerationBackbone=GPT-4o, Framework=single-pass prompting2025.10 | 0.3616 | 0.47 | 0.7723 | 0.328 | 0.4913 | 0.4846 | |
| Chain-of-Thought (CoT)Backbone=GPT-4o, Framework=single-pass prompting2025.10 | 0.2981 | 0.3958 | 0.7169 | 0.2888 | 0.4709 | 0.4341 | |
| Direct GenerationBackbone=Llama3-8B, Framework=single-pass prompting2025.10 | 0.2616 | 0.3636 | 0.7078 | 0.2942 | 0.4421 | 0.4139 | |
| Role AssignmentBackbone=Llama3-8B, Framework=single-pass prompting2025.10 | 0.2555 | 0.3303 | 0.705 | 0.3267 | 0.4441 | 0.4123 | |
| Chain-of-Thought (CoT)Backbone=Llama3-8B, Framework=single-pass prompting2025.10 | 0.16 | 0.2545 | 0.6291 | 0.2421 | 0.3812 | 0.3334 |