Prompt Optimization Evaluation on HelpSteer 1
51.83HelpfulnessMA-SAPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MA-SAPOBackbone=GPT-4o, Framework=multi-agent framework2025.10 | 51.83 | 62.6 | 86.14 | 50.13 | 73.63 | 64.86 | |
| MARSBackbone=GPT-4o, Framework=multi-agent framework2025.10 | 41.59 | 48.76 | 79.63 | 32.93 | 51.88 | 50.96 | |
| MA-SAPOBackbone=Llama3-8B, Framework=multi-agent framework2025.10 | 41.1 | 48.68 | 83.26 | 47.2 | 84.33 | 60.91 | |
| Role AssignmentBackbone=GPT-4o, Framework=single-pass prompting2025.10 | 39.88 | 46.79 | 80.24 | 34.27 | 50.08 | 50.25 | |
| MARSBackbone=Llama3-8B, Framework=multi-agent framework2025.10 | 39.01 | 45.91 | 77.45 | 35.12 | 58.01 | 51.1 | |
| MADBackbone=GPT-4o, Framework=multi-agent framework2025.10 | 37.74 | 47.64 | 79.54 | 32.1 | 52.48 | 49.9 | |
| MADBackbone=Llama3-8B, Framework=multi-agent framework2025.10 | 37.74 | 50.49 | 80.67 | 40.84 | 67.08 | 55.31 | |
| RAG (sparse, k=10)Backbone=GPT-4o, Framework=retrieval-augmented generation2025.10 | 37.51 | 44.02 | 78.71 | 31.16 | 47.79 | 47.84 | |
| Chain-of-Thought (CoT)Backbone=GPT-4o, Framework=single-pass prompting2025.10 | 32.23 | 38.76 | 75.95 | 29.35 | 41.92 | 43.64 | |
| Direct GenerationBackbone=GPT-4o, Framework=single-pass prompting2025.10 | 32.16 | 38.66 | 75.83 | 29.51 | 42.15 | 43.66 | |
| RAG (sparse, k=10)Backbone=Llama3-8B, Framework=retrieval-augmented generation2025.10 | 29.3 | 35.94 | 74.52 | 30.07 | 43.77 | 42.72 | |
| Role AssignmentBackbone=Llama3-8B, Framework=single-pass prompting2025.10 | 28.87 | 35.16 | 72.87 | 31.64 | 45.17 | 42.74 | |
| Direct GenerationBackbone=Llama3-8B, Framework=single-pass prompting2025.10 | 25.49 | 32.47 | 70.54 | 27.02 | 40.62 | 39.27 | |
| Chain-of-Thought (CoT)Backbone=Llama3-8B, Framework=single-pass prompting2025.10 | 23.59 | 30.77 | 69.06 | 26.27 | 39.67 | 37.87 |