End-to-end QPS Improvement on DeepDiverV2-38B system stack serving evaluation TP8 TP2 deployment
2.52QPS Improvement (Overall)Baseline + AgentCollab + AgentCompress + AgentSched + AgentSAM
Evaluation results
| Method | Links | |
|---|---|---|
| Baseline + AgentCollab + AgentCompress + AgentSched + AgentSAMN_parallel=162025.12 | 2.52 | |
| Baseline + AgentCollab + AgentCompress + AgentSchedN_parallel=162025.12 | 2.25 | |
| Baseline + AgentCollab + AgentCompressN_parallel=162025.12 | 2.01 | |
| Baseline + AgentCollab + AgentCompress + AgentSched + AgentSAMN_parallel=42025.12 | 1.97 | |
| Baseline + AgentCollab + AgentCompress + AgentSchedN_parallel=42025.12 | 1.71 | |
| Baseline + AgentCollab + AgentCompressN_parallel=42025.12 | 1.57 | |
| Baseline + AgentCollabN_parallel=162025.12 | 1.52 | |
| Baseline + AgentCollabN_parallel=42025.12 | 1.32 | |
| BaselineN_parallel=42025.12 | 1 | |
| BaselineN_parallel=162025.12 | 1 |