Efficiency and Optimization Evaluation on Multi-Objective Efficiency Evaluation
4.12LatencyGPTQ-INT4
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| GPTQ-INT4Model=OPT-6.7B2025.09 | 4.12 | 0.597 | 0.488 | |
| FairGPTQ-INT4Model=OPT-6.7B2025.09 | 4.12 | 0.589 | 0.481 | |
| GPTQ-INT4Model=Mistral-v0.3-7B2025.09 | 4.57 | 0.584 | 0.478 | |
| FairGPTQ-INT4Model=Mistral-v0.3-7B2025.09 | 4.57 | 0.607 | 0.497 | |
| GPTQ-INT4Model=Mistral-v0.3-7B-Instruct2025.09 | 4.57 | 0.542 | 0.444 | |
| FairGPTQ-INT4Model=Mistral-v0.3-7B-Instruct2025.09 | 4.57 | 0.403 | 0.331 | |
| GPTQ-INT4Model=Qwen-2.5-7B-Instruct2025.09 | 4.75 | 0.367 | 0.304 | |
| FairGPTQ-INT4Model=Qwen-2.5-7B-Instruct2025.09 | 4.75 | 0.366 | 0.304 | |
| GPTQ-INT4Model=LLaMA-3.1-8B-Instruct2025.09 | 5.02 | 0.679 | 0.56 | |
| FairGPTQ-INT4Model=LLaMA-3.1-8B-Instruct2025.09 | 5.02 | 0.601 | 0.497 | |
| GPTQ-INT4Model=Qwen-3-8B2025.09 | 5.34 | 0.374 | 0.323 | |
| FairGPTQ-INT4Model=Qwen-3-8B2025.09 | 5.34 | 0.362 | 0.313 | |
| Base-FP16Model=OPT-6.7B2025.09 | 9.75 | 0.58 | 0.677 | |
| Base-FP16Model=Qwen-2.5-7B-Instruct2025.09 | 9.84 | 0.358 | 0.572 | |
| Base-FP16Model=Mistral-v0.3-7B2025.09 | 10.25 | 0.598 | 0.718 | |
| Base-FP16Model=Mistral-v0.3-7B-Instruct2025.09 | 10.25 | 0.526 | 0.68 | |
| Base-FP16Model=LLaMA-3.1-8B-Instruct2025.09 | 10.73 | 0.579 | 0.739 | |
| Base-FP16Model=Qwen-3-8B2025.09 | 10.84 | 0.384 | 0.657 |