Honesty alignment on HHH alignment
86.1Win Rate (WR)AMBS
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| AMBSBackbone=DeepSeek-7B, Throughput (Tk)=230, Memory Usage (Me)=19, Compute Cost (GP)=42025.09 | 86.1 | 30.8 | 67.5 | 40.9 | |
| AMBSBackbone=Mistral-7B, Throughput (Tk)=218, Memory Usage (Me)=20, Compute Cost (GP)=52025.09 | 51.8 | 30.1 | 46.1 | 22.6 | |
| AMBSBackbone=Gemma-7B, Throughput (Tk)=229, Memory Usage (Me)=20, Compute Cost (GP)=52025.09 | 50.4 | 22.1 | 32.2 | 20.1 | |
| SteeringBackbone=Mistral-7B, Throughput (Tk)=215, Memory Usage (Me)=22, Compute Cost (GP)=62025.09 | 36.4 | 33 | 35.9 | 13.1 | |
| AlignerBackbone=Mistral-7B, Throughput (Tk)=182, Memory Usage (Me)=26, Compute Cost (GP)=82025.09 | 35.1 | 36.1 | 35.3 | 11.4 | |
| AMBSBackbone=LLaMA-2-7B, Throughput (Tk)=210, Memory Usage (Me)=21, Compute Cost (GP)=62025.09 | 34.9 | 30 | 37.9 | 14.3 | |
| DExpertsBackbone=Mistral-7B, Throughput (Tk)=179, Memory Usage (Me)=27, Compute Cost (GP)=92025.09 | 34.6 | 34 | 34.9 | 11.8 | |
| PPLMBackbone=Mistral-7B, Throughput (Tk)=155, Memory Usage (Me)=29, Compute Cost (GP)=102025.09 | 32.8 | 34.9 | 33.2 | 10.4 | |
| SteeringBackbone=Gemma-7B, Throughput (Tk)=221, Memory Usage (Me)=21, Compute Cost (GP)=62025.09 | 32.3 | 25.7 | 27.8 | 11.5 | |
| AlignerBackbone=Gemma-7B, Throughput (Tk)=188, Memory Usage (Me)=25, Compute Cost (GP)=72025.09 | 31.5 | 28.7 | 27.9 | 10.2 | |
| DExpertsBackbone=Gemma-7B, Throughput (Tk)=185, Memory Usage (Me)=26, Compute Cost (GP)=82025.09 | 30.9 | 26.8 | 27.2 | 10.4 | |
| PPLMBackbone=Gemma-7B, Throughput (Tk)=161, Memory Usage (Me)=28, Compute Cost (GP)=92025.09 | 29.7 | 27.4 | 26.1 | 9.5 | |
| SteeringBackbone=LLaMA-2-7B, Throughput (Tk)=209, Memory Usage (Me)=23, Compute Cost (GP)=72025.09 | 29.4 | 31.7 | 32.6 | 10.1 | |
| SteeringBackbone=DeepSeek-7B, Throughput (Tk)=227, Memory Usage (Me)=20, Compute Cost (GP)=62025.09 | 28.9 | 32.5 | 48.2 | 14.9 | |
| AlignerBackbone=LLaMA-2-7B, Throughput (Tk)=176, Memory Usage (Me)=27, Compute Cost (GP)=92025.09 | 28.6 | 34.9 | 3.9 | -0.8 | |
| AlignerBackbone=DeepSeek-7B, Throughput (Tk)=194, Memory Usage (Me)=24, Compute Cost (GP)=72025.09 | 28 | 35.6 | 48.8 | 13.7 | |
| DExpertsBackbone=LLaMA-2-7B, Throughput (Tk)=173, Memory Usage (Me)=28, Compute Cost (GP)=102025.09 | 27.9 | 32.6 | 31.7 | 9 | |
| DExpertsBackbone=DeepSeek-7B, Throughput (Tk)=191, Memory Usage (Me)=25, Compute Cost (GP)=82025.09 | 27.4 | 33.2 | 47.5 | 13.9 | |
| PPLMBackbone=LLaMA-2-7B, Throughput (Tk)=149, Memory Usage (Me)=30, Compute Cost (GP)=112025.09 | 26.5 | 33.5 | 30.7 | 7.9 | |
| PPLMBackbone=DeepSeek-7B, Throughput (Tk)=167, Memory Usage (Me)=27, Compute Cost (GP)=92025.09 | 26.2 | 34 | 46.7 | 13 |