Code Evaluation on CodeMMLU software principles
24.9AccuracyDPO + RM
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-GEMMA-27B2025.04 | 24.9 | 0.9 | |
| DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-LLAMA-8B2025.04 | 23.1 | 0.7 | |
| DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-GEMMA-27B2025.04 | 23.1 | 0.8 | |
| DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-GEMMA-27B2025.04 | 23 | 0.9 | |
| DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-LLAMA-8B2025.04 | 20.9 | 1 | |
| DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-LLAMA-8B2025.04 | 20.3 | 0.7 | |
| DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-GEMMA-27B2025.04 | 17.4 | 2.1 | |
| DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-GEMMA-27B2025.04 | 17.4 | 2.4 | |
| DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-GEMMA-27B2025.04 | 17.3 | 1.8 | |
| DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-LLAMA-8B2025.04 | 15.3 | 2 | |
| DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-LLAMA-8B2025.04 | 15.2 | 1.8 | |
| DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-LLAMA-8B2025.04 | 13.1 | 1.9 |