List your company
ResearchFormerly WizWand

Code Evaluation on CodeMMLU software principles

24.9AccuracyDPO + RM
12.62815.8141922.186Apr 7, 2025
Updated 3mo ago

Evaluation results

MethodLinks
DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-GEMMA-27B2025.0424.90.9
DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-LLAMA-8B2025.0423.10.7
DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-GEMMA-27B2025.0423.10.8
DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-GEMMA-27B2025.04230.9
DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-LLAMA-8B2025.0420.91
DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Sampling Majority@16, Reward Model=SK-LLAMA-8B2025.0420.30.7
DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-GEMMA-27B2025.0417.42.1
DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-GEMMA-27B2025.0417.42.4
DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-GEMMA-27B2025.0417.31.8
DPO + RMDBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-LLAMA-8B2025.0415.32
DPO + RMBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-LLAMA-8B2025.0415.21.8
DPO + RMCBase Model=LLAMA-3.1-8B, Decoding=Greedy, Reward Model=SK-LLAMA-8B2025.0413.11.9