List your company
ResearchFormerly WizWand

Binary Code Summarization on BCS 500 samples (test)

0.8595Win RateC-ing Clearly
0.103940.3000950.496250.692405Dec 16, 2025
Updated 8mo ago

Evaluation results

MethodLinks
C-ing ClearlyOpponent=Llama-3.1-8B-Instruct, Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.8595—
No C-ingOpponent=Llama-3.1-8B-Instruct, Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.826—
Multi-turn C-ing ClearlyOpponent=Llama-3.1-8B-Instruct, Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.784—
C-ing ClearlyOpponent=Multi-turn C-ing Clearly (MCC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.5745—
C-ing ClearlyOpponent=No C-ing (NOC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.5709—
No C-ingOpponent=Multi-turn C-ing Clearly (MCC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.524—
Multi-turn C-ing ClearlyOpponent=No C-ing (NOC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.4665—
No C-ingOpponent=C-ing Clearly (CC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.4115—
Multi-turn C-ing ClearlyOpponent=C-ing Clearly (CC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.4015—
Llama-3.1-8B-InstructOpponent=Multi-turn C-ing Clearly (MCC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.2105—
Llama-3.1-8B-InstructOpponent=No C-ing (NOC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.1645—
Llama-3.1-8B-InstructOpponent=C-ing Clearly (CC), Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.120.133—
C-ing ClearlyOpponent=Average, Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.12—66.8
Llama-3.1-8B-InstructOpponent=Average, Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.12—16.93
Multi-turn C-ing ClearlyOpponent=Average, Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.12—55.07
No C-ingOpponent=Average, Judge=GPT-4o, Sampling=Nucleus, Temperature=0.42025.12—58.71