List your company
ResearchFormerly WizWand

Code Generation on LiveCodeBench (pass@1, pass@3, pass@5)

1,784Pass@1Bifrost
-71.36410.328921,373.68Dec 22, 2025Jan 3, 2026Jan 15, 2026Jan 28, 2026Feb 9, 2026Feb 21, 2026Mar 6, 2026
Updated 7mo ago

Evaluation results

MethodLinks
BifrostModel=Llama-3.1-8B-Instruct2026.021,7842,0982,636
PaprikaModel=Llama-3.1-8B-Instruct2026.021,4982,0302,261
BifrostModel=Llama-3.2-3B-Instruct2026.021,3501,8932,148
ICLModel=Llama-3.1-8B-Instruct2026.021,2801,4741,822
BoTModel=Llama-3.1-8B-Instruct2026.021,0411,6391,886
ICLModel=Llama-3.2-3B-Instruct2026.021,0231,4001,545
BaseModel=Llama-3.1-8B-Instruct2026.029861,2361,575
BaseModel=Llama-3.2-3B-Instruct2026.028681,2201,398
PaprikaModel=Llama-3.2-3B-Instruct2026.028591,2331,420
RISEModel=Llama-3.1-8B-Instruct2026.027231,4261,795
ReflexionModel=Llama-3.1-8B-Instruct2026.02636807852
BoTModel=Llama-3.2-3B-Instruct2026.02570893989
RISEModel=Llama-3.2-3B-Instruct2026.024399021,205
BifrostModel=Llama-3.2-1B-Instruct2026.02405774955
DoTModel=Llama-3.2-3B-Instruct2026.02273295295
ICLModel=Llama-3.2-1B-Instruct2026.02184363466
DoTModel=Llama-3.1-8B-Instruct2026.02159182205
PaprikaModel=Llama-3.2-1B-Instruct2026.02130255330
ReflexionModel=Llama-3.2-3B-Instruct2026.02125159170
BaseModel=Llama-3.2-1B-Instruct2026.02107211284
RISEModel=Llama-3.2-1B-Instruct2026.0266172250
Gemini-2.5-ProInstitution=Google2026.0362.01——
ReflexiCoder-8B (Multiple)Institution=Ours, setup=full iterative reasoning-reflection setup2026.0354.12——
ReflexiCoder-8B (Single)Institution=Ours, setup=single-attempt without system prompt2026.0352.21——
Claude-Sonnet-4.5Institution=Anthropic2026.0350.78——
GPT-5.1Institution=OpenAI2026.0348.03——
GPT-4.1Institution=OpenAI2026.0342.77——
Qwen3-8BInstitution=Alibaba2026.0337.75——
DeepCoder-14B-PreviewInstitution=rLLM2026.0334.05——
BoTModel=Llama-3.2-1B-Instruct2026.022769102
BF16Model=DeepSeek-R1-Distill-Qwen-7B, Bit-width=BF16, Maximum number of generation tokens=32,7682025.1223.08——
DeepCoder-1.5B-PreviewInstitution=rLLM2026.0322.94——
Seed-Coder-8B-InstructInstitution=ByteDance2026.0321.03——
KIVIModel=DeepSeek-R1-Distill-Qwen-7B, Bit-width=KV4, Maximum number of generation tokens=32,7682025.1220.88——
MixKVQModel=DeepSeek-R1-Distill-Qwen-7B, Bit-width=C3.4, Maximum number of generation tokens=32,7682025.1220.33——
KVQuantModel=DeepSeek-R1-Distill-Qwen-7B, Bit-width=KV4, Maximum number of generation tokens=32,7682025.1219.44——
KVTunerModel=DeepSeek-R1-Distill-Qwen-7B, Bit-width=C3.92, Maximum number of generation tokens=32,7682025.1219.23——
RotateKVModel=DeepSeek-R1-Distill-Qwen-7B, Bit-width=KV4, Maximum number of generation tokens=32,7682025.1218.33——
KIVIModel=DeepSeek-R1-Distill-Qwen-7B, Bit-width=KV2, Maximum number of generation tokens=32,7682025.1216.48——
Qwen2.5-Coder-7B-InstructInstitution=Alibaba2026.0313.86——
BifrostModel=Llama-3.2-3B-Instruct2026.0212.7316.9918.75
ReflexionModel=Llama-3.2-1B-Instruct2026.02111111
PaprikaModel=Llama-3.2-3B-Instruct2026.029.3312.3713.7
ICLModel=Llama-3.2-3B-Instruct2026.028.7511.9313.64
DeepSeek-Coder-7B-InstructInstitution=DeepSeek2026.038.24——
CodeGemma-7B-ITInstitution=Google2026.038.12——
KVQuantModel=DeepSeek-R1-Distill-Qwen-7B, Bit-width=KV2, Maximum number of generation tokens=32,7682025.125.49——
RISEModel=Llama-3.2-3B-Instruct2026.024.818.8510.74
BoTModel=Llama-3.2-3B-Instruct2026.024.677.418.52
CodeLlama-7b-InstructInstitution=Meta2026.031.19——
DoTModel=Llama-3.2-1B-Instruct2026.02000