Coding on LiveCodeBench v6 (Score (%))
92Score (%)Fugu-Ultra
Evaluation Results
| Method | Links | |
|---|---|---|
| Fugu-Ultra2026.06 | 92 | |
| GPT-5.52026.06 | 90.7 | |
| Fugu2026.06 | 90.3 | |
| Claude Opus 4.82026.06 | 90.3 | |
| Gemini 3.12026.06 | 88.9 | |
| Confident DecodingModel=gpt-oss-120B, Decoding Strategy=Confident (Ours)2026.06 | 84.5 | |
| Last LayerModel=gpt-oss-120B, Decoding Strategy=Last Layer2026.06 | 83.1 | |
| Confident DecodingModel=Qwen3.5-122B-A10B, Decoding Strategy=Confident (Ours)2026.06 | 79.2 | |
| Confident DecodingModel=Gemma-4-31B, Decoding Strategy=Confident (Ours)2026.06 | 78.3 | |
| Confident DecodingModel=gpt-oss-20b, Decoding Strategy=Confident (Ours)2026.06 | 77.6 | |
| Last LayerModel=gpt-oss-20b, Decoding Strategy=Last Layer2026.06 | 77.1 | |
| Last LayerModel=Qwen3.5-122B-A10B, Decoding Strategy=Last Layer2026.06 | 76.8 | |
| Mellum 2 (SFT)Post-training Stage=SFT, Parameters=2.5B/12B, Variant=Thinking2026.05 | 75.1 | |
| Last LayerModel=Gemma-4-31B, Decoding Strategy=Last Layer2026.06 | 75.1 | |
| Confident DecodingModel=Qwen3.5-35B-A3B, Decoding Strategy=Confident (Ours)2026.06 | 74.4 | |
| Confident DecodingModel=Qwen3.5-27B, Decoding Strategy=Confident (Ours)2026.06 | 73.3 | |
| Last LayerModel=Qwen3.5-35B-A3B, Decoding Strategy=Last Layer2026.06 | 70.1 | |
| Mellum 2 (RL)Post-training Stage=RL, Parameters=2.5B/12B, Variant=Thinking2026.05 | 69.9 | |
| Qwen3.5-9BParameters=9B, Variant=Thinking2026.05 | 68.3 | |
| ZAYA1-8BActive / Total Parameters=0.7B / 8.0B, Sampling Settings=T=0.6, top-p=0.95, top-k=202026.05 | 64.8 | |
| Keye-VL-2.0-30B-A3BModel Size=30B-A3B2026.06 | 64.2 | |
| Last LayerModel=Qwen3.5-27B, Decoding Strategy=Last Layer2026.06 | 63.9 | |
| Qwen3.5Model Size=35B-A3B2026.06 | 62.8 | |
| OLMo-3-7BParameters=7B, Variant=Thinking2026.05 | 59.8 | |
| Qwen3.5-4BParameters=4B, Variant=Thinking2026.05 | 59.4 | |
| Qwen3.5-4BActive / Total Parameters=4.0B / 4.0B, Sampling Settings=Recommended sampling settings2026.05 | 55.8 | |
| Qwen3-4B-Thinking-2507Active / Total Parameters=4.0B / 4.0B, Sampling Settings=Recommended sampling settings2026.05 | 54.9 | |
| Gemma-4-E4B-itActive / Total Parameters=4.0B / 8.0B, Sampling Settings=Recommended sampling settings2026.05 | 54.2 | |
| GPT-5-miniRelease Date/Version=2025-08-072026.06 | 51.5 | |
| Ministral-3-14BParameters=14B, Variant=Thinking2026.05 | 42.7 | |
| Pair+Backbone=olmo-3.1-32b-it-sft2026.04 | 29.7 | |
| DPOBackbone=olmo-3.1-32b-it-sft2026.04 | 29 | |
| Mag+Backbone=olmo-3.1-32b-it-sft2026.04 | 28.6 | |
| Soft+Backbone=olmo-3.1-32b-it-sft2026.04 | 27.5 | |
| MPO+Backbone=olmo-3.1-32b-it-sft2026.04 | 26.6 | |
| RFTBackbone=olmo-3.1-32b-it-sft2026.04 | 25.3 | |
| Mag+Backbone=olmo-3-7b-it-sft2026.04 | 22.5 | |
| BaseBackbone=olmo-3.1-32b-it-sft2026.04 | 21.4 | |
| DPOBackbone=olmo-3-7b-it-sft2026.04 | 19.5 | |
| MPO+Backbone=olmo-3-7b-it-sft2026.04 | 19.5 | |
| RFTBackbone=olmo-3-7b-it-sft2026.04 | 18.7 | |
| Pair+Backbone=olmo-3-7b-it-sft2026.04 | 18 | |
| Soft+Backbone=olmo-3-7b-it-sft2026.04 | 16.1 | |
| BaseBackbone=olmo-3-7b-it-sft2026.04 | 14.9 | |
| Mag+Backbone=gemma-3-4b-sft2026.04 | 11.5 | |
| RFTBackbone=gemma-3-4b-sft2026.04 | 10.7 | |
| MPO+Backbone=gemma-3-4b-sft2026.04 | 10.7 | |
| DPOBackbone=gemma-3-4b-sft2026.04 | 10.6 | |
| Pair+Backbone=gemma-3-4b-sft2026.04 | 10.2 | |
| Soft+Backbone=gemma-3-4b-sft2026.04 | 9.8 | |
| BaseBackbone=gemma-3-4b-sft2026.04 | 9.4 |