Code Generation on HumanEval (held-out problems 80–163)
74.8pass@1 AccuracyState-offset
Evaluation results
| Method | Links | |
|---|---|---|
| State-offsetBackbone=Qwen3.5-4B (GatedDeltaNet), Params=12.6M, Overhead=Per-step2026.04 | 74.8 | |
| S0 (ours)Backbone=Qwen3.5-4B (GatedDeltaNet), Params=12.6M, Overhead=None2026.04 | 72.2 | |
| S0 (ours)Backbone=FalconH1-7B (Mamba-2), Params=34.6M, Overhead=None2026.04 | 71.8 | |
| LoRA (best config)Backbone=FalconH1-7B (Mamba-2), Params=22.2M, Overhead=Merge2026.04 | 71.4 | |
| LoRA (best config)Backbone=Qwen3.5-4B (GatedDeltaNet), Params=4.7M, Overhead=Merge2026.04 | 61.5 | |
| BaselineBackbone=Qwen3.5-4B (GatedDeltaNet), Params=—, Overhead=—2026.04 | 48.8 | |
| BaselineBackbone=FalconH1-7B (Mamba-2), Params=—, Overhead=—2026.04 | 40.5 |