Code Generation on KodCode (test)
61.13AccuracyFlashMem
Evaluation results
| Method | Links | |
|---|---|---|
| FlashMemModel Backbone=Qwen 3 4B Instruct, Inference Strategy=FlashMem2026.01 | 61.13 | |
| MemGenModel Backbone=Qwen 3 4B Instruct, Inference Strategy=MemGen2026.01 | 59.77 | |
| CoT-SCModel Backbone=Qwen 3 4B Instruct, Inference Strategy=CoT-SC2026.01 | 57.55 | |
| VanillaModel Backbone=Qwen 3 4B Instruct, Inference Strategy=Vanilla2026.01 | 56.48 | |
| MemGenModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=MemGen2026.01 | 55.7 | |
| FlashMemModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=FlashMem2026.01 | 50.2 | |
| SnapKVModel Backbone=Qwen 3 4B Instruct, Inference Strategy=SnapKV2026.01 | 49.7 | |
| SnapKVModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=SnapKV2026.01 | 39.7 | |
| CoT-SCModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=CoT-SC2026.01 | 29.3 | |
| VanillaModel Backbone=Qwen 2.5 1.5B Instruct, Inference Strategy=Vanilla2026.01 | 22.85 |