Code Execution Output Prediction on LiveCodeBench
90.92AccuracyVanilla
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VanillaModel=Qwen3-4B-Thinking-2507, Calibration Threshold Setup=calibrated on Bespoke-Stratos-17k without recalibration2026.06 | 90.92 | 2,898 | |
| DEERModel=Qwen3-4B-Thinking-2507, Calibration Threshold Setup=calibrated on Bespoke-Stratos-17k without recalibration2026.06 | 86.6 | 2,683 | |
| CUSUMModel=Qwen3-4B-Thinking-2507, Calibration Threshold Setup=calibrated on Bespoke-Stratos-17k without recalibration2026.06 | 86.6 | 2,621 | |
| DynasorModel=Qwen3-4B-Thinking-2507, Calibration Threshold Setup=calibrated on Bespoke-Stratos-17k without recalibration2026.06 | 85.9 | 2,596 |