Trace Extraction on CoRe Lite Control-Dependency Trace subtask n=489
94.58F1 ScoreGemini 2.5 Pro (Reproduction)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Gemini 2.5 Pro (Reproduction)Pipeline=PDF→CT→code, Snapshot=gemini-2.5-pro (current production endpoint), Context Handling=no-PDF (skeleton files only), Delta (Δ) F1 Score=+2.09, Delta (Δ) Correct Trace Rate=+7.742026.05 | 94.58 | 100 | |
| Gemini 2.5 Pro (Reported)Pipeline=Reported, Snapshot=gemini-2.5-pro-preview-05-06, Evaluation Framework=CoRe paper Table 42026.05 | 92.49 | 92.26 | |
| Gemini 2.5 Pro (Reproduction)Pipeline=PDF→code (direct), Snapshot=gemini-2.5-pro (current production endpoint), Prompts=zero guiding prompts + 2 scope/efficiency prompts, Delta (Δ) F1 Score=-12.29, Delta (Δ) Correct Trace Rate=+4.672026.05 | 80.2 | 96.93 |