ResearchBenchmarksSoftware Engineering Tasks on Claude CodeSoftware Engineering Tasks on Claude CodeFollow40Successful Tasks CountOriginal28.5631.5334.537.47May 5, 2026Evaluation resultsMethodMethodLinksSuccessful Tasks CountInput Tokens (M)Output Tokens (K)Cache Tokens (M)Total Tokens (M)Average Tokens per Task (M)OriginalCondition=OriginalCondition=Original2026.054032.957430.133.40.84VanillaCondition=VanillaCondition=Vanilla2026.053419.542117.219.90.59CompiledCondition=CompiledCondition=Compiled2026.052918.345915.818.70.65