Code Reasoning on Copilot
85.36AccuracyChaCAL
Evaluation results
| Method | Links | |
|---|---|---|
| ChaCALEvaluation protocol=fine-tuned, Substitution protocol=controlled substitution, Modified layer=layer 22026.05 | 85.36 | |
| Block-ChaCALBlock size=m=n//4, Evaluation protocol=fine-tuned, Substitution protocol=controlled substitution, Modified layer=layer 22026.05 | 83.67 | |
| Block-ChaCALBlock size=m=n//3, Evaluation protocol=fine-tuned, Substitution protocol=controlled substitution, Modified layer=layer 22026.05 | 83.65 | |
| TransformerEvaluation protocol=fine-tuned, Substitution protocol=controlled substitution, Modified layer=layer 22026.05 | 82.4 |