Python return type prediction on Pradel benchmarks Fried adaptation (test)
77.4Non-None F1StarCoderBase
Evaluation results
| Method | Links | ||
|---|---|---|---|
| StarCoderBasegeneration=greedy, conditioning=imports, body, and signature2023.05 | 77.4 | 86.6 | |
| StarCodergeneration=greedy, conditioning=imports, body, and signature2023.05 | 77.1 | 86.4 | |
| SantaCodergeneration=greedy, conditioning=imports, body, and signature2023.05 | 66.9 | 78.5 | |
| InCoder-6Bgeneration=greedy, conditioning=imports, body, and signature2023.05 | 59.1 | 46.8 |