Code Search on Code Search (test)
102,385ITBase
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| BaseModel=GPT-42025.05 | 102,385 | — | 24,535 | — | 127,920 | — | 82 | — | |
| DietCodeModel=GPT-4, Removal Ratio=10%2025.05 | 98,105 | 4.18 | 24,619 | 0.34 | 122,724 | 4.06 | 77.6 | 5.37 | |
| LeanCodeModel=GPT-4, Removal Ratio=10%2025.05 | 96,978 | 5.28 | 25,843 | 5.33 | 122,821 | 3.99 | 79.5 | 3.05 | |
| SlimCodeModel=GPT-4, Removal Ratio=10%2025.05 | 96,845 | 5.41 | 25,343 | 3.29 | 122,188 | 4.48 | 79 | 3.66 | |
| DietCodeModel=GPT-4, Removal Ratio=20%2025.05 | 93,268 | 8.23 | 24,508 | 0.11 | 117,776 | 7.92 | 81.3 | 0.85 | |
| SlimCodeModel=GPT-4, Removal Ratio=20%2025.05 | 91,736 | 10.4 | 25,217 | 2.99 | 116,953 | 8.57 | 80.8 | 1.46 | |
| LeanCodeModel=GPT-4, Removal Ratio=20%2025.05 | 91,173 | 10.95 | 25,043 | 2.07 | 116,216 | 9.15 | 79.8 | 2.68 | |
| DietCodeModel=GPT-4, Removal Ratio=30%2025.05 | 86,672 | 14.61 | 24,211 | 1.32 | 110,883 | 13.32 | 77.5 | 5.49 | |
| SlimCodeModel=GPT-4, Removal Ratio=30%2025.05 | 85,778 | 16.22 | 24,779 | 0.99 | 110,557 | 13.57 | 78.9 | 3.78 | |
| LeanCodeModel=GPT-4, Removal Ratio=30%2025.05 | 85,202 | 16.78 | 25,454 | 3.74 | 110,656 | 13.5 | 82.8 | -0.49 | |
| DietCodeModel=GPT-4, Removal Ratio=40%2025.05 | 80,196 | 20.98 | 24,536 | 0 | 104,732 | 18.14 | 78 | 4.88 | |
| SlimCodeModel=GPT-4, Removal Ratio=40%2025.05 | 79,686 | 22.17 | 24,734 | 0.81 | 104,420 | 18.37 | 79.6 | 2.93 | |
| LeanCodeModel=GPT-4, Removal Ratio=40%2025.05 | 79,102 | 22.74 | 25,162 | 2.55 | 104,264 | 18.49 | 79.3 | 3.54 | |
| DietCodeModel=GPT-4, Removal Ratio=50%2025.05 | 73,759 | 27.76 | 24,169 | 1.49 | 97,928 | 23.44 | 77.6 | 5.37 | |
| SlimCodeModel=GPT-4, Removal Ratio=50%2025.05 | 73,215 | 28.49 | 24,548 | 0.05 | 97,763 | 23.57 | 76.3 | 6.95 | |
| LeanCodeModel=GPT-4, Removal Ratio=50%2025.05 | 72,887 | 28.81 | 24,741 | 0.84 | 97,628 | 23.68 | 81 | 1.22 |