Coding on Coverage (test)
94.57PrecisionGPT4o
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| GPT4oJudge=Claude 3 Sonnet2024.08 | 94.57 | 86.32 | |
| Claude 3 SonnetJudge=Claude 3 Sonnet2024.08 | 93.82 | 87.42 | |
| Gemini 1.5 FlashJudge=Claude 3 Sonnet2024.08 | 93.59 | 82.92 | |
| GPT4oJudge=Gemini 1.5 Flash2024.08 | 90.78 | 58.43 | |
| Gemini 1.5 FlashJudge=Gemini 1.5 Flash2024.08 | 90.62 | 64.21 | |
| Gemma 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 90.47 | 80.26 | |
| GPT4oJudge=GPT4o2024.08 | 90.31 | 75.18 | |
| Claude 3 SonnetJudge=Gemini 1.5 Flash2024.08 | 89.84 | 60.46 | |
| Claude 3 SonnetJudge=GPT4o2024.08 | 88.76 | 75.23 | |
| Gemini 1.5 FlashJudge=GPT4o2024.08 | 88.51 | 75.62 | |
| Gemma 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 84.66 | 61.23 | |
| LLaMA3 8BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 83.97 | 73.51 | |
| Mistral 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 81.25 | 69.1 | |
| Gemma 2BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 80.92 | 67.24 | |
| Gemma 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 80.56 | 71.92 | |
| LLaMA3 8BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 77.47 | 69.46 | |
| LLaMA3 8BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 75.55 | 51.1 | |
| Gemma 2BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 72.98 | 44.08 | |
| Mistral 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 72.39 | 45.25 | |
| Mistral 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 68.32 | 61.01 | |
| Gemma 2BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 62.31 | 56.48 |