Classification on Coverage (test)
92.89PrecisionClaude 3 Sonnet
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Claude 3 SonnetJudge=Claude 3 Sonnet2024.08 | 92.89 | 75.07 | |
| LLaMA3 8BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 89.2 | 82.18 | |
| Gemini 1.5 FlashJudge=Claude 3 Sonnet2024.08 | 89.03 | 77.96 | |
| GPT4oJudge=Claude 3 Sonnet2024.08 | 87.5 | 72.28 | |
| Claude 3 SonnetJudge=Gemini 1.5 Flash2024.08 | 87.34 | 67.18 | |
| Gemini 1.5 FlashJudge=GPT4o2024.08 | 85.43 | 72.73 | |
| LLaMA3 8BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 83.63 | 72.54 | |
| GPT4oJudge=GPT4o2024.08 | 83.62 | 74.45 | |
| Gemini 1.5 FlashJudge=Gemini 1.5 Flash2024.08 | 83.35 | 64.25 | |
| GPT4oJudge=Gemini 1.5 Flash2024.08 | 82.68 | 63.06 | |
| Claude 3 SonnetJudge=GPT4o2024.08 | 82.32 | 71.53 | |
| LLaMA3 8BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 81.64 | 78.02 | |
| Gemma 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 78.78 | 67.76 | |
| Mistral 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 76.01 | 64.43 | |
| Gemma 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 70.73 | 64.67 | |
| Gemma 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 70.73 | 59.77 | |
| Gemma 2BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 69.98 | 56.31 | |
| Mistral 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 67.9 | 54.27 | |
| Mistral 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 67.53 | 61.65 | |
| Gemma 2BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 62.17 | 48.54 | |
| Gemma 2BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 58.47 | 52.76 |