Closed QA on Coverage (test)
98.25PrecisionGemini 1.5 Flash
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Gemini 1.5 FlashJudge=Claude 3 Sonnet2024.08 | 98.25 | 97.41 | |
| Claude 3 SonnetJudge=Claude 3 Sonnet2024.08 | 97.6 | 93.95 | |
| GPT4oJudge=Claude 3 Sonnet2024.08 | 97.21 | 92.56 | |
| GPT4oJudge=GPT4o2024.08 | 95.45 | 84.23 | |
| Gemini 1.5 FlashJudge=Gemini 1.5 Flash2024.08 | 95 | 85.66 | |
| Gemini 1.5 FlashJudge=GPT4o2024.08 | 94.63 | 87.43 | |
| Claude 3 SonnetJudge=GPT4o2024.08 | 94.03 | 85.28 | |
| Claude 3 SonnetJudge=Gemini 1.5 Flash2024.08 | 93.66 | 76.33 | |
| GPT4oJudge=Gemini 1.5 Flash2024.08 | 93.58 | 75.58 | |
| Gemma 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 88.83 | 83.95 | |
| Mistral 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 88.25 | 84.77 | |
| LLaMA3 8BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 86.03 | 77.64 | |
| Mistral 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 83.63 | 81.36 | |
| Gemma 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 82.51 | 72.24 | |
| Mistral 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 82.04 | 73.95 | |
| Gemma 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 81.85 | 79.02 | |
| Gemma 2BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 80.22 | 70.2 | |
| LLaMA3 8BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 79.09 | 68.78 | |
| LLaMA3 8BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 75.55 | 72.62 | |
| Gemma 2BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 70.33 | 59.68 | |
| Gemma 2BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 67.25 | 67.41 |