Summarization on Coverage dataset (test)
93.39PrecisionClaude 3 Sonnet
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Claude 3 SonnetJudge=Claude 3 Sonnet2024.08 | 93.39 | 85.55 | |
| GPT4oJudge=Claude 3 Sonnet2024.08 | 93.25 | 86.6 | |
| Gemini 1.5 FlashJudge=Claude 3 Sonnet2024.08 | 91.95 | 85.05 | |
| GPT4oJudge=GPT4o2024.08 | 90.71 | 82 | |
| Claude 3 SonnetJudge=GPT4o2024.08 | 88.04 | 78.18 | |
| Gemini 1.5 FlashJudge=GPT4o2024.08 | 87.9 | 79.14 | |
| GPT4oJudge=Gemini 1.5 Flash2024.08 | 87.1 | 67.45 | |
| LLaMA3 8BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 87.02 | 78.42 | |
| Claude 3 SonnetJudge=Gemini 1.5 Flash2024.08 | 86.7 | 64.1 | |
| Mistral 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 86.2 | 78.44 | |
| Gemma 7BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 86.19 | 77.41 | |
| Gemini 1.5 FlashJudge=Gemini 1.5 Flash2024.08 | 85.65 | 66.45 | |
| Mistral 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 76.38 | 69.65 | |
| LLaMA3 8BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 75.67 | 70.54 | |
| Gemma 2BJudge=Claude 3 Sonnet, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 74.89 | 64.09 | |
| Gemma 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 74.59 | 53.92 | |
| Gemma 7BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 73.54 | 68.58 | |
| Mistral 7BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 72.74 | 50.15 | |
| LLaMA3 8BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 72.74 | 52.23 | |
| Gemma 2BJudge=Gemini 1.5 Flash, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 61.9 | 42.15 | |
| Gemma 2BJudge=GPT4o, Fine-tuned=128K synthetic dataset via LlamaDuo2024.08 | 57.6 | 54.49 |