Tulu Generation Evaluation on Tulu
85Grammar AccuracyGemini 2.0 Flash
Evaluation Results
| Method | Links | ||
|---|---|---|---|
| Gemini 2.0 FlashPrompt Strategy=Full System2026.02 | 85 | 5 | |
| GPT-4oPrompt Strategy=Full System2026.02 | 82 | 7 | |
| Llama 3.1 70BPrompt Strategy=Full System2026.02 | 78 | 6 | |
| Gemini 2.0 FlashPrompt Strategy=+Gram+Const2026.02 | 72 | 10 | |
| GPT-4oPrompt Strategy=+Gram+Const2026.02 | 70 | 12 | |
| Llama 3.1 70BPrompt Strategy=+Gram+Const2026.02 | 65 | 15 | |
| Gemini 2.0 FlashPrompt Strategy=+Grammar2026.02 | 63 | 28 | |
| GPT-4oPrompt Strategy=+Grammar2026.02 | 62 | 32 | |
| Llama 3.1 70BPrompt Strategy=+Grammar2026.02 | 58 | 35 | |
| Gemini 2.0 FlashPrompt Strategy=Baseline2026.02 | 25 | 75 | |
| GPT-4oPrompt Strategy=Baseline2026.02 | 20 | 80 | |
| Llama 3.1 70BPrompt Strategy=Baseline2026.02 | 15 | 82 |