Summarization on Custom Hebrew Summarization (test)
56.86Win Rate (GPT-4o)DictaLM-3.0-24B-Thinking
Evaluation results
| Method | Links | |
|---|---|---|
| DictaLM-3.0-24B-ThinkingSize Category=Large, Reasoning Mode=Thinking2026.02 | 56.86 | |
| gemma3-27B-itSize Category=Large2026.02 | 44.54 | |
| gemma-3-12b-itSize Category=Smaller (~12B)2026.02 | 39.48 | |
| Llama-3.3-70B-InstructSize Category=Large2026.02 | 37.83 | |
| DictaLM-3.0-Nemotron-12B-InstructSize Category=Smaller (~12B)2026.02 | 33.27 | |
| aya-expanse-32BSize Category=Large2026.02 | 29.46 | |
| Qwen3-14B (think)Size Category=Smaller (~12B), Reasoning Mode=Thinking2026.02 | 15.83 | |
| DictaLM-3.0-1.7B-ThinkingSize Category=Tiny (~1.7B), Reasoning Mode=Thinking2026.02 | 10.22 | |
| DictaLM-3.0-1.7B-InstructSize Category=Tiny (~1.7B)2026.02 | 9.72 | |
| Qwen3-1.7B (think)Size Category=Tiny (~1.7B), Reasoning Mode=Thinking2026.02 | 0.4 | |
| gemma-3-1b-itSize Category=Tiny (~1.7B)2026.02 | 0.35 |