Sentence completion on ArabCulture
42.5Sentence Similarity ScoreQwen2.5-7B-IT (+CCKG N-Asrt)
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Qwen2.5-7B-IT (+CCKG N-Asrt)Augmentation=CCKG, Component=Native-language Assertions2026.01 | 42.5 | 67.8 | |
| Qwen2.5-7B-IT (+CCKG E-Asrt)Augmentation=CCKG, Component=English Assertions2026.01 | 42 | 67.4 | |
| Qwen2.5-7B-IT (+Mango)Augmentation=Mango (5-shot assertions)2026.01 | 39.2 | 67.4 | |
| Qwen2.5-7B-IT (Before Aug)Augmentation=None (Zero-shot baseline)2026.01 | 38.8 | 67.8 | |
| Qwen2.5-7B-IT (+CCKG N-Path)Augmentation=CCKG, Component=Native-language Paths2026.01 | 37.6 | 66.5 | |
| Gemma2-9B-IT (+CCKG N-Asrt)Augmentation=CCKG, Component=Native-language Assertions2026.01 | 37.2 | 68.7 | |
| Gemma2-9B-IT (+CCKG E-Asrt)Augmentation=CCKG, Component=English Assertions2026.01 | 36.5 | 68.7 | |
| Qwen2.5-7B-IT (+CCKG E-Path)Augmentation=CCKG, Component=English Paths2026.01 | 35.3 | 65.7 | |
| Gemma2-9B-IT (+CCKG N-Path)Augmentation=CCKG, Component=Native-language Paths2026.01 | 34.1 | 68.6 | |
| Gemma2-9B-IT (Before Aug)Augmentation=None (Zero-shot baseline)2026.01 | 33.9 | 68.6 | |
| Llama3.1-8B-IT (+CCKG N-Asrt)Augmentation=CCKG, Component=Native-language Assertions2026.01 | 33.5 | 64.5 | |
| Gemma2-9B-IT (+Mango)Augmentation=Mango (5-shot assertions)2026.01 | 33 | 68.4 | |
| Gemma2-9B-IT (+CCKG E-Path)Augmentation=CCKG, Component=English Paths2026.01 | 32.9 | 68.6 | |
| Llama3.1-8B-IT (+CCKG E-Asrt)Augmentation=CCKG, Component=English Assertions2026.01 | 31.2 | 63 | |
| Llama3.1-8B-IT (Before Aug)Augmentation=None (Zero-shot baseline)2026.01 | 29.9 | 65 | |
| Llama3.1-8B-IT (+CCKG N-Path)Augmentation=CCKG, Component=Native-language Paths2026.01 | 29.4 | 64 | |
| Llama3.1-8B-IT (+Mango)Augmentation=Mango (5-shot assertions)2026.01 | 29.1 | 63.5 | |
| Llama3.1-8B-IT (+CCKG E-Path)Augmentation=CCKG, Component=English Paths2026.01 | 26.6 | 63.2 |