Linguistic Implicatures Decoding on Open PL LLM Leaderboard Implicatures Component
67.38Average ScoreQwen2.5-72B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen2.5-72BParameters (B)=72.7, Shot count=5-shot, Model category=base models2025.12 | 67.38 | |
| Qwen2.5-32BParameters (B)=32.8, Shot count=5-shot, Model category=base models2025.12 | 66.73 | |
| Qwen-72BParameters (B)=72.7, Shot count=5-shot, Model category=base models2025.12 | 66.02 | |
| Qwen2.5-14BParameters (B)=14.8, Shot count=5-shot, Model category=base models2025.12 | 62.71 | |
| Meta-Llama-3-70BParameters (B)=70.6, Shot count=5-shot, Model category=base models2025.12 | 62.07 | |
| Qwen1.5-72BParameters (B)=72.7, Shot count=5-shot, Model category=base models2025.12 | 61.11 | |
| Meta-Llama-3.1-70BParameters (B)=70.6, Shot count=5-shot, Model category=base models2025.12 | 60.87 | |
| Mixtral-8x22B-v0.1Parameters (B)=141, Shot count=5-shot, Model category=base models2025.12 | 60.75 | |
| Mistral-Small-24B-Base-2501Parameters (B)=24, Shot count=5-shot, Model category=base models2025.12 | 59.9 | |
| Qwen1.5-32BParameters (B)=32.8, Shot count=5-shot, Model category=base models2025.12 | 58.71 | |
| Bielik-11B-v2Parameters (B)=11.2, Shot count=5-shot, Model category=base models2025.12 | 58.14 | |
| Llama-4-Scout-17B-16EParameters (B)=109, Shot count=5-shot, Model category=base models2025.12 | 57.77 | |
| Bielik-11B-v3-Base-20250730Parameters (B)=11.2, Shot count=5-shot, Model category=base models2025.12 | 55.16 | |
| Qwen2.5-7BParameters (B)=7, Shot count=5-shot, Model category=base models2025.12 | 53.35 | |
| EuroLLM-9BParameters (B)=9.2, Shot count=5-shot, Model category=base models2025.12 | 50.03 | |
| Qwen-7BParameters (B)=7, Shot count=5-shot, Model category=base models2025.12 | 49.39 | |
| SOLAR-10.7B-v1.0Parameters (B)=10.7, Shot count=5-shot, Model category=base models2025.12 | 47.54 | |
| Mistral-Nemo-Base-2407Parameters (B)=12.2, Shot count=5-shot, Model category=base models2025.12 | 47.28 | |
| internlm2-20bParameters (B)=20, Shot count=5-shot, Model category=base models2025.12 | 47.15 | |
| Bielik-4.5B-v3Parameters (B)=4.8, Shot count=5-shot, Model category=base models2025.12 | 45.47 | |
| Meta-Llama-3.1-8BParameters (B)=8, Shot count=5-shot, Model category=base models2025.12 | 43.77 | |
| Meta-Llama-3-8BParameters (B)=8, Shot count=5-shot, Model category=base models2025.12 | 43.3 | |
| Qwen1.5-72BParameters (B)=72.3, Shot count=5-shot, Model category=base models2025.12 | 39.51 | |
| Mistral-7B-v0.3Parameters (B)=7, Shot count=5-shot, Model category=base models2025.12 | 38.88 | |
| Mistral-7B-v0.2Parameters (B)=7, Shot count=5-shot, Model category=base models2025.12 | 38.81 | |
| Qwen1.5-7BParameters (B)=7, Shot count=5-shot, Model category=base models2025.12 | 37.92 | |
| Bielik-7B-v0.1Parameters (B)=7.2, Shot count=5-shot, Model category=base models2025.12 | 34.34 | |
| Qra-13bParameters (B)=13, Shot count=5-shot, Model category=base models2025.12 | 33.9 | |
| Llama-3.2-3BParameters (B)=3, Shot count=5-shot, Model category=base models2025.12 | 31.89 | |
| Qra-7bParameters (B)=7, Shot count=5-shot, Model category=base models2025.12 | 16.6 |