Polish Text Understanding on CPTUB
4.39Overall Avggemini-2.0-flash-001
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| gemini-2.0-flash-001Params (B)=4.292025.12 | 4.39 | 4.52 | 4.32 | 4.34 | 3.99 | — | |
| gemini-2.0-flash-001Params (B)=Unknown2026.03 | 4.29 | 4.39 | 4.52 | 4.32 | 4.34 | 3.99 | |
| gemini-2.0-flash-001Params (B)=Unknown2026.04 | 4.29 | 4.39 | 4.52 | 4.32 | 4.34 | 3.99 | |
| gemini-2.0-flash-lite-001Params (B)=4.092025.12 | 4.17 | 4.23 | 4.05 | 4.24 | 3.85 | — | |
| DeepSeek-R1Params (B)=685.02025.12 | 4.14 | 4.14 | 4.49 | 4.35 | 3.6 | 4.12 | |
| DeepSeek-R1Params (B)=685.02026.03 | 4.14 | 4.14 | 4.49 | 4.35 | 3.6 | 4.12 | |
| DeepSeek-R1Params (B)=685.02026.04 | 4.14 | 4.14 | 4.49 | 4.35 | 3.6 | 4.12 | |
| gemini-2.0-flash-lite-001Params (B)=Unknown2026.03 | 4.09 | 4.17 | 4.23 | 4.05 | 4.24 | 3.85 | |
| gemini-2.0-flash-lite-001Params (B)=Unknown2026.04 | 4.09 | 4.17 | 4.23 | 4.05 | 4.24 | 3.85 | |
| DeepSeek-V3-0324Params (B)=685.02025.12 | 4.03 | 4.03 | 4.36 | 4.2 | 3.54 | 4.02 | |
| DeepSeek-V3-0324Params (B)=685.02026.03 | 4.03 | 4.03 | 4.36 | 4.2 | 3.54 | 4.02 | |
| DeepSeek-V3-0324Params (B)=685.02026.04 | 4.03 | 4.03 | 4.36 | 4.2 | 3.54 | 4.02 | |
| Mistral-Large-Instruct-2411Params (B)=123.0, license=non-commercial2025.12 | 4 | 4.1 | 4.33 | 3.98 | 3.99 | 3.72 | |
| Mistral-Large-Instruct-2411†Params (B)=123.02026.03 | 4 | 4.1 | 4.33 | 3.98 | 3.99 | 3.72 | |
| Mistral-Large-Instruct-2411†Params (B)=123.02026.04 | 4 | 4.1 | 4.33 | 3.98 | 3.99 | 3.72 | |
| Qwen2.5-72B-InstructParams (B)=72.72025.12 | 3.95 | 3.99 | 4.08 | 3.97 | 3.93 | 3.81 | |
| Qwen2.5-72B-InstructParams (B)=72.72026.03 | 3.95 | 3.99 | 4.08 | 3.97 | 3.93 | 3.81 | |
| Qwen2.5-72B-InstructParams (B)=72.72026.04 | 3.95 | 3.99 | 4.08 | 3.97 | 3.93 | 3.81 | |
| Mistral-Large-Instruct-2407Params (B)=123.0, license=non-commercial2025.12 | 3.93 | 4.03 | 4.23 | 4 | 3.86 | 3.65 | |
| Llama-4-Maverick-17B-128E-InstructParams (B)=402.02025.12 | 3.93 | 3.99 | 4.39 | 4.11 | 3.48 | 3.76 | |
| Mistral-Large-Instruct-2407†Params (B)=123.02026.03 | 3.93 | 4.03 | 4.23 | 4 | 3.86 | 3.65 | |
| Llama-4-Maverick-17B-128E-InstructParams (B)=402.02026.03 | 3.93 | 3.99 | 4.39 | 4.11 | 3.48 | 3.76 | |
| Mistral-Large-Instruct-2407†Params (B)=123.02026.04 | 3.93 | 4.03 | 4.23 | 4 | 3.86 | 3.65 | |
| Llama-4-Maverick-17B-128E-InstructParams (B)=402.02026.04 | 3.93 | 3.99 | 4.39 | 4.11 | 3.48 | 3.76 | |
| gemma-3-27b-itParams (B)=27.42025.12 | 3.81 | 3.9 | 3.88 | 3.79 | 4.03 | 3.53 | |
| gemma-3-27b-itParams (B)=27.42026.03 | 3.81 | 3.9 | 3.88 | 3.79 | 4.03 | 3.53 | |
| gemma-3-27b-itParams (B)=27.42026.04 | 3.81 | 3.9 | 3.88 | 3.79 | 4.03 | 3.53 | |
| Bielik-PL-11B-v3.0-InstructParams (B)=11.22026.04 | 3.8 | 4.02 | 4.05 | 4.03 | 3.98 | 3.13 | |
| Meta-Llama-3-70B-InstructParams (B)=70.62025.12 | 3.78 | 3.81 | 4.13 | 3.82 | 3.47 | 3.71 | |
| Meta-Llama-3-70B-InstructParams (B)=70.62026.03 | 3.78 | 3.81 | 4.13 | 3.82 | 3.47 | 3.71 | |
| Meta-Llama-3-70B-InstructParams (B)=70.62026.04 | 3.78 | 3.81 | 4.13 | 3.82 | 3.47 | 3.71 | |
| Qwen2.5-32B-InstructParams (B)=32.82025.12 | 3.75 | 3.8 | 3.81 | 3.57 | 4.04 | 3.59 | |
| Llama-4-Scout-17B-16E-InstructParams (B)=109.02025.12 | 3.75 | 3.94 | 4.1 | 3.81 | 3.9 | 3.19 | |
| Qwen2.5-32B-InstructParams (B)=32.82026.03 | 3.75 | 3.8 | 3.81 | 3.57 | 4.04 | 3.59 | |
| Llama-4-Scout-17B-16E-InstructParams (B)=109.02026.03 | 3.75 | 3.94 | 4.1 | 3.81 | 3.9 | 3.19 | |
| Qwen2.5-32B-InstructParams (B)=32.82026.04 | 3.75 | 3.8 | 3.81 | 3.57 | 4.04 | 3.59 | |
| Llama-4-Scout-17B-16E-InstructParams (B)=109.02026.04 | 3.75 | 3.94 | 4.1 | 3.81 | 3.9 | 3.19 | |
| Bielik-11B-v3.0-InstructParams (B)=11.22025.12 | 3.73 | 3.92 | 3.88 | 3.91 | 3.96 | 3.19 | |
| Bielik-11B-v3.0-InstructParams (B)=11.22026.03 | 3.73 | 3.92 | 3.88 | 3.91 | 3.96 | 3.19 | |
| Bielik-11B-v3.0-InstructParams (B)=11.22026.04 | 3.73 | 3.92 | 3.88 | 3.91 | 3.96 | 3.19 | |
| Mistral-Small-24B-Instruct-2501Params (B)=23.62025.12 | 3.71 | 3.8 | 3.91 | 3.6 | 3.88 | 3.45 | |
| Mistral-Small-24B-Instruct-2501Params (B)=23.62026.03 | 3.71 | 3.8 | 3.91 | 3.6 | 3.88 | 3.45 | |
| Mistral-Small-24B-Instruct-2501Params (B)=23.62026.04 | 3.71 | 3.8 | 3.91 | 3.6 | 3.88 | 3.45 | |
| pllum-12b-nc-chat-250715Params (B)=12.2, license=non-commercial2025.12 | 3.67 | 3.92 | 4.36 | 3.96 | 3.46 | 2.9 | |
| pllum-12b-nc-chat-250715†Params (B)=12.22026.03 | 3.67 | 3.92 | 4.36 | 3.96 | 3.46 | 2.9 | |
| pllum-12b-nc-chat-250715†Params (B)=12.22026.04 | 3.67 | 3.92 | 4.36 | 3.96 | 3.46 | 2.9 | |
| Bielik-11B-v2.6-InstructParams (B)=11.22025.12 | 3.64 | 3.82 | 4.1 | 3.94 | 3.41 | 3.1 | |
| Bielik-11B-v2.6-InstructParams (B)=11.22026.03 | 3.64 | 3.82 | 4.1 | 3.94 | 3.41 | 3.1 | |
| Bielik-11B-v2.6-InstructParams (B)=11.22026.04 | 3.64 | 3.82 | 4.1 | 3.94 | 3.41 | 3.1 | |
| Mixtral-8x22B-Instruct-v0.1Params (B)=141.02025.12 | 3.56 | 3.67 | 3.78 | 3.68 | 3.55 | 3.24 | |
| Mixtral-8x22B-Instruct-v0.1Params (B)=141.02026.03 | 3.56 | 3.67 | 3.78 | 3.68 | 3.55 | 3.24 | |
| Mixtral-8x22B-Instruct-v0.1Params (B)=141.02026.04 | 3.56 | 3.67 | 3.78 | 3.68 | 3.55 | 3.24 | |
| Qwen2.5-14B-InstructParams (B)=14.82025.12 | 3.55 | 3.62 | 3.91 | 3.57 | 3.37 | 3.34 | |
| Qwen2.5-14B-InstructParams (B)=14.82026.03 | 3.55 | 3.62 | 3.91 | 3.57 | 3.37 | 3.34 | |
| Qwen2.5-14B-InstructParams (B)=14.82026.04 | 3.55 | 3.62 | 3.91 | 3.57 | 3.37 | 3.34 | |
| Bielik-PL-Minitron-7B-v3.0-InstructParams (B)=7.52026.04 | 3.55 | 3.87 | 3.88 | 3.82 | 3.92 | 2.58 | |
| Llama-PLLuM-70B-chatParams (B)=70.62025.12 | 3.53 | 3.63 | 3.94 | 3.61 | 3.35 | 3.21 | |
| Llama-PLLuM-70B-chatParams (B)=70.62026.03 | 3.53 | 3.63 | 3.94 | 3.61 | 3.35 | 3.21 | |
| Llama-PLLuM-70B-chatParams (B)=70.62026.04 | 3.53 | 3.63 | 3.94 | 3.61 | 3.35 | 3.21 | |
| Bielik-4.5B-v3.0-InstructParams (B)=4.82025.12 | 3.38 | 3.68 | 3.76 | 3.61 | 3.67 | 2.46 | |
| Bielik-Minitron-7B-v3.0-InstructParams (B)=7.352026.03 | 3.38 | 3.59 | 3.72 | 3.83 | 3.23 | 2.74 | |
| Bielik-4.5B-v3.0-InstructParams (B)=4.82026.03 | 3.38 | 3.68 | 3.76 | 3.61 | 3.67 | 2.46 | |
| Bielik-4.5B-v3.0-InstructParams (B)=4.82026.04 | 3.38 | 3.68 | 3.76 | 3.61 | 3.67 | 2.46 | |
| Bielik-Minitron-7B-v3.0-InstructParams (B)=7.52026.04 | 3.38 | 3.59 | 3.72 | 3.83 | 3.23 | 2.74 | |
| Llama-PLLuM-70B-instructParams (B)=70.62026.03 | 3.33 | 3.56 | 3.78 | 3.63 | 3.26 | 2.63 | |
| phi-4Params (B)=14.72025.12 | 3.3 | 3.5 | 3.72 | 3.54 | 3.24 | 2.72 | |
| phi-4Params (B)=14.72026.03 | 3.3 | 3.5 | 3.72 | 3.54 | 3.24 | 2.72 | |
| phi-4Params (B)=14.72026.04 | 3.3 | 3.5 | 3.72 | 3.54 | 3.24 | 2.72 | |
| EuroLLM-9B-InstructParams (B)=9.02025.12 | 3.15 | 3.28 | 3.37 | 3.3 | 3.17 | 2.75 | |
| EuroLLM-9B-InstructParams (B)=9.02026.03 | 3.15 | 3.28 | 3.37 | 3.3 | 3.17 | 2.75 | |
| EuroLLM-9B-InstructParams (B)=9.22026.04 | 3.15 | 3.28 | 3.37 | 3.3 | 3.17 | 2.75 | |
| PLLuM-12B-chatParams (B)=12.22025.12 | 3.14 | 3.32 | 3.32 | 3.21 | 3.43 | 2.59 | |
| PLLuM-12B-chatParams (B)=12.22026.03 | 3.14 | 3.32 | 3.32 | 3.21 | 3.43 | 2.59 | |
| PLLuM-12B-chatParams (B)=12.22026.04 | 3.14 | 3.32 | 3.32 | 3.21 | 3.43 | 2.59 | |
| PLLuM-8x7B-nc-instructParams (B)=46.7, license=non-commercial2025.12 | 3.11 | 3.56 | 3.88 | 3.59 | 3.22 | 1.76 | |
| PLLuM-8x7B-nc-instruct†Params (B)=46.72026.03 | 3.11 | 3.56 | 3.88 | 3.59 | 3.22 | 1.76 | |
| PLLuM-8x7B-nc-instruct†Params (B)=46.72026.04 | 3.11 | 3.56 | 3.88 | 3.59 | 3.22 | 1.76 | |
| Qwen2.5-7B-InstructParams (B)=7.622025.12 | 3.07 | 3.23 | 3.56 | 3.03 | 3.1 | 2.58 | |
| Qwen2.5-7B-InstructParams (B)=7.622026.03 | 3.07 | 3.23 | 3.56 | 3.03 | 3.1 | 2.58 | |
| Qwen2.5-7B-InstructParams (B)=7.62026.04 | 3.07 | 3.23 | 3.56 | 3.03 | 3.1 | 2.58 | |
| PLLuM-8x7B-nc-chatParams (B)=46.7, license=non-commercial2025.12 | 3.03 | 3.44 | 3.76 | 3.48 | 3.08 | 1.8 | |
| PLLuM-8x7B-nc-chat†Params (B)=46.72026.03 | 3.03 | 3.44 | 3.76 | 3.48 | 3.08 | 1.8 | |
| PLLuM-8x7B-nc-chat†Params (B)=46.72026.04 | 3.03 | 3.44 | 3.76 | 3.48 | 3.08 | 1.8 | |
| Meta-Llama-3.1-8B-InstructParams (B)=8.02025.12 | 3.01 | 3.31 | 3.97 | 3.38 | 2.58 | 2.11 | |
| PLLuM-8x7B-chatParams (B)=46.72025.12 | 3.01 | 3.41 | 3.44 | 3.45 | 3.35 | 1.78 | |
| Meta-Llama-3.1-8B-InstructParams (B)=8.02026.03 | 3.01 | 3.31 | 3.97 | 3.38 | 2.58 | 2.11 | |
| PLLuM-8x7B-chatParams (B)=46.72026.03 | 3.01 | 3.41 | 3.44 | 3.45 | 3.35 | 1.78 | |
| Meta-Llama-3.1-8B-InstructParams (B)=8.02026.04 | 3.01 | 3.31 | 3.97 | 3.38 | 2.58 | 2.11 | |
| PLLuM-8x7B-chatParams (B)=46.72026.04 | 3.01 | 3.41 | 3.44 | 3.45 | 3.35 | 1.78 | |
| Meta-Llama-3-8B-InstructParams (B)=8.02025.12 | 3 | 3.17 | 3.33 | 3.15 | 3.04 | 2.48 | |
| Meta-Llama-3-8B-InstructParams (B)=8.02026.03 | 3 | 3.17 | 3.33 | 3.15 | 3.04 | 2.48 | |
| Meta-Llama-3-8B-InstructParams (B)=8.02026.04 | 3 | 3.17 | 3.33 | 3.15 | 3.04 | 2.48 | |
| Llama-PLLuM-8B-chatParams (B)=8.02025.12 | 2.92 | 3.14 | 3.13 | 2.93 | 3.36 | 2.25 | |
| Llama-PLLuM-8B-chatParams (B)=8.02026.03 | 2.92 | 3.14 | 3.13 | 2.93 | 3.36 | 2.25 | |
| Llama-PLLuM-8B-chatParams (B)=8.02026.04 | 2.92 | 3.14 | 3.13 | 2.93 | 3.36 | 2.25 | |
| Bielik-7B-Instruct-v0.1Params (B)=7.22025.12 | 2.88 | 3.13 | 3.59 | 3.48 | 2.32 | 2.16 | |
| Bielik-7B-Instruct-v0.1Params (B)=7.22026.03 | 2.88 | 3.13 | 3.59 | 3.48 | 2.32 | 2.16 | |
| Bielik-7B-Instruct-v0.1Params (B)=7.22026.04 | 2.88 | 3.13 | 3.59 | 3.48 | 2.32 | 2.16 |