# General Language Capability on K&R, IFEval-PT, HumanEval

Source: [General Language Capability on K&R, IFEval-PT, HumanEval](<https://www.sota2.com/research/sota/general-language-capability-on-k-and-r-ifeval-pt-humaneval>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-03-05T07:37:32.458Z

Aggregate evaluation assessing general language capabilities across foundational reasoning (K&R), instruction following (IFEval-PT), and coding tasks (HumanEval).

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

| Scope | Recorded value |
| --- | --- |
| Task | General Language Capability |
| Dataset (including any stated split) | Aggregate K&R, IFEval-PT, HumanEval |
| Metric | Average Score |
| Unit | SCALAR |
| Direction | Higher is better |
| Results loaded | 14 of 14 |
| Coverage | All results currently returned for this benchmark |

Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol.

## Metrics

- [Average Score](<https://www.sota2.com/research/sota/general-language-capability-on-k-and-r-ifeval-pt-humaneval.md?metric=Average%20Score>): Higher is better; unit: SCALAR.

## Recorded results

| Method | Average Score | Result date | Report paper |
| --- | --- | --- | --- |
| Tucano2-qwen-3.7B-Instruct | 53.64 | Not recorded | 69a90de92d603e6de3e40680 |
| Jurema-7B | 53.03 | Not recorded | 69a90de92d603e6de3e40680 |
| Qwen2.5-3B-Instruct | 51.71 | Not recorded | 69a90de92d603e6de3e40680 |
| Qwen3-4B | 51.36 | Not recorded | 69a90de92d603e6de3e40680 |
| Gemma-3-Gaia-PT-BR-4b-it | 49.93 | Not recorded | 69a90de92d603e6de3e40680 |
| SmolLM3-3B | 49.54 | Not recorded | 69a90de92d603e6de3e40680 |
| Llama-3.2-3B-Instruct | 45.82 | Not recorded | 69a90de92d603e6de3e40680 |
| Qwen2.5-1.5B-Instruct | 41.39 | Not recorded | 69a90de92d603e6de3e40680 |
| Tucano2-qwen-1.5B-Instruct | 37.54 | Not recorded | 69a90de92d603e6de3e40680 |
| Qwen3-1.7B | 36.3 | Not recorded | 69a90de92d603e6de3e40680 |
| Tucano2-qwen-0.5B-Instruct | 26.08 | Not recorded | 69a90de92d603e6de3e40680 |
| Qwen3-0.6B | 22.21 | Not recorded | 69a90de92d603e6de3e40680 |
| Llama-3.2-1B-Instruct | 20.14 | Not recorded | 69a90de92d603e6de3e40680 |
| Qwen2.5-0.5B-Instruct | 17.8 | Not recorded | 69a90de92d603e6de3e40680 |

### Tucano2-qwen-3.7B-Instruct

Average Score: 53.64. Result ID: 69a9329d6dbb7075b952429d.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"3.7B"}. This record does not assert independent reproduction.

### Jurema-7B

Average Score: 53.03. Result ID: 69a9329d6dbb7075b952429e.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"7B"}. This record does not assert independent reproduction.

### Qwen2.5-3B-Instruct

Average Score: 51.71. Result ID: 69a9329d6dbb7075b952429f.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa767e1e209131cea6d6a7.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"3B"}. This record does not assert independent reproduction.

### Qwen3-4B

Average Score: 51.36. Result ID: 69a9329d6dbb7075b95242a0.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"4B"}. This record does not assert independent reproduction.

### Gemma-3-Gaia-PT-BR-4b-it

Average Score: 49.93. Result ID: 69a9329d6dbb7075b95242a1.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"4B"}. This record does not assert independent reproduction.

### SmolLM3-3B

Average Score: 49.54. Result ID: 69a9329d6dbb7075b95242a2.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"3B"}. This record does not assert independent reproduction.

### Llama-3.2-3B-Instruct

Average Score: 45.82. Result ID: 69a9329d6dbb7075b95242a3.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"3B"}. This record does not assert independent reproduction.

### Qwen2.5-1.5B-Instruct

Average Score: 41.39. Result ID: 69a9329d6dbb7075b95242a4.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa767e1e209131cea6d6a7.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"1.5B"}. This record does not assert independent reproduction.

### Tucano2-qwen-1.5B-Instruct

Average Score: 37.54. Result ID: 69a9329d6dbb7075b95242a5.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"1.5B"}. This record does not assert independent reproduction.

### Qwen3-1.7B

Average Score: 36.3. Result ID: 69a9329d6dbb7075b95242a6.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"1.7B"}. This record does not assert independent reproduction.

### Tucano2-qwen-0.5B-Instruct

Average Score: 26.08. Result ID: 69a9329d6dbb7075b95242a7.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"0.5B"}. This record does not assert independent reproduction.

### Qwen3-0.6B

Average Score: 22.21. Result ID: 69a9329d6dbb7075b95242a8.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa788e1e209131cea79116.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"0.6B"}. This record does not assert independent reproduction.

### Llama-3.2-1B-Instruct

Average Score: 20.14. Result ID: 69a9329d6dbb7075b95242a9.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"1B"}. This record does not assert independent reproduction.

### Qwen2.5-0.5B-Instruct

Average Score: 17.8. Result ID: 69a9329d6dbb7075b95242aa.

[Reporting paper](<https://www.sota2.com/research/paper/69a90de92d603e6de3e40680.md>)

[Method origin paper](<https://www.sota2.com/research/paper/68fa767e1e209131cea6d6a7.md>)

Recorded evaluation configuration: {"Variant":"Instruct","Parameters":"0.5B"}. This record does not assert independent reproduction.

Page 1 of 1. This is an index page, not the full collection.
