# Reading Comprehension on BELEBELE (Korean, Bengali, Swahili, Telugu) Source: [Reading Comprehension on BELEBELE (Korean, Bengali, Swahili, Telugu)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:35:34.560Z Evaluation of Reading Comprehension performance across multiple languages (Korean, Bengali, Swahili, Telugu) within the BELEBELE benchmark suite, reporting language-specific and average scores. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Reading Comprehension | | Dataset (including any stated split) | BELEBELE | | Metric | Average RC Score (BELEBELE) | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 31 of 31 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [Average RC Score (BELEBELE)](): Higher is better; unit: PERCENT. - [Korean RC Score](): Higher is better; unit: PERCENT. - [Bengali RC Score](): Higher is better; unit: PERCENT. - [Swahili RC Score](): Higher is better; unit: PERCENT. - [Telugu RC Score](): Higher is better; unit: PERCENT. ## Recorded results | Method | Average RC Score (BELEBELE) | Result date | Report paper | | --- | --- | --- | --- | | Trinity Large (MoE) | 80 | Not recorded | 6995476c09c33900fcd6d4aa | | Qwen3 8B | 77 | Not recorded | 6995476c09c33900fcd6d4aa | | Qwen3 4B | 71 | Not recorded | 6995476c09c33900fcd6d4aa | | Module-wise Swapping | 70.6 | Not recorded | 69802f65a8b01e76cc8e14bf | | Layer-wise Swapping | 70.5 | Not recorded | 69802f65a8b01e76cc8e14bf | | Task Arithmetic | 70.4 | Not recorded | 69802f65a8b01e76cc8e14bf | | Ties Merging | 70.4 | Not recorded | 69802f65a8b01e76cc8e14bf | | DARE | 70.4 | Not recorded | 69802f65a8b01e76cc8e14bf | | Safety-Only | 70.3 | Not recorded | 69802f65a8b01e76cc8e14bf | | base | 70.2 | Not recorded | 69802f65a8b01e76cc8e14bf | | Lang-Only | 70.1 | Not recorded | 69802f65a8b01e76cc8e14bf | | Layer Swapping | 70.1 | Not recorded | 69802f65a8b01e76cc8e14bf | | Mixed SFT | 70 | Not recorded | 69802f65a8b01e76cc8e14bf | | Lang-Only | 56.3 | Not recorded | 69802f65a8b01e76cc8e14bf | | Layer Swapping | 56.3 | Not recorded | 69802f65a8b01e76cc8e14bf | | Layer-wise Swapping | 56.2 | Not recorded | 69802f65a8b01e76cc8e14bf | | DARE | 56.1 | Not recorded | 69802f65a8b01e76cc8e14bf | | Module-wise Swapping | 55.8 | Not recorded | 69802f65a8b01e76cc8e14bf | | Task Arithmetic | 55.7 | Not recorded | 69802f65a8b01e76cc8e14bf | | Ties Merging | 55.6 | Not recorded | 69802f65a8b01e76cc8e14bf | | Mixed SFT | 55.5 | Not recorded | 69802f65a8b01e76cc8e14bf | | Sarvam1 2B | 55 | Not recorded | 6995476c09c33900fcd6d4aa | | Safety-Only | 54.2 | Not recorded | 69802f65a8b01e76cc8e14bf | | Llama-3.1 8B | 53 | Not recorded | 6995476c09c33900fcd6d4aa | | Datology 8B | 51 | Not recorded | 6995476c09c33900fcd6d4aa | | Granite-4.0 Micro | 51 | Not recorded | 6995476c09c33900fcd6d4aa | | base | 50.1 | Not recorded | 69802f65a8b01e76cc8e14bf | | Datology 3B | 43 | Not recorded | 6995476c09c33900fcd6d4aa | | Llama-3.2 3B | 39 | Not recorded | 6995476c09c33900fcd6d4aa | | SmolLM3 3B | 30 | Not recorded | 6995476c09c33900fcd6d4aa | | Llama-3.2 1B | 25 | Not recorded | 6995476c09c33900fcd6d4aa | ### Trinity Large (MoE) Average RC Score (BELEBELE): 80. Result ID: 699ab0c55698665ec43e3405. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MoE","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Qwen3 8B Average RC Score (BELEBELE): 77. Result ID: 699ab0c55698665ec43e3402. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Qwen3 4B Average RC Score (BELEBELE): 71. Result ID: 699ab0c55698665ec43e3401. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"4B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Module-wise Swapping Average RC Score (BELEBELE): 70.6. Result ID: 69965d4c5698665ec4a30531. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Layer-wise Swapping Average RC Score (BELEBELE): 70.5. Result ID: 69965d4c5698665ec4a30530. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Task Arithmetic Average RC Score (BELEBELE): 70.4. Result ID: 69965d4c5698665ec4a3052c. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Ties Merging Average RC Score (BELEBELE): 70.4. Result ID: 69965d4c5698665ec4a3052d. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### DARE Average RC Score (BELEBELE): 70.4. Result ID: 69965d4c5698665ec4a3052e. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Safety-Only Average RC Score (BELEBELE): 70.3. Result ID: 69965d4c5698665ec4a3052a. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### base Average RC Score (BELEBELE): 70.2. Result ID: 69965d4c5698665ec4a30528. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Lang-Only Average RC Score (BELEBELE): 70.1. Result ID: 69965d4c5698665ec4a30529. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Layer Swapping Average RC Score (BELEBELE): 70.1. Result ID: 69965d4c5698665ec4a3052f. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Mixed SFT Average RC Score (BELEBELE): 70. Result ID: 69965d4c5698665ec4a3052b. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"Qwen 2 7B","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Lang-Only Average RC Score (BELEBELE): 56.3. Result ID: 69965d4c5698665ec4a3051f. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Layer Swapping Average RC Score (BELEBELE): 56.3. Result ID: 69965d4c5698665ec4a30525. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Layer-wise Swapping Average RC Score (BELEBELE): 56.2. Result ID: 69965d4c5698665ec4a30526. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### DARE Average RC Score (BELEBELE): 56.1. Result ID: 69965d4c5698665ec4a30524. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Module-wise Swapping Average RC Score (BELEBELE): 55.8. Result ID: 69965d4c5698665ec4a30527. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Task Arithmetic Average RC Score (BELEBELE): 55.7. Result ID: 69965d4c5698665ec4a30522. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Ties Merging Average RC Score (BELEBELE): 55.6. Result ID: 69965d4c5698665ec4a30523. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Mixed SFT Average RC Score (BELEBELE): 55.5. Result ID: 69965d4c5698665ec4a30521. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Sarvam1 2B Average RC Score (BELEBELE): 55. Result ID: 699ab0c55698665ec43e3406. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"2B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Safety-Only Average RC Score (BELEBELE): 54.2. Result ID: 69965d4c5698665ec4a30520. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Llama-3.1 8B Average RC Score (BELEBELE): 53. Result ID: 699ab0c55698665ec43e3400. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"8B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Datology 8B Average RC Score (BELEBELE): 51. Result ID: 699ab0c55698665ec43e33fd. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Granite-4.0 Micro Average RC Score (BELEBELE): 51. Result ID: 699ab0c55698665ec43e3404. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"Micro","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### base Average RC Score (BELEBELE): 50.1. Result ID: 69965d4c5698665ec4a3051e. [Reporting paper]() Recorded evaluation configuration: {"Backbone":"LLaMA 3.1 8B it","Evaluation Protocol":"0-shot"}. This record does not assert independent reproduction. ### Datology 3B Average RC Score (BELEBELE): 43. Result ID: 699ab0c55698665ec43e33fc. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"3B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Llama-3.2 3B Average RC Score (BELEBELE): 39. Result ID: 699ab0c55698665ec43e33ff. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"3B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### SmolLM3 3B Average RC Score (BELEBELE): 30. Result ID: 699ab0c55698665ec43e3403. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"3B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Llama-3.2 1B Average RC Score (BELEBELE): 25. Result ID: 699ab0c55698665ec43e33fe. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"1B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.