# Question Answering on ARC Bengali (test) Source: [Question Answering on ARC Bengali (test)]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:23:37.828Z Evaluation of model performance on the test split of the Bengali version of the Abstract Reasoning Corpus (ARC), focusing on Question Answering capabilities. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Question Answering | | Dataset (including any stated split) | ARC Bengali (test) | | Metric | ARC Accuracy | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 11 of 11 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [ARC Accuracy](): Higher is better; unit: PERCENT. ## Recorded results | Method | ARC Accuracy | Result date | Report paper | | --- | --- | --- | --- | | Trinity Large (MoE) | 79 | Not recorded | 6995476c09c33900fcd6d4aa | | Qwen3 8B | 76 | Not recorded | 6995476c09c33900fcd6d4aa | | Qwen3 4B | 67 | Not recorded | 6995476c09c33900fcd6d4aa | | Sarvam1 2B | 56 | Not recorded | 6995476c09c33900fcd6d4aa | | Datology 8B | 48 | Not recorded | 6995476c09c33900fcd6d4aa | | Datology 3B | 42 | Not recorded | 6995476c09c33900fcd6d4aa | | Llama-3.1 8B | 42 | Not recorded | 6995476c09c33900fcd6d4aa | | Granite-4.0 Micro | 42 | Not recorded | 6995476c09c33900fcd6d4aa | | Llama-3.2 3B | 33 | Not recorded | 6995476c09c33900fcd6d4aa | | SmolLM3 3B | 28 | Not recorded | 6995476c09c33900fcd6d4aa | | Llama-3.2 1B | 24 | Not recorded | 6995476c09c33900fcd6d4aa | ### Trinity Large (MoE) ARC Accuracy: 79. Result ID: 699ab0c55698665ec43e33fa. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Architecture":"MoE","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Qwen3 8B ARC Accuracy: 76. Result ID: 699ab0c55698665ec43e33f7. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Qwen3 4B ARC Accuracy: 67. Result ID: 699ab0c55698665ec43e33f6. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"4B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Sarvam1 2B ARC Accuracy: 56. Result ID: 699ab0c55698665ec43e33fb. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"2B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Datology 8B ARC Accuracy: 48. Result ID: 699ab0c55698665ec43e33f2. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"8B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Datology 3B ARC Accuracy: 42. Result ID: 699ab0c55698665ec43e33f1. [Reporting paper]() [Method origin paper]() Recorded evaluation configuration: {"Parameters":"3B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Llama-3.1 8B ARC Accuracy: 42. Result ID: 699ab0c55698665ec43e33f5. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"8B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Granite-4.0 Micro ARC Accuracy: 42. Result ID: 699ab0c55698665ec43e33f9. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"Micro","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Llama-3.2 3B ARC Accuracy: 33. Result ID: 699ab0c55698665ec43e33f4. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"3B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### SmolLM3 3B ARC Accuracy: 28. Result ID: 699ab0c55698665ec43e33f8. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"3B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. ### Llama-3.2 1B ARC Accuracy: 24. Result ID: 699ab0c55698665ec43e33f3. [Reporting paper]() Recorded evaluation configuration: {"Parameters":"1B","Protocol":"Zero-shot"}. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.