# Mathematical Information Retrieval on CLEF ARQMath 2 Source: [Mathematical Information Retrieval on CLEF ARQMath 2]() Collection index: [llms.txt]() Data updated: 2026-02-26T05:53:03.787Z Evaluation of model performance on the CLEF ARQMath 2 dataset for Mathematical Information Retrieval tasks, typically measured using ranking metrics like NDCG', MAP', and BPref. A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. | Scope | Recorded value | | --- | --- | | Task | Mathematical Information Retrieval | | Dataset (including any stated split) | CLEF ARQMath 2 | | Metric | NDCG' | | Unit | PERCENT | | Direction | Higher is better | | Results loaded | 11 of 11 | | Coverage | All results currently returned for this benchmark | Evaluation settings may differ between results. The original papers and recorded configurations are needed to establish comparability. A shared metric name alone does not establish an identical protocol. ## Metrics - [NDCG'](): Higher is better; unit: PERCENT. - [MAP'](): Higher is better; unit: PERCENT. - [P@10](): Higher is better; unit: PERCENT. - [BPref](): Higher is better; unit: PERCENT. - [Judged %](): Higher is better; unit: PERCENT. ## Recorded results | Method | NDCG' | Result date | Report paper | | --- | --- | --- | --- | | ColBERT | 32.9 | Not recorded | 68fa6dbe1e209131cea3bca8 | | DPR | 27 | Not recorded | 68fa6dbe1e209131cea3bca8 | | MIRMU CompuBERT | 26.2 | Not recorded | 68fa6dbe1e209131cea3bca8 | | TU\_DBS ColARQBERT | 22.5 | Not recorded | 68fa6dbe1e209131cea3bca8 | | further-pretrained BERT | 20 | Not recorded | 68fa6dbe1e209131cea3bca8 | | further-pretrained BERT | 19.5 | Not recorded | 68fa6dbe1e209131cea3bca8 | | TU\_DBS ColBERT | 18.3 | Not recorded | 68fa6dbe1e209131cea3bca8 | | SciBERT | 17.6 | Not recorded | 68fa6dbe1e209131cea3bca8 | | BERT | 17.4 | Not recorded | 68fa6dbe1e209131cea3bca8 | | FormulaEmb | 16.1 | Not recorded | 68fa6dbe1e209131cea3bca8 | | TU\_DBS ColSciBERT | 2.8 | Not recorded | 68fa6dbe1e209131cea3bca8 | ### ColBERT NDCG': 32.9. Result ID: 698fe35a3a6c314849fc9519. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training stage":"fully-trained"}. This record does not assert independent reproduction. ### DPR NDCG': 27. Result ID: 698fe35a3a6c314849fc9518. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Training stage":"fully-trained"}. This record does not assert independent reproduction. ### MIRMU CompuBERT NDCG': 26.2. Result ID: 698fe35a3a6c314849fc9516. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### TU\_DBS ColARQBERT NDCG': 22.5. Result ID: 698fe35a3a6c314849fc9515. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### further-pretrained BERT NDCG': 20. Result ID: 698fe35a3a6c314849fc9512. [Reporting paper]() Recorded evaluation configuration: {"Pre-trained backbone":"BERT","Fine-tuning epochs":"1","Further-pretraining epochs":"7"}. This record does not assert independent reproduction. ### further-pretrained BERT NDCG': 19.5. Result ID: 698fe35a3a6c314849fc9511. [Reporting paper]() Recorded evaluation configuration: {"Pre-trained backbone":"BERT","Fine-tuning epochs":"1","Further-pretraining epochs":"3"}. This record does not assert independent reproduction. ### TU\_DBS ColBERT NDCG': 18.3. Result ID: 698fe35a3a6c314849fc9514. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### SciBERT NDCG': 17.6. Result ID: 698fe35a3a6c314849fc9510. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-trained backbone":"SciBERT","Fine-tuning epochs":"1"}. This record does not assert independent reproduction. ### BERT NDCG': 17.4. Result ID: 698fe35a3a6c314849fc950f. [Reporting paper]() [Method origin paper]() [Implementation]() Recorded evaluation configuration: {"Pre-trained backbone":"BERT","Fine-tuning epochs":"1"}. This record does not assert independent reproduction. ### FormulaEmb NDCG': 16.1. Result ID: 698fe35a3a6c314849fc9517. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. ### TU\_DBS ColSciBERT NDCG': 2.8. Result ID: 698fe35a3a6c314849fc9513. [Reporting paper]() Recorded evaluation configuration: Not recorded. This record does not assert independent reproduction. Page 1 of 1. This is an index page, not the full collection.