Long Context Evaluation on Long Context Benchmarks
32.3MDQA-10 ScoreDD
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DDStep Time (ms)=244, Data Prep. Cost=$, Curriculum=Grow-P2 with 8 cycles2024.05 | 32.3 | 26.9 | 24.6 | 30.7 | 34.2 | 30.9 | |
| Baseline-8kStep Time (ms)=304, Data Prep. Cost=$, Document Masking=false2024.05 | 29 | 23.8 | 20.5 | 26.2 | 32 | 27.5 | |
| ICLMStep Time (ms)=304, Data Prep. Cost=$$$2024.05 | 26.7 | 20 | 22 | 28.7 | 34.6 | 28.7 | |
| Pack-8k+DMStep Time (ms)=304, Data Prep. Cost=$$, Document Masking=true2024.05 | 25.6 | 19.8 | 16.9 | 29.2 | 33.1 | 27.7 | |
| Baseline-8k+DMStep Time (ms)=304, Data Prep. Cost=$, Document Masking=true2024.05 | 24.4 | 20 | 16 | 29.2 | 32 | 27.1 |