ResearchBenchmarksLanguage Model Training Performance on OLMo-1B 32k sequence lengthLanguage Model Training Performance on OLMo-1B 32k sequence lengthFollow8,076Training Time (ms)AGoQ8,044.68,256.558,468.58,680.45May 1, 2026Evaluation resultsMethodMethodLinksTraining Time (ms)Training Memory (MB)AGoQSeq. Len.=32kSeq. Len.=32k2026.058,07686,012COATSeq. Len.=32kSeq. Len.=32k2026.058,86195,664