ResearchBenchmarksLLM fine-tuning on CB (test)Follow6,894Number of QueriesMeZO-Adam1,296.722,749.864,2035,656.14Nov 11, 2025Evaluation ResultsMethodMethodLinksNumber of QueriesTime TakenMeZO-AdamBackbone=LLaMA-3-8BBackbone=LLaMA-3-8B2025.116,8940.96MeZOBackbone=LLaMA-3-8BBackbone=LLaMA-3-8B2025.115,7360.8HiZOOBackbone=LLaMA-3-8BBackbone=LLaMA-3-8B2025.114,3560.76MeZO-SVRGBackbone=LLaMA-3-8BBackbone=LLaMA-3-8B2025.113,2160.53LOZOBackbone=LLaMA-3-8BBackbone=LLaMA-3-8B2025.113,1980.15LORENBackbone=LLaMA-3-8BBackbone=LLaMA-3-8B2025.111,5120.47