# LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet? Source: [LLMs for Bayesian Optimization in Scientific Domains: Are We There Yet?]() Collection index: [llms.txt]() Data updated: 2025-10-26T05:43:27.824Z Authors: Rushil Gupta, Jason Hartford, Bang Liu Published: 2025-09-24T15:50:17.000Z ## Abstract Large language models (LLMs) have recently been proposed as general-purpose agents for experimental design, with claims that they can perform in-context experimental design. We evaluate this hypothesis using both open- and closed-source instruction-tuned LLMs applied to genetic perturbation and molecular property discovery tasks. We find that LLM-based agents show no sensitivity to experimental feedback: replacing true outcomes with randomly permuted labels has no impact on performance. Across benchmarks, classical methods such as linear bandits and Gaussian process optimization consistently outperform LLM agents. We further propose a simple hybrid method, LLM-guided Nearest Neighbour (LLMNN) sampling, that combines LLM prior knowledge with nearest-neighbor sampling to guide the design of experiments. LLMNN achieves competitive or superior performance across domains without requiring significant in-context adaptation. These results suggest that current open- and closed-source LLMs do not perform in-context experimental design in practice and highlight the need for hybrid frameworks that decouple prior-based reasoning from batch acquisition with updated posteriors. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Cumulative target feature discovery on JUMP Cell Painting](): Cumulative target feature discovery; JUMP Cell Painting; F0 Score: 28.6. ## Original sources and implementation [Original paper]() [PDF]()