# Self-Improving In-Context Learning Source: [Self-Improving In-Context Learning]() Collection index: [llms.txt]() Data updated: 2026-05-26T05:00:05.667Z Authors: Baturay Saglam, Dionysis Kalogerias Published: 2026-05-22T03:01:34.000Z ## Abstract We propose to improve in-context learning (ICL) by optimizing the continuous embeddings of a fixed few-shot prompt at test time. The key observation is that the log-probabilities a model assigns to its demonstrated outputs$\\unicode{x2013}$available from a single forward pass without generating any tokens$\\unicode{x2013}$provide a meaningful signal for how well the model has inferred the task from its demonstrations. We formalize this signal as a bounded, self-supervised confidence proxy and maximize it via zeroth-order optimization over the prompt embeddings, yielding a test-time calibration procedure. The approach requires no finetuning, no token generation, no predefined label set, and no external data, making it equally applicable to both classification and free-form generation tasks. Across a comprehensive suite of ICL tasks, the proposed calibration consistently matches or improves upon the base model and outperforms classification-specific baselines on most tasks. The statistically significant correlation between proxy improvement and downstream accuracy gain confirms that the proposed proxy encodes a reliable optimization signal for in-context learning. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Format Check on ICLEval classification](): Format Check; ICLEval classification; Accuracy: 36. - [Order Check on ICLEval classification](): Order Check; ICLEval classification; Accuracy: 100. ## Original sources and implementation [Original paper]() [PDF]()