LiFT: How to Enable In-Context Learning for Longitudinal Modelling
About
Longitudinal NLP tasks such as mental health monitoring and stance evolution require modeling temporally ordered text to track persistence and detect change. Such tasks also suffer from data scarcity, often involving rare events and sparsely annotated data. Large Language Models (LLMs) can learn from small amounts of data through in-context learning (ICL), which is particularly important in low-data resource scenarios. However, when it comes to tracking evolving interactions or identifying rare events or changes, LLMs fall short in both zero- and few-shot ICL. To address this limitation, we introduce LiFT, a model-agnostic longitudinal Instruction Fine-Tuning framework that is able to induce ICL behaviour in LLMs for longitudinal tasks. LiFT unifies diverse tasks through a shared instruction schema that leverages sequential dependencies, curriculum learning, and temporal conditioning. We evaluate LiFT on five longitudinal datasets under zero-, one-, and three-shot settings, demonstrating that LiFT delivers superior performance to IFT models under identical ICL conditions. Furthermore, we demonstrate that LiFT yields larger gains for minority classes, meaningfully attends to recent demonstration history, and develops transferable longitudinal modeling capabilities beyond its training tasks