Cautious Context Steering for Language Model Personalization

Personalizing language models (LMs) is important for aligning responses with diverse user goals and backgrounds. Existing approaches either train separate adapters per user or learn user-dependent reward models, both of which suffer from data sparsity and poor generalization. In-context learning (ICL) and Context Steering (CoS) avoid per-user training by conditioning on user context directly, but ICL leaves context influence uncontrolled and CoS applies a fixed steering coefficient while requiring two forward passes per step.

The proposed Cautious Context Steering (CCS) adds a lightweight adapter to a frozen backbone LM. At each decoding step, the adapter decides whether and how strongly user context should affect generation, learning this behavior from an oracle context-conditioned LM. When context is unhelpful, CCS preserves the base LM. Notably, a single CCS adapter trained on one dataset improves generation quality both in-domain and across four out-of-distribution personalization benchmarks, showing robust generalization to new users and domains. Additionally, CCS avoids per-user fine-tuning and eliminates the context-conditioned forward pass required by CoS, substantially reducing inference cost.