When Privileged Guidance Misaligns: State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents

Privileged on-policy distillation gives dense supervision by having a teacher with access to training-only references re-score a student's actions each turn. However, in interactive environments, the student's changing execution state can cause the teacher's guidance to become misaligned with the actual rollout. The authors propose state-matched routing to select appropriate teacher states and contextualized self-distillation to refine the student's policy using its own context. This approach helps mitigate distribution shift and improves sample efficiency in multi-turn tasks like dialogue and embodied agents. The paper is from arXiv (2608.05219) and is relevant to researchers working on RL, imitation learning, and agent training.