Beyond Surface Style: Aligning Multi-Turn User Simulators with Behavioral Consistency

Faithful user simulation is important for building, evaluating, and improving interactive AI at scale, but plausible individual responses do not guarantee that simulated users reproduce the intent evolution and outcomes seen in real interactions.

To address this, the authors propose TRACER, a multi-turn user simulator that explicitly models users' evolving intent and learns to align simulated behavior with real interaction trajectories. TRACER is trained in two stages: supervised fine-tuning on real user dialogues, followed by multi-turn reinforcement learning. The RL stage combines hierarchical outcome- and trajectory-level rewards with deviation-aware advantage modulation, jointly mitigating reward sparsity and credit assignment in long dialogues.

On real customer-service sessions organized into reference cohorts, TRACER-7B surpasses the strongest baseline by 11.4 conversion F1. It also achieves the lowest group-level conversion-rate error and semantic trajectory distance, and generalizes to out-of-distribution scenarios. Human Turing tests yield identification accuracy close to chance, supporting the perceived naturalness of generated conversations.

Building on this simulator, the authors introduce the Dynamic Marketing Benchmark, which jointly evaluates persuasion effectiveness and response quality of LLMs through simulated interactions. It reveals that higher response quality does not necessarily correspond to higher conversion rates.