Enhancing Social Intelligence in LLMs with Hierarchical Reasoning and Utterance-Level Goal Rewarding

The paper addresses the challenge of LLMs in dynamic social interactions, where long-term goal coordination and rapid adaptation are crucial. Existing methods apply uniform goal-based rewards to every utterance, ignoring the specificity of objectives at each dialogue turn and the rationale behind potential strategies. Inspired by the Theory of Planned Behavior, the authors propose TSR, which decomposes social dialogue into high-level strategic planning and low-level linguistic execution. To optimize this, they introduce LHRL-VGR, a novel reinforcement learning algorithm that dynamically routes rewards based on the variance of goal achievement scores, balancing goal completion and strategy adherence. Experiments on the SOTOPIA benchmark show that fine-tuning a Qwen2.5-7B agent with this approach surpasses the GPT-4o baseline by 7.32% in goal completion success, demonstrating state-of-the-art performance in multi-agent social negotiation tasks.