CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits

Existing LLM steering methods such as Contrastive Activation Addition (CAA) rely on fixed single-layer interventions derived from aggregate activation differences. These approaches impose a single intervention across semantically diverse inputs and often fail to sustain consistent behavioral changes across layers, limiting their effectiveness.

CircuitSteer addresses this by leveraging Sparse Autoencoders (SAEs) to identify coherent semantic circuits distributed across multiple layers. The method constructs a feature flow circuit based on feature co-activation and geometric alignment of decoder directions, isolating multi-layer subcircuits responsible for a target behavior. It then synthesizes dense steering vectors from these sparse features and applies multi-point interventions to guide the model's internal semantic trajectory.

Evaluated on contrastive examples across toxicity, emotion-intensity, sycophancy, and refusal tasks spanning two model families, CircuitSteer was the only method to consistently produce fluency-preserving interventions. Competing methods either sacrificed text quality or lacked coverage, failing entirely on complex behaviors like sycophancy and refusal. These results demonstrate that multi-layer circuit steering with enforced geometric alignment yields strictly more robust and effective behavioral control than static single-point interventions. Code is available at the project repository.