Cache-to-Cache: Direct Semantic Communication Between LLMs (2025)

Multi-LLM systems combine the complementary strengths of different large language models to achieve performance and efficiency gains beyond what a single model can attain. Existing designs, however, make LLMs communicate through text, which forces internal representations into output token sequences, losing rich semantic information and incurring token-by-token generation latency.

Motivated by these limitations, the authors ask whether LLMs can communicate beyond text. Oracle experiments show that enriching KV-Cache semantics can improve response quality without increasing cache size, supporting the KV-Cache as an effective medium for inter-model communication. They propose Cache-to-Cache (C2C), a new paradigm for direct semantic communication between LLMs. C2C uses a neural network to project and fuse the source model's KV-cache with that of the target model, enabling direct semantic transfer. A learnable gating mechanism selects the target layers that benefit from cache communication. Compared with text communication, C2C exploits the deep, specialized semantics from both models while avoiding explicit intermediate text generation.

Experiments show that C2C achieves 6.4–14.2% higher average accuracy than individual models. It further outperforms the text-communication paradigm by approximately 3.1–5.4%, while delivering an average 2.5x speedup in latency. Code is available, and the paper was published at ICLR 2026.