Training Graph Foundation Models on The Web Graph

Existing graph foundation models often require training additional classification heads or feature projectors to accommodate new graphs or new labels, and many gain their capabilities by being stitched together with pretrained LLMs. This work presents Acacia as an alternative approach.

Acacia is a graph foundation model trained from scratch using only the Common Crawl web graph. It is designed to support arbitrary feature dimensionalities and semantics without additional training, and it does not rely on pretrained LLMs.

The model supports a wide range of tasks without additional training, including node classification, link prediction, node clustering, and graph generation. It also has in-context learning capabilities.

The authors frame this as an important result because it provides evidence that graph models can acquire emergent capabilities from scratch like LLMs, rather than depending on LLM-based components or task-specific adaptation.