Sparse Mutual Information Graph Averaging for Improving Random Indexing Embeddings

Random Indexing (RI) is a sparse embedding technique that builds word vectors from corpus statistics without dense co-occurrence matrices or gradient-based training. The authors refine RI vectors by computing a sparse PPMI graph and applying weighted averaging over neighboring words. On a fairytales corpus, the covered semantic analogy set includes 272 Google family-category questions, suggesting the method preserves relational semantics. The work highlights that sparse global statistics can be as effective as dense factorization for certain tasks. Future work might explore scaling the approach to larger corpora or integrating it with other embedding refinements.