---
title: "MOEReview Kaushik: Universal Subspace Hypothesis"
source: https://www.jemoka.com/posts/kbhmoereview_kaushik_universal_subspace_hypothesis/
---

One-Liner There&rsquo;s a low-rank &ldquo;shared&rdquo; universal subspace across many pretrained LMs, which could be thus leveraged to adapt a model to new tasks easier.
Notable Methods Did a PCA, and projected variance from one architecture to others (i.e. LoRAs trained for different things).
