How do neural networks work? It’s a question that can confuse novices and experts alike. A team from MIT’s Computer Science and Artificial Intelligence Lab (CSAIL) says that understanding these representations, as well as how they inform the ways that neural networks learn from data, is crucial for improving the interpretability, efficiency, and generalizability of deep learning models.

With that mind, the CSAIL researchers have developed a new framework for understanding how representations form in neural networks. Their Canonical Representation Hypothesis (CRH) posits that, during training, neural networks inherently align their latent representations, weights, and neuron gradients within each layer. This alignment implies that neural networks naturally learn compact representations based on the degree and modes of deviation from the CRH.

Senior author Tomaso Poggio says that, by understanding and leveraging this alignment, engineers can potentially design networks that are more efficient and easier to understand. The research is posted to the arXiv preprint server.

The team’s corresponding Polynomial Alignment Hypothesis (PAH) posits that, when the CRH is broken, distinct phases emerge in which the representations, gradients, and weights become polynomial functions of each other. Poggio says that the CRH and PAH offer a potential unifying theory for key deep learning phenomena such as neural collapse and the neural feature ansatz (NFA).

The new CSAIL paper about the project provides experimental results across various settings to support the CRH and PAH on tasks that include image classification and self-supervised learning. The CRH suggests possibilities for manually injecting noise into neuron gradients to engineer specific structures in the model’s representations. Poggio says that a key future direction is to understand the conditions that lead to each phase and how these phases affect the behavior and performance of models.

“The paper offers a new perspective on understanding the formation of representations in neural networks through the CRH and PAH,” says Poggio. “This provides a framework for unifying existing observations and guiding future research in deep learning.”

Co-author Liu Ziyin, a postdoc at CSAIL, says the CRH may explain certain phenomena in neuroscience, as it implies that neural networks tend to learn an orthogonalized representation, which has been observed in recent brain studies. It may also have algorithmic implications: if representations align with the gradients, it might be possible to manually inject noise into neuron gradients to engineer specific structures in the model’s representations.

Ziyin and Poggio co-wrote the paper with professor Isaac Chuang and former postdoc Tomer Galanti, now an assistant professor of computer science at Texas A&M University. They will present it later this month at the International Conference on Learning Representations (ICLR 2025) in Singapore.