18/02/2025
🥳🥳🥳 We are pleased to announce the publication of our paper, “Density of States in Neural Networks: An In-Depth Exploration of Learning in Parameter Space” in Transactions on Machine Learning Research.
In this work, we apply advanced sampling techniques from statistical physics—specifically, the Wang-Landau algorithm—to map the entire loss landscape of neural networks. Our findings provide a detailed characterization of the density of states across network configurations, shedding light on the interplay between data structure, network architecture, and class imbalance.
This research is the result of a collaborative effort among the teams at the University of Trento, INFN-TIFPA, and the Donders Institute, and we are grateful for the support of our colleagues and institutions.
To read the full paper, please visit our OpenReview page: https://openreview.net/forum?id=BLDtWlFKhn
Learning in neural networks critically hinges on the intricate geometry of the loss landscape associated with a given task. Traditionally, most research has focused on finding specific weight...