Emergence of compositional representations in restricted Boltzmann machines

schedule le mardi 12 décembre 2017 de 16h30 à 18h00

Organisé par : Bastien Fernandez, Nicolas Fournier, Sandrine Péché

Intervenant : R. Monasson (LPTENS)
Lieu : Salle 0011, Sophie Germain (Université Paris Diderot)

Sujet : Emergence of compositional representations in restricted Boltzmann machines

Résumé :
Machine-learning algorithms nowadays show spectacular performances, but the reasons for these successes remain poorly understood. We investigate, with tools and concepts from statistical physics as well as numerical
study on real data, the working principles of Restricted Boltzmann Machines (RBM), capable of learning data features without any prior knowledge. We show that RBM may operate in a compositional phase, in which data are modelled as distributed and graded compositions of many of those features, and the dynamics of sampling is very efficient. The structural conditions on RBM, such as  the sparsity and strength of interactions, the nonlinearities of processing units, … underlying this novel compositional phase are elucidated.
Reference:  J. Tubiana, R. Monasson, Physical Review Letters 118, 138301

(2017)