My name is Stefano Spigler. I am a postdoc in the Laboratory of Physics of Complex Systems at the École Polytechnique Fédérale de Lausanne (EPFL, Lausanne, Switzerland).
I am funded by the Simons Collaboration on Cracking the glass problem.
My current research focuses on the study of the learning dynamics and generalization performance of deep neural networks. As a statistical physicist, I study the behavior of deep networks in the limit of (i) very wide (overparametrized) networks, and (ii) in the limit of large training sets (mostly on kernels).
Education & academic positions
Download the full CV (pdf)
2017-2020
Postdoc
Eneergy landscape and learning dynamics in deep learning
In collaboration with Prof. Matthieu Wyart
Physics of Complex Systems Laboratory
École Polytechnique Fédérale de Lausanne
Eneergy landscape and learning dynamics in deep learning
In collaboration with Prof. Matthieu Wyart
École Polytechnique Fédérale de Lausanne
2014-2017
PhD in Physics
Distribution of avalanches in disordered systems
Supervisor:Silvio Franz
Laboratoire de Physique Théorique et Modèles Statistiques
Université Paris Sud (Université Paris-Saclay)
Scholarship by the École Normale Supérieure
You can read here my Ph.D. thesis
Distribution of avalanches in disordered systems
Supervisor:
Université Paris Sud (Université Paris-Saclay)
Scholarship by the École Normale Supérieure
2012-2014
Master in Physics of Complex Systems (link)
Politecnico di Torino ,
International School for Advanced Studies ,
International Centre for Theoretical Physics ,
Université Pierre et Marie Curie ,
Université Paris Diderot ,
Université Paris Sud ,
École Normale Supérieure de Cachan
Ranked 1st among all the participants
You can read here my M.Sc. thesis done at the Laboratoire de Physique Théorique et Modèles Statistiques under the supervision of Silvio Franz
Ranked 1st among all the participants
2009-2012
Scuola Galileiana di Studi Superiori (link) (scholarship)
Skills
Informatics
Debian/Red Hat based linux distributions (15 years)
Nowadays I mostly use Python, especially to program neural networs with Pytorch (4 years)
Data anlysis with Python libraries Pandas, NumPy, SciPy (4 years)
Data visualization with the business intelligence application Apache Superset (1 year)
I have designed several personal websites using (X)HTML, XML, CSS, PHP; basic knowledge of JavaScript (AJAX) and SQL (15 years)
Developed complex applications in C++ (10 years)
I know how to redact documents in LaTeX; I know how to use the Office suite (Word, Excel) (10 years)
Languages
Italian (native)
English (full command)
French (fluent)
German (beginner)
Swedish (beginner)
Publications
2020
M. Geiger, S. Spigler, A. Jacot, M. Wyart
Disentangling feature and lazy training in deep neural networks
submitted to conference (arXiv preprint)
Disentangling feature and lazy training in deep neural networks
submitted to conference (arXiv preprint)
2019
S. Spigler, M. Geiger, M. Wyart
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm
submitted to conference (arXiv preprint)
Asymptotic learning curves of kernel methods: empirical data v.s. Teacher-Student paradigm
submitted to conference (arXiv preprint)
2019
M. Geiger, A. Jacot, S. Spigler, F. Gabriel, L. Sagun, S. d'Ascoli, G. Biroli, C. Hongler, M. Wyart
Scaling description of generalization with number of parameters in deep learning
to be submitted (arXiv preprint)
Scaling description of generalization with number of parameters in deep learning
to be submitted (arXiv preprint)
2018
S. Spigler, M. Geiger, S. d'Ascoli, L. Sagun, M. Baity-Jesi, G. Biroli, M. Wyart
A jamming transition from under- to over-parametrization affects loss landscape and generalization
NeurIPS 2018 workshop "Integration of Deep Learning Theories" (arXiv preprint)
Journal of Physics A: Mathematical and Theoretical 52 (47), 474001 (JP A)
A jamming transition from under- to over-parametrization affects loss landscape and generalization
NeurIPS 2018 workshop "Integration of Deep Learning Theories" (arXiv preprint)
Journal of Physics A: Mathematical and Theoretical 52 (47), 474001 (JP A)
2018
M. Geiger, S. Spigler, S. d'Ascoli, L. Sagun, M. Baity-Jesi, G. Biroli, M. Wyart
The jamming transition as a paradigm to understand the loss landscape of deep neural networks
Phys. Rev. E 100(1), 012115 (PR E)
The jamming transition as a paradigm to understand the loss landscape of deep neural networks
Phys. Rev. E 100(1), 012115 (PR E)
2018
M. Baity-Jesi, L. Sagun, M. Geiger, S. Spigler, G.B. Arous, C. Cammarota, Y. LeCun, M. Wyart, G. Biroli
Comparing dynamics: deep neural networks versus glassy systems
ICML, PMLR 80:314-323 (PMLR)
Comparing dynamics: deep neural networks versus glassy systems
ICML, PMLR 80:314-323 (PMLR)
2016
S. Franz, G. Gradenigo, S. Spigler
Random-diluted triangular plaquette model: Study of phase transitions in a kinetically constrained model
Phys. Rev. E 93(3), 032601 (PR E)
Random-diluted triangular plaquette model: Study of phase transitions in a kinetically constrained model
Phys. Rev. E 93(3), 032601 (PR E)
Main talks and presentations
-
Slides: Jamming transition in deep networks. Good performance of overparametrized networks. Double descent in generalization error. Fluctuations and ensemble averaging.
- Slides: Learning curves of kernel methods. Curse of dimensionality? Performance on real data. Teacher-Student kernel regression. Effective dimension and smoothness of real data. RKHS assumption. Dimensionality reduction due to task invariance for kernel regression and classification.