Title: A margin-theoretic perspective on feature selection in deep networks
Abstract:This talk will first motivate and illustrate the use of margins as a way to interpret and analyze the behavior of deep network training, and then present a variety of results where this behavior can be analyzed,
leading to learning guarantees with few samples, in particular in
regimes where features move far from initialization. In further detail,
the first part will illustrate a variety of behaviors of deep network
training, and show how these correspond to margin maximization; notably,
these behaviors can be counter-intuitive, and contradict common beliefs
regarding simplicity bias in deep networks. The second part will then
present a variety of analyses where margins capture the motion of
weights far from initialization; these analyses will be able to give the
best sample and computational complexities for a variety of problems,
and moreover the only general guarantees which are not forced to be near
initialization.
Bio: Matus Telgarsky is an assistant professor at the University of Illinois,
Urbana-Champaign, specializing in deep learning theory. He was
fortunate to receive a PhD at UCSD under Sanjoy Dasgupta. Other
highlights include: co-founding, in 2017, the Midwest ML Symposium
(MMLS) with Po-Ling Loh; receiving a 2018 NSF CAREER award; and
organizing a 2019 Simons Institute programs.