Towards Theoretical Understanding of Deep Learning

Organizers: Amit Daniely (amit.daniely@mail.huji.ac.il), Rong Ge (rongge@cs.duke.edu), Tengyu Ma (tengyuma@cs.stanford.edu), Ohad Shamir (ohad.shamir@weizmann.ac.il)

Description

Deep learning has resulted in breakthroughs in many areas of artificial intelligence including computer vision, speech, natural language processing, reinforcement learning, robotics etc. However, theoretical understanding of deep learning has been scarce.

Why can simple algorithms (such as stochastic gradient descent) find high-quality solutions even though the objective function is non-convex? Why do the learned neural networks generalize to the test data even though the number of parameters is much more than the number of examples? These fundamental problems about the optimization and generalization  have attracted a lot of recent attention, but we are still far from satisfying answers.

The new models and formulations in deep learning has also introduced new algorithmic challenges. For example, Generative Adversarial Nets (GANs) are very effective in generating images, but its training procedure is still very unstable. Could we design algorithms with convergence guarantees for GANs?Neural network models are often susceptible to adversarial examples. How can we find models that are robust to adversarial perturbations?

Goal

The workshop will serve as an introduction to recent developments in theoretical understanding of deep neural networks, including techniques, results, and research directions. We hope to bring together researchers in theory community, foster research discussions between theory and practice, and eventually lead to interesting results that could impact deep learning practice.

Schedule

Full talk abstracts