Optimization for Data Sciences @École Polytechnique

by Samuel Vaiter

Schedule

Sept 10: Introduction (S. Vaiter) - Rémi Flamary's slides - Notes on convexity - Homework: review on your undergrad notes the notion of gradient, Hessian, eigenvalues/eigenvectors and singular value decomposition

Sept 17: Gradient descent (S. Vaiter) - Rémi Flamary's slides - Notes on GD - Practical .ipynb (NOT RATED!) - Homework 1: Finish the practical - Homework 2: Do the exercice sheet

Sept 24: Quadratic problems and linesearch methods (A. Gramfort)

Oct 1: Proximal methods (S. Vaiter) - Rémi Flamary's slides

Oct 8: Proximal methods (S. Vaiter + A. Van Elst) - Practical .ipynb (to submit on Moodle, email ignored!)

Oct 15: Stochastic Gradient methods (S. Vaiter) - Rémi Flamary's slides - Notes on SGD

Nov 05: Stochastic Gradient methods (S. Vaiter + A. Van Elst) - Practical .ipynb (to submit on Moodle, email ignored!)

References

• Convex Optimization by Boyd and Vandenberghe.
• Lecture Notes on Convex Optimization by Nesterov.
• Numerical Optimization by Nocedal and Wright.
• Optimization for Modern Data Analysis by Recht and Wright.