Optimization for Data Sciences

École Polytechnique, Master 2 · 2025–2026

Schedule

Sept 10: Introduction (S. Vaiter)

• Rémi Flamary’s slides
• Notes on convexity
• Homework: review on your undergrad notes the notion of gradient, Hessian, eigenvalues/eigenvectors and singular value decomposition

Sept 17: Gradient descent (S. Vaiter)

• Rémi Flamary’s slides
• Notes on GD
• Practical .ipynb (NOT RATED!)
• Homework 1: Finish the practical
• Homework 2: Do the exercice sheet

Sept 24: Quadratic problems and linesearch methods (A. Gramfort)

Oct 1: Proximal methods (S. Vaiter)

• Rémi Flamary’s slides

Oct 8: Proximal methods (S. Vaiter + A. Van Elst)

• Practical .ipynb (to submit on Moodle, email ignored!)

Oct 15: Stochastic Gradient methods (S. Vaiter)

• Rémi Flamary’s slides
• Notes on SGD

Nov 05: Stochastic Gradient methods (S. Vaiter + A. Van Elst)

• Practical .ipynb (to submit on Moodle, email ignored!)

References

• Convex Optimization by Boyd and Vandenberghe.
• Lecture Notes on Convex Optimization by Nesterov.
• Numerical Optimization by Nocedal and Wright.
• Optimization for Modern Data Analysis by Recht and Wright.

Other years: 2026–2027· all years