#+title: Optimization for Data Sciences
#+date: 2025-09-10
#+status: finished

* Schedule
Sept 10: Introduction (S. Vaiter)

- [[https://remi.flamary.com/cours/optim_ds/01_intro_optim_ml.pdf][Rémi Flamary's slides]]
- [[lecture:a-first-look-at-convex-analysis][Notes on convexity]]
- /Homework/: review on your undergrad notes the notion of gradient, Hessian, eigenvalues/eigenvectors and singular value decomposition

Sept 17: Gradient descent (S. Vaiter)

- [[https://remi.flamary.com/cours/optim_ds/02_grad_desc.pdf][Rémi Flamary's slides]]
- [[lecture:following-the-steepest-direction][Notes on GD]]
- [[media:teaching/odsx/2025-2026/lab0.ipynb][Practical .ipynb]] (NOT RATED!)
- /Homework 1/: Finish the practical
- /Homework 2/: Do the [[media:teaching/odsx/2025-2026/ex_intro_gd.pdf][exercice sheet]]

Sept 24: Quadratic problems and linesearch methods (A. Gramfort)

Oct 1: Proximal methods (S. Vaiter)

- [[https://remi.flamary.com/cours/optim_ds/04_nonsmooth.pdf][Rémi Flamary's slides]]

Oct 8: Proximal methods (S. Vaiter + A. Van Elst)

- [[media:teaching/odsx/2025-2026/lab1.ipynb][Practical .ipynb]] (to submit on Moodle, email ignored!)

Oct 15: Stochastic Gradient methods (S. Vaiter)

- [[https://remi.flamary.com/cours/optim_ds/05_stochastic.pdf][Rémi Flamary's slides]]
- [[lecture:adding-some-noise][Notes on SGD]]

Nov 05: Stochastic Gradient methods (S. Vaiter + A. Van Elst)

- [[media:teaching/odsx/2025-2026/lab2.ipynb][Practical .ipynb]] (to submit on Moodle, email ignored!)

* References
- [[https://web.stanford.edu/~boyd/cvxbook/][Convex Optimization by Boyd and Vandenberghe]].
- Lecture Notes on Convex Optimization by Nesterov.
- [[https://www.ime.unicamp.br/~pulino/MT404/TextosOnline/NocedalJ.pdf][Numerical Optimization by Nocedal and Wright]].
- Optimization for Modern Data Analysis by Recht and Wright.