---
title: Optimization for Data Sciences
author: Samuel Vaiter
date: 2025-09-10
status: finished
---

## Schedule

Sept 10: Introduction (S. Vaiter)

- [Rémi Flamary's slides](https://remi.flamary.com/cours/optim_ds/01_intro_optim_ml.pdf)
- [Notes on convexity](/a-first-look-at-convex-analysis/2025-2026/)
- *Homework*: review on your undergrad notes the notion of gradient, Hessian, eigenvalues/eigenvectors and singular value decomposition

Sept 17: Gradient descent (S. Vaiter)

- [Rémi Flamary's slides](https://remi.flamary.com/cours/optim_ds/02_grad_desc.pdf)
- [Notes on GD](/following-the-steepest-direction/2025-2026/)
- [Practical .ipynb](https://files.samuelvaiter.com/teaching/odsx/2025-2026/lab0.ipynb) (NOT RATED!)
- *Homework 1*: Finish the practical
- *Homework 2*: Do the [exercice sheet](https://files.samuelvaiter.com/teaching/odsx/2025-2026/ex_intro_gd.pdf)

Sept 24: Quadratic problems and linesearch methods (A. Gramfort)

Oct 1: Proximal methods (S. Vaiter)

- [Rémi Flamary's slides](https://remi.flamary.com/cours/optim_ds/04_nonsmooth.pdf)

Oct 8: Proximal methods (S. Vaiter + A. Van Elst)

- [Practical .ipynb](https://files.samuelvaiter.com/teaching/odsx/2025-2026/lab1.ipynb) (to submit on Moodle, email ignored!)

Oct 15: Stochastic Gradient methods (S. Vaiter)

- [Rémi Flamary's slides](https://remi.flamary.com/cours/optim_ds/05_stochastic.pdf)
- [Notes on SGD](/adding-some-noise/)

Nov 05: Stochastic Gradient methods (S. Vaiter + A. Van Elst)

- [Practical .ipynb](https://files.samuelvaiter.com/teaching/odsx/2025-2026/lab2.ipynb) (to submit on Moodle, email ignored!)

## References

- [Convex Optimization by Boyd and Vandenberghe](https://web.stanford.edu/~boyd/cvxbook/).
- [Lecture Notes on Convex Optimization by Nesterov](https://shuyuej.com/books/Lectures%20on%20Convex%20Optimization.pdf).
- [Numerical Optimization by Nocedal and Wright](https://www.ime.unicamp.br/~pulino/MT404/TextosOnline/NocedalJ.pdf).
- [Optimization for Modern Data Analysis by Recht and Wright](https://people.eecs.berkeley.edu/~brecht/opt4ml_book/).