1. Loss functions

2. Gradient descent (batch, stochastic, mini-batch)

3. Learning rate

4. Momentum

5. Adaptive methods (AdaGrad, RMSprop, Adam)

6. Convex vs. non-convex optimization

7. Regularization (L1 / Lasso, L2 / Ridge)

8. Lagrange multipliers

9. Constrained optimization