1. Sample spaces & events

2. Axioms of probability

3. Conditional probability

4. Independence

5. Joint, marginal, conditional distributions

6. Bayes' theorem

7. Random variables (discrete & continuous)

8. PMF, PDF, CDF

9. Expected value & variance

10. Common distributions

11. Law of Large Numbers

12. Central Limit Theorem

13. Entropy (Shannon)

14. Cross-entropy

15. KL Divergence

16. Mutual Information

17. Likelihood & log-likelihood