1. Bit as a unit of information

2. Entropy (H)

3. Conditional entropy

4. Mutual information

5. Information gain

6. Perplexity (key NLP metric)

7. Compression basics

8. Huffman coding concept