Bit as a unit of information

Entropy (H)

Conditional entropy

Mutual information

Information gain

Perplexity (key NLP metric)

Compression basics

Huffman coding concept