1. Transfer learning & fine-tuning

2. Pre-training objectives (MLM, CLM, NSP)

3. Learning rate scheduling

4. Mixed precision training

5. Gradient clipping

6. Early stopping

7. Model checkpointing

8. Distributed training concepts