Datasets and Evaluation

Course overview

Design and evaluation of datasets for fine-tuning: instruction-tuning formatting, train/validation/test split strategies, baseline evaluation before fine-tuning, and checks for bias, leakage and data quality.

Methods to design and format datasets for instruction tuning, including expected data structures.

Train/validation/test split strategies and their influence on evaluation and generalization.

Baseline evaluation to set performance references before applying fine-tuning.

Quality controls: detecting bias, preventing data leakage and best practices to ensure dataset robustness.