Large Language Model Foundations

Training programme overview

Exploration of large language model fundamentals: Transformer architecture, tokenization, dataset preparation and training workflows. A technical approach focused on understanding components and training stages.

Transformer architecture: attention mechanisms, feed-forward layers and optimization principles.

Tokenizers and sequence representation: splitting methods, encoding and handling context-length limits.

Datasets: collection, cleaning, sampling and preparation for supervised or staged training.

Training workflows: fine-tuning pipelines, evaluation, hyperparameter management and training monitoring.

Training · Intermediate