Course overview
Introduction to model serving with vLLM: serving architecture patterns, OpenAI-compatible endpoints, batching and KV-cache optimizations, plus routing and autoscaling strategies for production stacks.
Compares serving architecture patterns: centralized inference, microservices and edge serving according to latency and cost needs.
Introduces vLLM basics and establishing OpenAI-compatible endpoints for seamless client integration.
Describes techniques to boost throughput and reduce latency: batching, KV cache and runtime optimizations.
Covers designing the production stack: request routing, autoscaling policies and integration considerations.
4lessons
4lessons
20activities
0exercises
3videos
Skills and topics covered
Access the platform
Open the catalogue to review course details, access conditions, and your personal progress. Sign in.