Course Outline
Preparing Machine Learning Models for Production Deployment
- Packaging models using Docker
- Exporting models from TensorFlow and PyTorch
- Considerations for versioning and storage
Serving Models on Kubernetes
- Overview of inference server architectures
- Deploying TensorFlow Serving and TorchServe
- Configuration of model endpoints
Optimizing Inference Performance
- Strategies for batching requests
- Managing concurrent request handling
- Tuning for latency and throughput
Autoscaling Machine Learning Workloads
- Horizontal Pod Autoscaler (HPA)
- Vertical Pod Autoscaler (VPA)
- Kubernetes Event-Driven Autoscaling (KEDA)
GPU Provisioning and Resource Management
- Setting up GPU-enabled nodes
- Overview of the NVIDIA device plugin
- Defining resource requests and limits for ML tasks
Strategies for Model Rollout and Release
- Blue/green deployment techniques
- Canary rollout patterns
- A/B testing for model evaluation
Monitoring and Observability for Production ML
- Key metrics for inference workloads
- Best practices for logging and tracing
- Dashboard creation and alerting setups
Security and Reliability Factors
- Securing access to model endpoints
- Implementing network policies and access controls
- Ensuring high availability standards
Summary and Future Directions
Requirements
- A solid understanding of containerized application workflows
- Hands-on experience with Python-based machine learning models
- Familiarity with core Kubernetes concepts
Target Audience
- ML Engineers
- DevOps Engineers
- Platform Engineering Teams
Testimonials (3)
basic understanding of container/kubernetes and how they interact features of the openshift plattform
Eric Scholze - NOW IT GmbH
Course - Introduction to Containers, Kubernetes & OpenShift
About the microservices and how to maintenance kubernetes
Yufri Isnaini Rochmat Maulana - Bank Indonesia
Course - Advanced Platform Engineering: Scaling with Microservices and Kubernetes
The training met expectations with its clear explanations, real-world examples, and hands-on labs that made complex topics easy to understand. It provided valuable insights into container orchestration, security, scaling and many other advanced topics.