Download as PDF, PPTX
Uploaded byDr. Anish Cheriyan (PhD)
Introducing MLOps.pdf
The document discusses the concept of MLOps, which integrates machine learning system development and operations to enhance productivity and collaboration in the software industry. It covers the evolution, capabilities, and challenges of MLOps, emphasizing the need for continuous model monitoring and training to adapt to changing business dynamics. Additionally, it highlights the importance of building a comprehensive ML system that focuses on data management, model training, and performance evaluation.
In this document
Powered by AI
Introduction to MLOps presented by Anish and Vimal; event on April 9, 2022, organized by BSPIN and ASQ.
Agenda includes topics like Industry 4.0, Agile, DevOps, MLOps evolution, and case studies.
BSPIN supports Indian Software industry with a focus on quality. Details about membership and contributions.
ASQ aims to advance quality across 130+ countries, fostering connections and knowledge among professionals.
AI applications highlighted include self-driving cars and cancer detection systems.
Reference to a 2018 incident where a self-driving Uber car failed to recognize a jaywalking pedestrian.
Discussion on Industry 4.0 and its components.
Overview of Agile methodologies, Continuous Integration/Continuous Delivery, and DevOps.
Definition of MLOps, its processes in ML system development, operations, and capabilities.
Comparison between MLOps and DevOps practices in managing machine learning workflows.
An outline of how MLOps has evolved over time.
Exploration of capabilities required for effective MLOps.
Identification of challenges like training-serving skews, data quality, and model fairness.
Benefits of MLOps including shorter development cycles, better collaboration, and increased reliability.
How Data Engineering supports ML Engineering through data curation and integration.
Overview of stages involved in the MLOps lifecycle.
Detailed workflow of MLOps implementations.
Technical capabilities crucial for effective ML platforms in MLOps.
Experimental design for data scientists to create models collaboratively.
Data preparation techniques for ML readiness and continuous training.
Efficient execution of algorithms for training ML models.
Processes for model effectiveness assessment during experimentation and in production.
Deployment and serving models in production with near-real-time capabilities.
Methods to compare new models with existing ones prior to deployment.
Tracking the performance of models in production to maintain quality.
Automation and orchestration of complex ML training and prediction processes.
Governance and lifecycle management of ML models through a registry.
Storage solutions for ML data assets to streamline feature engineering.
Role of metadata repositories in reproducibility and debugging of ML tasks.
Focus on automated customer support routing use case.
Detailed approach for automating support ticket classification and routing process.
Visual representation of the complete MLOps workflow from development to monitoring.
Essence of MLOps in creating adaptive ML systems that deliver business value.
Announcement of a book by Anish, Rajith, and Vimal, related to MLOps topics.
List of references used for MLOps knowledge.
Expressing gratitude to participants and attendees.
More Related Content
Introducing MLOps.pdf
- 1. Introducing MLOps Anish Cheriyan Vimal DasK, Inputs from Jayaraj J 9th April, 2022. Event organized jointly by BSPIN and ASQ Bengaluru LMC Image- https://medium.com/analytics-vidhya/applications-and-types-of -machine-learning-c177a844bf38
- 2. Agenda ★ Industry 4.0 ★Agile , CI/CD, DevOps ★ DevOps and MLOps ★ Evolution of MLOPS ★ MLOps Capabilities ★ AI Platform Pipelines ★ Training and Tuning AI Platform ★ Case Study
- 3. BSPIN has beenactive since 1992 with the support of individuals and organizations in Bangalore. BSPIN’s Mission is to help the Indian Software industry to achieve breakthrough in software quality and productivity by active practice enabled by collaborations, learning, sharing and innovating from the practitioners’ level. BSPIN (Bangalore SPIN) is currently the largest operational SPIN across the globe. More details about BSPIN is available on www.bspin.org For MEMBERSHIP- https://bspin.org/?page_id=1480#!/SignUp/Up membership@bspin.org
- 4. ● ASQ isa global community of people passionate about quality, who use the tools, their ideas and expertise to make our world work better. ASQ: The Global Voice of Quality. ● ASQ is a global organization with members in more than 130 countries. Headquartered in Milwaukee, Wisconsin, we also operate centers in Mexico, India, and China. Our Society consists of member-led communities that help members connect with other quality professionals and practitioners, advance their knowledge and careers, and grow as thought leaders. For MEMBERSHIP- https://asq.org.in/membership/
- 5. Self Driving Car AIusage for Cancer Detection https://www.drugtargetreview.com/news/34555/ai-system-detects-canc er-tumours-missed-by-conventional-diagnostics/ Image- https://medium.com/analytics-vidhya/applications-and-types-of -machine-learning-c177a844bf38
- 6. 2018- Self-driving Ubercar that hit and killed woman did not recognize that pedestrians jaywalk https://www.nbcnews.com/tech/tech-news/self-driving-uber-car-hit-killed-woman-did-not-recognize-n1079281
- 7. Industry 4.0 andABCs Image Reference- https://hrishikeshiyengar.wordpress.com/2021/01/31/components-of-industry-4-0-the-heart-and-soul/
- 8.
- 9. What is MLOps? Anapproach, like DevOps, developed in the context of ML engineering Unifies ML System Development and Operations Standardized Processes and Technology Capabilities for building,deploying, & operationalizing ML systems rapidly and reliably
- 10. MLOps & DevOps References-https://en.wikipedia.org/wiki/MLOps
- 11.
- 12.
- 13. Challenges of Practical Applications of ML Avoidingtraining-serving skews that are due to inconsistencies in data, Handling concerns about model fairness and adversarial attacks. Maintaining the veracity of models by continuously retraining Performing ongoing experimentation of new data sources, Preparing and maintaining high-quality data for training ML models. Tracking models in production to detect performance degradation. 05 01 02 03 04 Challenges
- 14. Benefits of MLOps Shorterdevelopment cycles, and as a result, shorter time to market. Better collaboration between teams. Increased reliability, performance, scalability, and security of ML systems. Streamlined operational and governance processes. Increased return on investment of ML projects.
- 15. Relationship of Data Engineering,ML Engineering, and Application Engineering. • Data engineering involves ingesting, integrating, curating, and refining data to facilitate a broad spectrum of operational tasks, data analytics tasks, and ML tasks. • ML models are built and deployed in production using curated data that is usually created by the data engineering team.
- 16.
- 17.
- 18. Core MLOps Technical Capabilities of MLPlatforms Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 19. Experimentation Lets data scientistsand ML researchers collaboratively perform EDA, create prototype model architectures, and implement training routines. Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 20. Data Processing Lets youprepare and transform large amounts of data for ML at scale in ML development, in continuous training pipelines, and in prediction serving. Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 21. Model training lets youefficiently and cost-effectively run powerful algorithms for training ML models Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 22. Model evaluation lets youassess the effectiveness of your model, interactively during experimentation and automatically in production. Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 23. Model serving lets youdeploy and serve your models in production environments. Key functionalities in model serving include support for near-real-time, low latency prediction, logging etc… Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 24. Online experimentation lets you understandhow newly trained models perform in production settings compared to the current models before you release the new model to production.
- 25. Model monitoring lets youtrack the efficiency and effectiveness of the deployed models in production to ensure predictive quality and business continuity. Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 26. ML pipelines lets youinstrument, orchestrate, and automate complex ML training and prediction pipelines in production. Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 27. Model registry lets yougovern the lifecycle of the ML models in a central repository. This ensures the quality of the production models and enables model discovery. Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 28. Dataset & feature repository letsyou unify the definition and the storage of the ML data assets. Helps data scientists and ML researchers save time on data preparation and feature engineering Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 29. ML metadata & artifactrepository Enables reproducibility and debugging of complex ML tasks and pipelines. Metadata about ML artifacts such as descriptive statistics, data schemas, trained models, and evaluation results are tracked in it. Image: https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 30.
- 31. Problem Statement Automatically classify customersupport ticket and route to the right support agent Automated customer support routing Data Selection & Exploration - Ticket Info - Trip info - Customer info Feature engineering - Ticket message - Time after trip - … Model prototyping & validation - learning-to-rank approach - retrieval-based pointwise ranking Training pipeline - Batch jobs - Yarn/Mesos cluster Data pipeline - Data transformation - Kafka (pub/sub) - Samza (stream processing) - Cassandra (for training) Model Refresh - Model tuning - Model evaluation - Model validation Service integration - Offline mode - Online mode - Library mode CI/CD pipeline - Dynamic model loading - Artifacts Validation - Serving validations Online experimentation - A/B testing Monitoring integration - Kafka - Kibana (dashboards) Model monitoring - RMSLE - RMSE - R-suqared Data & feature repository - HDFS (data) - Cassandra (feature metadata) Public Launch - Gradual rollout - Online /Batch/Embedded inference Model repository - HDFS (zip archive) - Cassandra (model metadata) ML Development Training operationalization Continuous training Prediction serving Model deployment Model monitoring
- 32. MLOps End-to-End Workflow Image:https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf
- 33. Take Away Delivering businessvalue through ML is not only about building the best ML model for the use case at hand, but also about building an integrated ML system that operates continuously to adapt to changes in the dynamics of the business environment. Such an ML system involves ❑ Collecting, processing, and managing ML datasets and features; ❑ Training, and evaluating models at scale; ❑ Serving the model for predictions; ❑ Monitoring the model performance in production; and ❑ Tracking model metadata and artifacts. … and MLOps enables building such an ML System.
- 34. BOOK COMING SOON Anish Cheriyan, RajithRaveendran, Vimal Das Kammath With Sheena Lakshmi https://softwareforindustrynext.blogspot.com/
- 35. References • Practitioners guideto MLOps:A framework for continuous delivery and automation of machine learning.Khalid Salama, Jarek Kazmierczak, Donna Schut, Google Cloud White Paper, 2021 • • Engineering MLOps: Rapidly build, test, and manage production-ready machine learning life cycles at scale- Emmanuel Raj
- 36.