Social Media
Follow us on our social media channels
Contact Us
Thank you! We will get back to you as soon as possible.
Oops! Something went wrong while submitting the form.
2024-03 Training Machine Learning Models on Kubernetes Using Airflow
Event Start:
March 27, 2024 5:30 PM
Event End:
March 28, 2024
Location:
Join us at Platform Calgary on Wednesday March 27th at 5:30pm for our combined PyData and PyYYC meetup group, where Sourabh from Orennia will present on training ML models on Kubernetes using Airflow.
**Schedule:**
5:30 - Pizza and Networking
6:00 - Presentations and Discussion
7:30 - Wrap up
**Training Machine Learning Models on Kubernetes Using Airflow**
**Presenter:** Sourabh Mokhasi from Orennia
There are high costs associated with training machine learning models, particularly on platforms like AWS SageMaker. This project leverages our existing Kubernetes cluster on AWS EKS and airflow orchestrator, to design a highly scalable MLOps pipeline that is suitable for orchestrating and running machine learning jobs such as data pre-processing, model training and model inference. Furthermore, the project harnesses the power of Airflow's mapped tasks feature, enabling the concurrent training of hundreds of machine learning models. This innovative approach not only optimizes resources but also significantly reduces the overall expense of machine learning model training.
**Bio:**
I am a cloud-native services specialist, deeply focused on data engineering, ML Ops, and Kubernetes-based implementations. My expertise spans designing and implementing scalable cloud solutions that drive innovation and efficiency. Throughout my career, I have contributed to a variety of projects, such as a self-hosted Kafka cluster, a Datalake implementation using Apache Iceberg, real-time data engineering pipelines and many more. In addition to my project work, I share my knowledge and experiences through technical blog posts, aiming to enrich the tech community with insights from real-world applications.
Visit Event PageEvent Has Passed