Accelerate machine learning (ML) workflows for data scientists with Red Hat OpenShift, a self-service Kubernetes hybrid cloud platform
AI is the capability of machines to imitate intelligent human behavior, and perform tasks that normally require humans. The term “artificial intelligence” was coined in 1955 by John McCarthy, a mathematics professor at Dartmouth college.1
ML is a subset of AI that gives computers the ability to learn without being explicitly programmed. Computers use algorithms and statistical models to perform specific tasks without explicit instructions, relying on patterns and inference instead.2
Deep learning is a subset of machine learning that uses multiple layers to progressively extract higher level features from the raw input. Deep learning architectures have been applied to fields including computer vision, natural language processing, image analysis, etc.3
Machine learning lifecycle is a multi phase process to obtain the power of large volumes and variety of data, abundant compute, and open source machine learning tools to build intelligent applications.
At a high level, there are three steps in the lifecycle:
Data scientists are primarily responsible for ML modelling to ensure the selected ML model continues to provide the highest prediction accuracy.
The key challenges data scientists face are:
Containers and Kubernetes are key to accelerating the ML lifecycle as these technologies provide data scientists the much needed agility, flexibility, portability, and scalability to train, test, and deploy ML models.
Red Hat OpenShift is the industry's leading containers and Kubernetes hybrid cloud platform. It provides all these benefits, and through the integrated DevOps capabilities and integration with hardware accelerators, OpenShift enables better collaboration between data scientists and software developers, and accelerate the roll out of intelligent applications across hybrid cloud (data center, edge, and public clouds).
Integrations with popular hardware accelerators e.g. NVIDIA GPUs and NGC containers ensures that OpenShift can seamlessly meet the high compute resource requirements to help select the best ML model providing the highest prediction accuracy, and ML inferencing jobs as the model experiences new data in production.
Extending OpenShift DevOps automation capabilities to the ML lifecycle enables collaboration between data scientists, software developers, and IT operations so that ML models can be quickly integrated into the development of intelligent applications. This helps boost productivity, and simplify lifecycle management for ML powered intelligent applications.
OpenShift is helping organizations across various industries to accelerate business and mission critical initiatives by developing intelligent applications in the hybrid cloud. Some example use cases include fraud detection, data driven diagnostics and cure, connected cars, autonomous driving, oil and gas exploration, automated insurance quotes, claims processing, etc.
Red Hat Ceph Storage was built to address petabyte-scale storage requirements in the ML lifecycle, from data ingestion and preparation, ML modeling, to the inferencing phase. It is an open source software defined storage system which provides comprehensive support for S3 object, block, and file storage, and delivers massive scalability on industry standard commodity hardware.
For example, you can present scalable Ceph storage to containerized Jupyter notebooks on OpenShift via S3 or persistent volumes.
Open Data Hub Project is a functional architecture based on OpenShift, Red Hat Ceph Storage, Red Hat AMQ Streams, and several upstream open source projects to help build an open ML platform with the necessary ML tooling.
Customer success stories