Friday, February 26, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Machine Learning

Google Announces Cloud AI Platform Pipelines to Simplify Machine Learning Development

March 29, 2020
in Machine Learning
Google Announces Cloud AI Platform Pipelines to Simplify Machine Learning Development
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

In a recent blog post, Google announced the beta of Cloud AI Platform Pipelines, which provides users with a way to deploy robust, repeatable machine learning pipelines along with monitoring, auditing, version tracking, and reproducibility. 

With Cloud AI Pipelines, Google can help organizations adopt the practice of Machine Learning Operations, also known as MLOps – a term for applying DevOps practices to help users automate, manage, and audit ML workflows. Typically, these practices involve data preparation and analysis, training, evaluation, deployment, and more. 

You might also like

New machine learning tool facilitates analysis of health information, clinical forecasting

Supercomputer-Powered Machine Learning Supports Fusion Energy Reactor Design

Something’s Fishy — New Funding To Tackle Illegal Activities At Sea Using Machine Learning And Data Analytics

Google product manager Anusha Ramesh and staff developer advocate Amy Unruh wrote in the blog post: 

When you’re just prototyping a machine learning (ML) model in a notebook, it can seem fairly straightforward. But when you need to start paying attention to the other pieces required to make an ML workflow sustainable and scalable, things become more complex.

Moreover, when complexity grows, building a repeatable and auditable process becomes more laborious.

Cloud AI Platform Pipelines – which runs on a Google Kubernetes Engine (GKE) Cluster and is accessible via the Cloud AI Platform dashboard – has two major parts: 

  • The infrastructure for deploying and running structured AI workflows integrated with GCP services such as BigQuery, Dataflow, AI Platform Training and Serving, Cloud Functions, and
  • The pipeline tools for building, debugging and sharing pipelines and components.

With the Cloud AI Platform Pipelines users can specify a pipeline using either the Kubeflow Pipelines (KFP) software development kit (SDK) or by customizing the TensorFlow Extended (TFX) Pipeline template with the TFX SDK. The latter currently consists of libraries, components, and some binaries and it is up to the developer to pick the right level of abstraction for the task at hand. Furthermore, TFX SDK includes a library ML Metadata (MLMD) for recording and retrieving metadata associated with the workflows; this library can also run independently. 

Google recommends using KPF SDK for fully custom pipelines or pipelines that use prebuilt KFP components, and TFX SDK and its templates for E2E ML Pipelines based on TensorFlow. Note that over time, Google stated in the blog post that these two SDK experiences would merge. The SDK, in the end, will compile the pipeline and submit it to the Pipelines REST API; the AI Pipelines REST API server stores and schedules the pipeline for execution.

An open-source container-native workflow engine for orchestrating parallel jobs on Kubernetes called Argo runs the pipelines, which includes additional microservices to record metadata, handle components IO, and schedule pipeline runs. The Argo workflow engine executes each pipeline on individual isolated pods in a GKE cluster – allowing each pipeline component to leverage Google Cloud services such as Dataflow, AI Platform Training and Prediction, BigQuery, and others. Furthermore, pipelines can contain steps that perform sizeable GPU and TPU computation in the cluster, directly leveraging features like autoscaling and node auto-provisioning.
 
Source: https://cloud.google.com/blog/products/ai-machine-learning/introducing-cloud-ai-platform-pipelines

AI Platform Pipeline runs include automatic metadata tracking using the MLMD – and logs the artifacts used in each pipeline step, pipeline parameters, and the linkage across the input/output artifacts, as well as the pipeline steps that created and consumed them.

With Cloud AI Platform Pipelines, according to the blog post customers will get:

  • Push-button installation via the Google Cloud Console
  • Enterprise features for running ML workloads, including pipeline versioning, automatic metadata tracking of artifacts and executions, Cloud Logging, visualization tools, and more 
  • Seamless integration with Google Cloud managed services like BigQuery, Dataflow, AI Platform Training and Serving, Cloud Functions, and many others 
  • Many prebuilt pipeline components (pipeline steps) for ML workflows, with easy construction of your own custom components

The support for Kubeflow will allow a straightforward migration to other cloud platforms, as a respondent on a Hacker News thread on Google AI Cloud Pipeline stated:

Cloud AI Platform Pipelines appear to use Kubeflow Pipelines on the backend, which is open-source and runs on Kubernetes. The Kubeflow team has invested a lot of time on making it simple to deploy across a variety of public clouds, such as AWS, and Azure. If Google were to kill it, you could easily run it on any other hosted Kubernetes service.

The release of AI Cloud Pipelines shows Google’s further expansion of Machine Learning as a Service (MLaaS) portfolio – consisting of several other ML centric services such as Cloud AutoML, Kubeflow and AI Platform Prediction. The expansion is necessary to allow Google to further capitalize on the growing demand for ML-based cloud services in a market which analysts expect to reach USD 8.48 billion by 2025, and to compete with other large public cloud vendors such as Amazon offering similar services like SageMaker and Microsoft with Azure Machine Learning.

Currently, Google plans to add more features for AI Cloud Pipelines. These features are:

  • Easy cluster upgrades 
  • More templates for authoring ML workflows
  • More straightforward UI-based setup of off-cluster storage of backend data
  • Workload identity, to support transparent access to GCP services, and 
  • Multi-user isolation – allowing each person accessing the Pipelines cluster to control who can access their pipelines and other resources.

Lastly, more information on Google’s Cloud AI Pipeline is available in the getting started documentation.


Credit: Google News

Previous Post

The Zen Of Kaggle Mastery With Mathurin Aché: Kaggle Top 20

Next Post

Coronavirus: Making us love Facebook and Google all over again

Related Posts

Basic laws of physics spruce up machine learning
Machine Learning

New machine learning tool facilitates analysis of health information, clinical forecasting

February 26, 2021
Supercomputer-Powered Machine Learning Supports Fusion Energy Reactor Design
Machine Learning

Supercomputer-Powered Machine Learning Supports Fusion Energy Reactor Design

February 26, 2021
Something’s Fishy — New Funding To Tackle Illegal Activities At Sea Using Machine Learning And Data Analytics
Machine Learning

Something’s Fishy — New Funding To Tackle Illegal Activities At Sea Using Machine Learning And Data Analytics

February 26, 2021
Cloudera aims to fast track enterprise machine learning use cases with Applied ML Prototypes
Machine Learning

Cloudera aims to fast track enterprise machine learning use cases with Applied ML Prototypes

February 25, 2021
Machine learning‐based analysis of alveolar and vascular injury in SARS‐CoV‐2 acute respiratory failure – Calabrese – – The Journal of Pathology
Machine Learning

Machine learning‐based analysis of alveolar and vascular injury in SARS‐CoV‐2 acute respiratory failure – Calabrese – – The Journal of Pathology

February 25, 2021
Next Post
Coronavirus: Making us love Facebook and Google all over again

Coronavirus: Making us love Facebook and Google all over again

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

Basic laws of physics spruce up machine learning
Machine Learning

New machine learning tool facilitates analysis of health information, clinical forecasting

February 26, 2021
Creative Destruction and Godlike Technology in the 21st Century | by Madhav Kunal
Neural Networks

Creative Destruction and Godlike Technology in the 21st Century | by Madhav Kunal

February 26, 2021
Spy agency: Artificial intelligence is already a vital part of our missions
Internet Security

Spy agency: Artificial intelligence is already a vital part of our missions

February 26, 2021
Blockchain lags behind other technologies in finance adoption for now, says Broadridge
Blockchain

Blockchain lags behind other technologies in finance adoption for now, says Broadridge

February 26, 2021
Supercomputer-Powered Machine Learning Supports Fusion Energy Reactor Design
Machine Learning

Supercomputer-Powered Machine Learning Supports Fusion Energy Reactor Design

February 26, 2021
How 3D Cuboid Annotation Service is better than free Tool? | by ANOLYTICS
Neural Networks

How 3D Cuboid Annotation Service is better than free Tool? | by ANOLYTICS

February 26, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • New machine learning tool facilitates analysis of health information, clinical forecasting February 26, 2021
  • Creative Destruction and Godlike Technology in the 21st Century | by Madhav Kunal February 26, 2021
  • Spy agency: Artificial intelligence is already a vital part of our missions February 26, 2021
  • Blockchain lags behind other technologies in finance adoption for now, says Broadridge February 26, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates