Gartner Research

Building a Framework for Managing Effective Machine Learning Workloads

Published: 10 April 2019

ID: G00384678

Analyst(s): Sumit Agarwal

Summary

Many organizations struggle to take data science projects from prototyping to production. This research provides a framework for data and analytics technical professionals to establish best practices through the build, train, deploy and monitor phases of the machine learning development life cycle.

Table Of Contents

Problem Statement

The Gartner Approach

The Guidance Framework

  • Prework: The Building Blocks
    • Establish the Role of a Machine Learning Architect
    • Incorporate a Model Management System
    • Ensure Effective DevOps and DataOps
  • Step 1: Build the ML Model
    • Data Wrangling
    • Feature Engineering
    • Model Algorithm Selection
    • Toolsets and Compute Resource Provisioning
    • Gartner Insights
  • Step 2: Train and Test the Selected Models
    • Define Model Accuracy Metrics
    • Identify Training Data and Optimize Hyperparameters
    • Model Deployment for Training
    • Gartner Insights
  • Step 3: Deploy the Trained Model for Inference
    • Performance-Related Service Levels, Access Control and Rollback Strategy
    • Configure Model Serving
    • Model Deployment in Containers
    • Gartner Insights
  • Step 4: Monitor the Inference Engine for Deviation and Performance
  • Follow-Up
    • Multiple Paths
    • Roles and Responsibilities

Risks and Pitfalls

  • Teams Working in Silos
  • Working Software Is Not Enough
  • Technology Focus

Gartner Recommended Reading

©2019 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form without Gartner’s prior written permission. It consists of the opinions of Gartner’s research organization, which should not be construed as statements of fact. While the information contained in this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartner research may address legal and financial issues, Gartner does not provide legal or investment advice and its research should not be construed or used as such. Your access and use of this publication are governed by Gartner’s Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its research is produced independently by its research organization without input or influence from any third party. For further information, see Guiding Principles on Independence and Objectivity.

Already have a Gartner Account?

Become a client