Overcoming the challenges of machine learning at scale

Machine learning (ML) and artificial intelligence (AI) technologies are increasingly on the investment list for IT leaders. Among the many benefits of these technologies, building and deploying ML models can add automation to mundane, repetitive tasks and let humans focus on more mission-critical work. ML also gives businesses the power to extract meaning from the massive amounts of data they are collecting and generating annually.

The productivity and analytics gains are not lost on business leaders. In IDG’s 2019 Digital Business Study, 78% of IT and business leaders said their organization is considering or has already deployed machine learning technologies as part of their digital business strategy. And as CIO.com observes, machine learning is one of the highest in-demand skills in today’s technology job market.

As ML models gain more traction throughout the enterprise, we asked our IDG Influencer community of experts about the biggest challenges to scaling machine learning across the enterprise, and the best ways to overcome those challenges. Here’s a summary of their insights.

Time and complexity

Common roadblocks as organizations look to scale machine learning across the enterprise are, perhaps not surprisingly, time and complexity.

“One challenge we regularly encounter when working with clients is the time it takes to develop and deploy machine learning models and associated machine learning applications,” said Gene De Libero (@GeneDeLibero), Chief Strategy Officer and Head of Consulting with GeekHive. “We’ve found that applying continuous delivery principles to scaling machine learning across the enterprise helps us deliver services faster and more reliably. This approach also helps reduce overall risk and shorten the time it takes to attain speed and scale.”

Scott Schober (@ScottBVS), President and CEO of Berkeley Varitronics Systems, Inc. noted the larger the organization, the harder it is to extract enterprise-wide value.

“Large companies can be fragmented and tend to build up many siloed teams over time,” he said. “This independence among employees and teams can hinder homogenous adoption of ML systems, which are only effective when implemented throughout the entire organization and its shared data sets. Only when a company carefully structures itself by implementing a clear centralized effort, can It greatly reduce or eliminate the fragmented processes and overcome the challenges of scaling machine learning.”

Quality data

Data fuels machine learning – and one of the hurdles is ensuring that ML models and underlying systems are set up to deliver accurate results and information, said Jeff Cutler (@JeffCutler), a technology journalist.

“The biggest challenge to machine learning is believing the simulations and being able to make decisions based on the data mined and processed by your systems,” he said. “If you put your trust in bad data –or in theories and projections based on bad data – you’re lost.”

Creating a track record of positive, data-driven decisions that you can share with the organization will help to justify the value of machine learning, Cutler added.

Jo Peterson (@digitalcloudgal), VP of the Cloud Services Practice at Clarify360, also stressed the importance of high-quality data. “AI technology is data driven,” she said. “Clean, quality data that is comprehensively managed and integrates across systems is a good starting point.”

The volume and quality of data are both critical for scaling a machine learning proof of concept (POC) into production, says Sri Elaprolu (@SriElaprolu), Senior Leader, Amazon Machine Learning Solutions Lab. “Make sure the data you’re using in the POC is representative of the real world,” he said. “Not just doing it one time, but rather thinking about what you’ll need as you’re collecting more data.”

Factors to consider include: What are the best mechanisms for gathering the data you need? How do you store the data for machine learning purposes? How do you apply models on top of the data? And, importantly, how do you iterate? “In the real world, conditions change, and therefore data will also change over time,” Elaprolu said. “So you need to have that continuous loop of retraining and redeploying.”

Culture and KPIs

As with any emerging technology, another challenge is ensuring a positive return on investment with respect to business objectives. Success requires adjustments to both process and culture.

“Organizations that are serious about scaling machine learning and bringing more models from the lab to production are investing in the processes, tools, and skills to support model management and operations,” said Isaac Sacolick (@nyike), President of StarCIO and author of Driving Digital. “Organizations should start with high-value and easy-to-execute experiments, but then must recognize that scaling requires an investment in an end-to-end machine learning lifecycle.”

Tim Crawford (@tcrawford), CIO Strategic Advisor with AVOA, also emphasized the importance of process and culture. “First step, create a methodology and culture that supports ML and prioritizes how to engage ML,” he said. “Identifying the right projects, prioritizing, ensuring that you have enough good data and creating a culture that embraces ML across the enterprise.”

A lack of alignment between ML projects and the business can hobble efforts to scale the technology, said Will Kelly (@willkelly), a technical writer. “Data scientists and cloud engineers can overcome such challenges by taking a ‘land and expand’ approach,” Kelly said. “Such an approach leverages DevOps and cloud analytics, starting with small iterative pilot projects focused on business unit problems, then working up to enterprise-level production projects.”

Measuring success along the way is critical. “The economics of ML can be elusive, especially if the business goals to be accomplished are not crystal clear,” said Frank Cutitta (@fcutitta), CEO of HealthTech Decisions Lab. “Far too many enterprises see ML as being a box to be checked on a board presentations slide without a clear ROI success metric for the investment in talent and technology.”

KPIs will evolve over time as machine learning capabilities evolve. For example, in security, the influx of data from an expanding technology ecosystem “creates a digital haystack in which criminals can hide,” said Mark Sangster (@mbsangster), VP and Industry Security Advocate at eSentire Inc. “At eSentire, we enlisted several universities and research institutions to examine the end-to-end investigation process in our SOC [security operations center] to identify not only indicators of concern (signatures of malicious activity) but the thought process and steps analysts go through to validate a threat.” This is one example of how AI and machine learning can evolve “in a scaled manner from rules-based to true feedback-fed improvement cycles,” Sangster said.

Education and awareness

A fundamental challenge for complex technology such as machine learning is helping business leaders cut through the hype and dispel misconceptions.

“The first challenge is to reinforce that ML is not simply the second acronym after AI,” said Cutitta. “Machine learning drives artificial intelligence insights.”

“Many tools are still very ‘techy’ and difficult for the average end-user to use,” added Dave Evans (@DaveTheFuturist), CIO & VP of Technology at The Computer History Museum. “AI companies need to focus on obfuscating a lot of the ML jargon and make it easy for users. … A user-friendly ‘front-end’ will be key to scaling and adoption.”

Learn more about machine learning and AI services for your business.