ML Deployment Woes Persist

Despite greater spending on staffing and use cases, investors in machine learning have so far reaped few returns as they struggle with life cycle issues related to data governance, security and auditing requirements.

An annual assessment released this week by Algorithmia on enterprise trends in machine learning found that machine learning investments are up, but adopters continue to struggle to reach production due to regulatory requirements. The survey released on Thursday (Dec. 10) found that 67 percent of the more than 400 executives polled said they must comply with multiple regulations covering data used in machine learning models.

Still, 83 percent of organizations said they are pressing on with ML development, increasing budget and hiring more data scientists. Staffing increased 76 percent over the previous year, Seattle-based Algorithmia reported.

The slow deployment of machine learning models in production has fueled a growing MLOps sector. Vendors such as Algorithmia maintain their tools are helping adopters manage each phase of the machine learning life cycle to maintain model performance. For example, Algorithmia touts its approach as automated ML deployment while leveraging existing continuous integration/continuous delivery processes. All that, plus tighter security and improved data governance.

“Today, organizations are worrying more about how to get [machine learning] models into production faster and how to ensure their performance over time,” Algorithmia CEO Diego Oppenheimer noted in releasing the annual report.

Other data points from the machine learning study include a 74-percent increase since last year in the number of organizations with five or more AI and machine learning use cases. Business process automation and customer relations were the leading candidates, widely seen as delivering the fastest return on AI investments.

The study also found that the time required to deploy a trained machine learning model in production continues to increase, with nearly two-thirds of respondents taking at least a month to deploy a model. Meanwhile, 38 percent said data scientists spend more time on model deployment, a trend that increased as more models are trained.

“The bottom line is, organizations have increased their AI/ML resources without solving underlying challenges with operational efficiency,” the survey found. “This has exacerbated the problem and led to organizations spending more time and resources on model deployment.”

The report echoes earlier studies and vendor surveys that emphasize the impact of the pandemic on enterprise automation efforts. Last year, for example, only 7 percent of those companies polled by Algorithmia said they were increasing spending on machine learning development. This year, 20 percent report comparable AI and machine learning budget increases since the pandemic hit.

Last year’s ML report stressed scaling problems, including version control and model reproducibility. “Executive buy-in” was also cited as a stumbling block. At least that issue appears to have been resolved as development and staffing budgets have increased over the last year.

Still, stepped up development efforts have thus far failed to produce comparable increases in machine learning production workloads. Moreover, data scientists remain bogged down in deployment issues rather than tracking down training data for new models.

Algorithmia’s annual report on machine learning trends can be downloaded here.

Recent items:

Algorithmia Laser-Focused on ML Deployment and Management

Machine Learning Hits a Scaling Bump