AWS launches human review service for machine learning models

AWS has launched a service that improves machine learning model accuracy by continuously identifying and improving low confidence predictions. 

Amazon Augmented Artificial Intelligence (A2I), which is now generally available, allows for human review of model predictions using reviewers from Mechanical Turk, third-party vendors, or an organisations’ own employees.

The service aims to help developers build human review systems, structure the review process and manage the review workforce. AWS suggests that developers could use A2I to quickly spin up and manage a workforce of humans to review and validate the accuracy of machine learning predictions. 

This could be used for an application that collects financial information from scanned mortgage documents or even one that uses image recognition to identify counterfeit items online. The idea is that the review helps the predictive model get better over time. 

A2I could be used by AWS customers that use services such as Amazon SageMaker, Amazon Rekognition or Amazon Textract, which are often used in critical and sensitive cases that need often need human judgements. 

“Even with these advancements, our customers still say there are critical use cases where human judgment is required like in law enforcement investigations or times when human review can be used to resolve the ambiguity in predictions when confidence levels fall below a given threshold for less sensitive use cases, and the current human review process involves a lot of custom effort and cost,” said Swami Sivasubramanian, VP for Amazon Machine Learning.”

A2I is already being used by a number of organisations, including the National Health Service (NHS) Business Services Authority, which provides a range of support services to NHS organisations, NHS contractors, and patients. The organisation processes 54 million paper prescriptions each month, according to AWS. 

“Human judgment is critical and in fact is often required for decisions involving medical payments,” said Chris Suter, the head of cloud platforms and innovation at NHS BSA.