Artificial Intelligence is a hot topic in different industries. Nowadays, it has been mentioned a lot during business meetings and product planning. People tend to think of it as the magic that will solve their business challenges. Other entrepreneurs think of it as a short cut to create great values within their products or services.
Is this true? Is it as those people expect it to be? There is no doubts that AI has opened up a horizon of creativity and innovation. This will be continuously reflected in different industries. I am a big fan of AI in Healthcare, and it is fascinating to me how AI is transforming the industry specifically these days with the corona epidemic. Indeed, This was my motive for diving in the field, and shifting my career from key account management into AI product management.
However and as an AI product manager, I had the chance to meet with different clients in order to discuss how our AI products can be a great benefit in their workflow. Many of the responses were surprising to me! Some of clients were expecting that AI will eliminate human work or input completely, at least in the first stages. Others were discussing why they should expect some errors in the models’ output! They were thinking: “It is an AI; it should be 100% correct”. I found myself in a position where I need to deliver the idea of machine learning and how the model must be generic neither overfitted nor underfitted.
Yet, the most effective metrics that I explained to them was what I call it “the business evaluation metrics”. Because, I have realized that delivering the technical evaluation metrics or the model performance only (such as model’s accuracy, recall, precision …etc) is not enough for the business to make the decision of adopting the product. However, setting the business metrics with your client can make him/her more comfortable with taking the decision quicker. On the other hand, it can assure the deliverables and set the expectations for both of you. I can see this beneficial to you as an AI product manager! This way you can communicate with your team (data scientist and machine learning engineers) the desired acceptance criteria for the product.
Firstly, I have to point out that you can add more dimensions to the metrics depending on the business you are addressing. In general, we can define the model’s values in three main criteria’s:
- Performance: technical performance such as accuracy for the model on accomplishing a certain amount of tasks.
- Time: how much time to accomplish a certain amount of tasks
- Cost: how much it cost to accomplish a certain amount of tasks.
Simply you can think of your AI model as a new employee for the client. Those three criteria’s are the main KPIs that you can benchmark your model “the new employee” against the current workflow.
Let us say that you have developed a model that helps doctors and radiologists to detect COVID-19 in the x-ray images of the patients’ chest. Your model has to classify the image whether it’s healthy or Corona Virus-Infected. Since it is a new virus and due to limited data resources, the model’s accuracy is 70% only. Is this enough?! Let us make the comparison for screening 100 images between the employee and the AI model:
2. AI Model
Although, the AI model is less accurate than humans. But it has lower false-negative rates. It takes much less time to inspect the 100 images which means 18 times faster than manual inspection. Moreover, it saves $500 per 100 cases. We can see that the AI model is adding a great benefit to the healthcare institution. Especially those days where time and money are very valuable in such an epidemic.
#StayHome #StaySafe #WashYourHands