As the ‘AI era’ of increasingly complex, smart, autonomous, big-data-based tech comes upon us, the algorithms that fuel it are getting under more and more scrutiny.
Whether you’re a data scientist or not, it becomes obvious that the inner workings of machine learning, deep learning, and black-box neural networks are not exactly transparent. In the wake of high-profile news reports concerning user data breaches, leaks, violations, and biased algorithms, that is rapidly becoming one of the biggest — if not the biggest — sources of problems on the way to mass AI integration in both the public and private sectors.
Here’s where the push for better AI interpretability and explainability takes root. Already a focal point of machine learning consulting and a notable topic in the 2019 AI discussions, it’s only likely to accelerate and become one of the central conversations of 2020 regarding the questions of both security and ethics of artificial intelligence.
The days of the ideas like ‘machines will become too smart and independent and will rise against humanity’ are long behind us, with the sentiment firmly relegated to the realm of science fiction and entertainment. By now, much more justifiable apprehensions, grounded in the socio-economic reality, took place in the public consciousness:
● When AI is making judgements and appraising risks, why and how does it come to the conclusions it presents?
● What is considered failure and success? Why?
● If there’s an error or a biased logic, how do we know?
● How do we identify and fix such issues?
● Are we sure we can trust AI?
These are the questions that need to be answered in order to be able to rely on AI, and be sure about its accountability. Here’s where AI interpretability and explainability comes into play.
Where machine learning and AI is concerned, “interpretability” and “explainability” are often used interchangeably, though it’s not correct for 100% of situations. While closely related, these terms denote different aspects of predictability and understanding one can have of complex systems, algorithms, and vast sets of data. See below:
● Interpretability refers to the ability to observe cause-and-effect situations in a system, and, essentially, predict which changes will cause what type of shifts in the results (without necessarily understanding the nitty-gritty of it all).
● Explainability is basically the ability to understand and explain ‘in human terms’ what is happening with the model; how exactly it works under the hood.
The difference is subtle enough, but it’s there. While usually both can co-exist, some situations might require one and not the other: for example, when explaining what’s behind a predictive model to the higher-ups of the banking or the pharmaceutical industry, demonstrating the measures taken to minimize or eliminate the possibility of bias in the risk assessment models for their legal systems.
Being able to present and explain extremely complex mathematical functions behind predictive models in understandable terms to human beings is an increasingly necessary condition for real-world AI applications.
As algorithms become more complicated, fears of undetected bias, mistakes, and miscomprehensions creeping into decision-making grow among policymakers, regulators, and the general public. In such an environment, interpretability and explainability are crucial for achieving fair, accountable and transparent (FAT) machine learning, complying with the needs and standards for:
It is paramount for any business predictions to be easily explained to a boss, a customer, or a commercial legal adviser. Simply speaking, when any justification for an important business decision is reduced to “the algorithm made us do it,” you’ll have a hard time making anyone — be it investors, CEOs, CIOs, end customers, or legal auditors — buy the fairness, reliability, and business logic of this algorithm.
Applying regulations, such as the GDPR, regional and local laws, to machine learning models can only be fully achieved with the FAT principles at the core. For example, Article 22 Recital 71 of the GDPR specifically states: “The data subject shall have the right not to be subject to a decision based solely on automated processing, including profiling, which produces legal effects concerning him or her or similarly significantly affects him or her.”
In turn, Articles 13 and 15 stress repeatedly that data subjects have a right to the disclosure of “meaningful information about the logic involved” and of “the significance and the envisaged consequences” of automated decision-making.
To make a GDPR-compliant AI is to make an interpretable, explainable AI. In the world of rapidly developing and spreading laws regarding data, that can soon mean “to make any compliant AI is to make an interpretable, explainable AI.”
The problem of algorithmic bias and the dangers it can harbor when allowed into machine learning systems are well-known and documented. While the main reason behind biased AI is the poor quality of data fed into it, the lack of transparency in the proceedings and, as a result, inability to quickly detect bias are among the key factors here, as well.
Imagine the times when interpretable and explainable AI becomes the norm. Then the ability to understand not only the fundamental techniques used in a model but also particular cause-and-effect ties found in those specific algorithms would allow for faster and better bias detection. This has a potential to eliminate the problem itself, or at least to allow for a much quicker and more effective solution to it, which is one of the main socio-economic reasons behind the current push for both fair and ethical AI.
Regardless of the type and scope of a software development project, probably no one has ever described documentation keeping as fun. Yet it must be done, and predictive models are no exception.
Where AI, machine learning, and especially black-box deep learning are concerned, in some cases this usually tedious task can become impossible altogether. Basically speaking, black-box modeling can be great for dealing with data regardless of a particular mathematical structure of the model, but if you need to document the specifics — be it for a commercial, educational, or other project — you’re out of luck. This model would need to become both interpretable and explainable, in order for an efficient documentation to be created.
In many ways, the shift to interpretable and explainable AI is akin to mandatory safe working conditions for employees: not there from the get-go, costly to provide, and probably still seen by some employers as something their life would be much easier without. Yet, it’s absolutely necessary and deemed so by the majority of people.
The problem behind explainable AI and ML is that it’s more difficult. It adds complexity to an already complex project, just to explain how and why it’s complex. The sentiment like “if it works well enough, it’s good enough” are still strong, so the idea of adding extra difficulty to serve as a safety net makes interpretability seem redundant and time-consuming. Yet, despite all that, for the time being it seems like the only path for AI to truly become mass-adopted.