Along with the growth of AI have come serious questions about our ability to build unbiased, “fair” algorithms. And it’s true that without intervention, machine learning algorithms will reflect any biases in the underlying data. But precisely because ML requires us to instruct it in highly precise ways about what sort of outcomes we’ll find ethically acceptable, it’s also giving us the tools to have these discussions in clearer and more productive ways. We’re defining a whole new vocabulary and set of concepts to talk about fairness.
Bias is machine learning’s original sin. It’s embedded in machine learning’s essence: the system learns from data, and thus is prone to picking up the human biases that the data represents. For example, an ML hiring system trained on existing American employment is likely to “learn” that being a woman correlates poorly with being a CEO.
Cleaning the data so thoroughly that the system will discover no hidden, pernicious correlations can be extraordinarily difficult. Even with the greatest of care, an ML system might find biased patterns so subtle and complex that they hide from the best-intentioned human attention. Hence the necessary current focus among computer scientists, policy makers, and anyone concerned with social justice on how to keep bias out of AI.
Yet machine learning’s very nature may also be bringing us to think about fairness in new and productive ways. Our encounters with machine learning (ML) are beginning to give us concepts, a vocabulary, and tools that enable us to address questions of bias and fairness more directly and precisely than before.
AI and BiasSponsored by SAS
Building fair and equitable machine learning systems.
We have long taken fairness as a moral primitive. If you ask someone for an example of unfairness, the odds are surprisingly high that they’ll talk about two children who receive different numbers of cookies. That’s clearly unfair, unless there is some relevant difference between them that justifies the disparity: one of the children is older and bigger, or agreed to do extra chores in return for a cookie, etc. In this simple formulation, fairness gets defined as the equal treatment of people unless there is some relevant distinction that justifies unequal treatment.
But what constitutes a “relevant distinction”? The fact is that we agree far more easily about what is unfair than what is fair. We may all agree that racial discrimination is wrong, yet sixty years later we’re still arguing about whether Affirmative Action is a fair remedy.
For example, we can all agree that in the 1970s, it was unfair that women musicians made up as little as 5% of the top five symphony orchestras. In this case, we might agree that the actual remedy orchestras institute seems far fairer: by having applicants audition behind a curtain to mask their gender, the percentage of women in the five top symphony orchestras rose to 25% in 1997, and to 30% now.
But is a gender-blind process enough to make the outcome actually fair? Perhaps cultural biases confer non-biological advantages on male musicians — if more men were accepted to top conservatories, for example, they may have received better musical education. Perhaps standards of performance in music have been shaped over the centuries around typically male traits or preferences, such as palm sizes or the aggressiveness of performance. And is 30% enough for us to declare that the orchestras are now fair in their treatment of women? Perhaps the gender breakdown of musicians should be 51% to mirror the overall national gender demographics? Or perhaps it should reflect the percentage of male and female applicants for seats in the orchestra? Or perhaps higher than that to partially redress the centuries of historical bias that have led to the overrepresentation of men in orchestras? (Not to mention that this entire discussion assumes that gender is binary, which it isn’t.)
Machine learning can help us with these sorts of discussions because it requires us to instruct it in highly precise ways about what sort of outcomes we’ll find ethically acceptable. It gives us the tools to have these discussions — often arguments — in clearer and more productive ways.
Those tools include a vocabulary that arises from machine learning’s most common task: deciding which bin to put a given input into. If the input is a real-time image of a tomato on a conveyor belt in a spaghetti sauce factory, the bins might be labeled “Acceptable” or “Discard.” Each input will be assigned to bin with a confidence level attached: a 72% certainty that this tomato is edible, for example.
If sorting tomatoes is your system’s basic task, then you’re going to care how many tomatoes get sorted wrong: how many good tomatoes the ML is putting in the Discard pile, and how many bad tomatoes it’s putting in the Acceptable bin – mistaken approvals and missed opportunities. And because the assignments to bins are always based on a confidence level, ML gives its designers sliders to play with to adjust the outcomes to reflect different definitions of fairness.
For example, if it’s your tomato factory, you might care most about the overall accuracy of your new ML tomato sorting app. But a regulator may be more concerned about bad tomatoes making into the Approved bin than good tomatoes getting tossed into the Discard bin. Or, if you’re a sleazy tomato factory owner, you may be more upset by throwing out good tomatoes than by including some rotten tomatoes in your sauce.
ML requires us to be completely clear about what we want. If you’re worried about the bad tomatoes making it into your sauce, you’ll have to decide what percentage of bad tomatoes you (and your customers and probably your lawyers) can live with. You can control this percentage by adjusting the confidence level required to put a tomato into the Approved bin: do you want to set the threshold confidence level to 98% or lower it to just 60%? As you move that slider to the left or right, you’ll be consigning more good tomatoes to the Discard bin, or putting more bad tomatoes into the Approved bin.
In ML’s parlance, the overlooked good tomatoes sitting in the Discard bin are false negatives, and bad tomatoes put into the Approved bin are false positives.
These terms become useful when we talk about processes like sorting loan applications into Approved or Rejected bins. (For the purpose of this hypothetical, we’re ignoring any regulations governing loan approval processes.) Let’s say 30% of the applicants are women, but only 10% of the applications in the Approved bin come from women. But instead of looking at the percentage of approvals that go to women, or the percentage of men and women who default on their loans, perhaps we should be looking at whether the percentage of false positives in the Rejected Women bin is higher than the percentage of false positives in the Rejected Men bin.
The types of fairness we’ve discussed here, and more, have also been given precise definitions by researchers in the ML field, with names like “Demographic Parity,” “Predictive Rate Parity,” and “Counterfactual Fairness.” Having them available when talking through these issues with experts can make those discussions go more easily, with more comprehension on all sides of the argument. They don’t tell us what type of fairness to adopt in any situation, but they make it easier for us to have productive arguments about the question.
This is true at a higher level of abstraction as well, for we get to decide what counts as success for an ML system. For example, we could train our ML loan application sorter to optimize itself for the highest profit for our business. Or for the highest revenues. Or for the maximum number of customers. We could even decide for reasons of economic justice that we want to provide some loans to poorer people, rather than always going for the richest people around. Our ML system should enable us to judge the risk, to adjust the percentage of lower income people we want in the Approved bin, or to set a minimum profitability level for the loans we make.
ML also makes it clear that we can’t always, or even usually, optimize our outcomes for every value we may hold. For example, the loan company may find — in this hypothetical — that admitting more lower-income applicants into the Approved bin affects the percentage of women in that bin. It’s conceivable that you can’t simultaneously optimize the system for both. In such a case, you may well want to find another value you’re willing to modify in order to create outcomes fairer to both low income folks and women. Perhaps if you increase your company’s risk by an acceptable amount, you can accomplish both goals. Machine learning systems give us the levers to make such adjustments and to anticipate their results.
As we look at higher levels of abstraction — from using sliders to adjust the mixes in the bins, to questions about optimizing possibly inconsistent values — ML is teaching us that fairness is not simple but complex, and that it is not an absolute but a matter of trade-offs.
The decisions that ML’s helpless literalness requires from us can naturally lead to discussions that sound less like high-minded arguments over morality — or jargon-laden arguments over technology — and more like political arguments among people with different values: Great tomato sauce, or cheap sauce that maximizes our profit? Increase the percentage of female musicians in the orchestra or maintain the current configuration of instruments? Grant loans to lower income folks but perhaps lower the percentage of women in the mix?
If machine learning raises these questions with a new precision, gives us a vocabulary for talking about them, and lets us try out adjustments to see the best ways to optimize the system for the values we care about, then that is a step forward. And if machine learning leads us to talk about remedies to unfair situations in terms of the values we care about, ready to make realistic compromises, then that too is not a bad model for many moral arguments.
Credit: Google News