Solving Problems with Math, AI and a Data Science Team Working with Business Managers
Over the past five years, Wally Lo Faro has built the data science capabilities and team at Mastercard Operations and Technology. He has applied techniques from areas of mathematics, machine learning, information retrieval, text analysis and statistics. He has designed systems for highly-scalable string matching, data quality monitoring, process automation, rewards marketing, personal recommendation engines and marketing analysis. He holds three patents and has over 20 applications pending. He was previously a professor of mathematics at the University of Wisconsin, Stevens Point and at Boise State University. He received his PhD in mathematics from the University of Iowa in 1998. He recently took some time to speak to AI Trends Editor John P. Desmond.
(Ed. Note: Wally Lo Faro will be speaking at the 2019 AI World Conference & Expo in Boston, being held Oct. 23-25.)
AI Trends: Thank you for taking the time to talk to the readers of AI Trends today. First, could you describe your responsibilities at Mastercard?
Wally Lo Faro: I lead a team of data scientists in the Operations and Technology division. Our division builds the platforms that powers our business.
My team has three primary responsibilities. The first, probably not surprising, is internal consulting. We build applications to solve business problems for various teams across the company.
The second responsibility is testing and conducting proof of concepts (POCs) with various technology vendors that are AI related.
And the third focus we have is research and development. This ranges from solving a business problem that nobody ever asked us to solve, but maybe should have. Or it might take the form of speaking at a university computer science or math department. It also takes the form of publishing peer-reviewed academic papers.
So your work has a bit of an academic slant.
Yes. It’s applied research. We’re not publishing a new method for training a deep neural network. It’s more about problem solving.
What is the role of AI in your work and what AI technologies are being employed at Mastercard?
Overall for Mastercard, our focus is on the application of AI – and we look for opportunities to apply cutting edge AI research to everything we do. That means using AI to manage and keep our network secure and as a tool to automate tasks to free capacity. It’s not about bigger systems of flashier bells, it’s about working smarter.
To me, machine learning problems are optimization problems. I have a mathematician’s view of it. The technology tools we use across the organization are standard. We have a Hadoop cluster. It’s a Cloudera shop with Spark on it. We write applications in Python and R primarily and work within a Jupyter Notebook setting.
[Ed. Note: the Jupyter Notebook is an open-source web application that allows users to create and share documents that contain live code, equations, visualizations and narrative text.]
We also have some home-grown AI technology that we use particularly around fraud modeling that was developed by Brighterion, a company that is now part of Mastercard. They created technology for doing both supervised and unsupervised learning for training models and we’ve incorporated that into our environment too.
[Ed. Note: Mastercard acquired Brighterion in July 2017 for an undisclosed sum. Brighterion had been using AI and machine learning to provide real-time intelligence from a range of data sources. The products were being applied in anti-money laundering (AML), real-time cross-channel fraud prevention, onboarding and risk monitoring, data breach detection, marketing, healthcare and biotech.]
What are the challenges around using AI at Mastercard?
At Mastercard, we do not have one monolithic data science team. We have about 10 teams distributed around the company. Domains that have enough need, fraud detection being one of them, have dedicated teams of data scientists.
My team helps departments that don’t have enough need to support their own team. The challenge is in that distribution. It can make it hard to ensure that we’re not working in silos and duplicating efforts. That’s our biggest challenge right now.
We’re starting to adopt some methods and bring forth a guild model for how we can organize ourselves, from a work point of view. We are moving towards some standardized documentation, code sharing, technical reviews, things like that. To me, it’s like the American Bar Association model. They don’t tell lawyers who to represent. We don’t have a guild in Mastercard telling data scientists what problems to work on. What the Bar Association does do is set professional standards, offer continuing education, things like that, to make sure the quality of the overall practice of law stays high.
I will say, at Mastercard we are constantly pushing boundaries and barriers and we are free to innovate and that’s exactly what we did around the idea of concept drift.
[Ed Note: In predictive analytics and machine learning, the concept drift means that the statistical properties of the target variable, which the model is trying to predict, change over time in unforeseen ways. As a result, the predictions become less accurate as time passes.]
How do you ensure data is not biased?
The entire profession is working on this. I’m comfortable telling you what my team’s done to address it. I just mentioned our work around concept drift so can use that example. Concept drift is a well-known challenge amongst technologists. Several common remedies exist to address both. However, our solution is different from what’s currently being done in the market because it employs a separate, independent machine learning model that operates autonomously.
Essentially, we are addressing certain elements of AI bias by having AI keep tabs on AI. This gives our systems the ability to avoid false positive results based on changes to the data at scale and ultimately helps us provide the security that our customers and partners expect from Mastercard.
Do you have any black box AI models and if so, can you explain their results?
My team has not ventured far into the world of deep learning as it trends particularly to things like music, video, and images. This isn’t the typical kind of Mastercard data.
My understanding is that the further along a self-learning model goes, the more complex the model gets. And that might make it more difficult to explain why it recommended what it did.
I know that some data scientists are building models so complex, particularly neural networks, that it’s almost impossible to explain why a certain piece of data is scored the way it did. So they’ll build another model to predict why the original model did what it did. I think that’s kind of funny; I hope I don’t get to that point.
How well do data scientists speak the language of business? Or how effectively do data scientists at Mastercard communicate with business managers?
It depends on the data scientists for sure. It’s such an important skill for any technologist, and I don’t typically give data scientists high marks on it because we tend to be more on the math side of things.
However, this is a skill I look for when I am hiring—the ability to effectively communicate. I feel lucky that I have more experience from my former life as a college professor. I’m used to talking to people about things that might be hard for them and that they don’t really want to take the time to learn. (That’s called being a math professor.) It’s also what it’s like to talk to an executive at times. They don’t have much time and they’re not a mathematician, so you really must be an effective communicator to make your time count. It’s important for me to develop my team on this important skill.
In the last decade, what I’ve seen, at least at Mastercard, is leadership wanting us to have a seat at the table when it comes time to make decisions. And when we can speak effectively and to the business needs, it’s beneficial for all.
Are there new job titles that have resulted or come about in the last five years or so at Mastercard as a result of all the work going on with AI?
Yes, in fact, I wrote the job descriptions. I wrote the job family originally about five years ago. It’s been a bit refined since then but there’s now a data scientist job family that wasn’t before. Creating it, my philosophy was to be hard about it to ensure we were attracting and retaining the best talent.
Can you give me some examples of new titles?
I am vice president, data scientists. That’s the top of this. Then it goes to principal data scientist, lead data scientist, and then data scientists one, two and three.
You could not stay away from the numbers.
Right. Ask me again in a year and it will be different. Right now, a fresh data science PhD would start as a lead data scientist, for context.
Are you able to find the qualified people you need to staff your efforts in AI?
So far, personally, I’ve been lucky enough to build a really strong team. I know the market is tight. I have had luck because of my previous career; I leverage my connections from academia as well as Mastercard’s university partnerships. It’s one of the reasons we go speak at the Summer Research Experience for undergraduates at Washington University. And we go to the University of Illinois, and many other local colleges, to talk and maintain those relationships.
It’s also a reason I have the team do academic research. It’s good for Mastercard, but it’s also a good way to retain people. By devoting time for my team to write papers and present them at is also a good way to retain them.
Also, Mastercard is committed to bringing in and retaining top tech talent—not just leaving it on the hiring managers and talent acquisition. They offer tons of upskilling opportunities—in fact we just helped create a learning and development path for AI.
They also make it a place you want to work—because you’re working with smart people, solving tough challenges that you know have a meaningful impact on the world.
Do you have any advice for either students or early career people in how to move into the AI field and create value for themselves? Any do’s and don’ts along those lines?
There are three ways to participate in this profession. The obvious one is a practitioner, who builds algorithms that go into production. For that track, you need to get some serious education, which a master’s degree or PhD in a relevant field.
Another way to participate, which is still a critical, are roles like a program management leadership role. One of my team members worked on Wall Street, then worked in the data side of Mastercard. She got interested and skilled in that area and now leads and manages our team’s projects.
The last way, if you’re an engineer, is to be what I call machine learning engineer. This is someone who specializes in deploying and maintaining AI applications in a production system. That is non-trivial. It is not like deploying an analytics reporting cube and putting Tableau on top of it. Our code blurs the line between code and data. There is a lot of opportunity for good software engineers with an interest in data to get some training and make this career shift towards that specialized role of half data scientist and half software engineer. There’s absolutely a need for this skill set and there will continue to be.
Learn more at Wally Lo Faro’s LinkedIn page.