Friday, February 26, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Data Science

Model Transparency and the Complication of Model-Based vs Model-Free Learning

April 15, 2020
in Data Science
Model Transparency and the Complication of Model-Based vs Model-Free Learning
585
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

Summary:  High stakes models like those that allocate scarce resources to competing hospitals are headline news.  New thinking contrasting model-based versus model-free learning are emerging to describe new conditions we must consider before building or evaluating those models.

 

You might also like

How Machine Learning Discretely Assists Data Scientists

A Plethora of Machine Learning Articles: Part 1

AI Chatbot Platforms: The Best in the Market and Why to Consider

At this point we’re all familiar with the conversation around model transparency.  Whether that’s driven by the common sense desire for understanding or regulation driven as in finance it seems like a fairly straightforward goal. 

In the context of supervised model building where almost all of us live bias in a model is assumed to be the result of a dataset that doesn’t truly represent the whole population.  Occasionally we might come across an example where the decision making process represented by the model is flawed but this is rare.

We wrote recently of the dangers of using models where the impact on an individual or a group might have life-changing implications.  This is not simply deciding who gets the promotion or the special price.  This is about who gets bail or which hospitals receive how many masks and respirators.

The good news is that most of us will probably never be in the position of having to build or defend a model with such wide ranging implications.  The bad news is that the better we get at model building and the more advanced data science becomes, the greater will be the desire to ‘let the machine decide’.

Our upbringing in this modern era still causes us to think that machines don’t make mistakes.  They are built to perform their function, and unlike humans will perform that flawlessly each and every time.  The complication is that we are being lulled into the belief that AI is just another machine that will always give the right answer.

And it should in fact always give the same (or a consistent) answer.  But will we all believe the answer is ‘right’?  Especially when the model is allocating scarce resources.

 

Planning for Moral AGI

Fortunately there are folks thinking about this.  This paper: “Building Machines That Learn and Think about Morality” by Christopher Burr and Geoff Keeling is a good place to start.

The context of this is planning for AGI, Artificial General Intelligence.  That world is still divided between those trying to make computers learn and think like humans and those who believe the same results can be had more simply by allowing the computer to discover certain heuristic rules that explain most behavior.

AGI is many years off but like self-driving cars it’s most likely to arrive in stages from augmentation through full autonomy.  Burr and Keeling are attempting to identify some questions and approaches to researching one of the more knotty aspects of AGI.  That is, will we agree that the answers that our AGI entity provides will be good?  In this case good needs to be understood as qualitative as in moral or ethical.

The reason this is relevant now is that we have entered the age where we are calling on models to guide decisions that have strong moral or ethical implications.  By the way, ‘moral’ and ‘ethical’ are both defined as describing “right behavior” which immediately suggests a subjective component.

 

Why Should Data Scientists Care

From a data science perspective the aspect that caught my attention was the distinction between model-based versus model-free learning.  These phrases are beginning to pop up in our literature so we’re using this topic to shed a little light here. 

These terms derive from reinforcement learning and represent two competing schools of practice.  A model-based RL system starts with a detailed understanding of its environment, thereby its limits and capabilities.  A model-free RL system simply allows the system to conduct trial-and-error actions until a solution is found.  The limits of a full model understanding aren’t judged to be necessary; the RL will figure it out eventually.

In the field of human learning these two terms also represent the two originalist schools of thought about how humans learn.  The model-based school believes the human infant comes equipped with ‘startup software’ that rapidly (much more rapidly than today’s RL) allows them to organize experiences of the world into successful behaviors and transfer learning between dissimilar circumstances.

The model-free school says that no model is required.  That the infant learns through simple heuristics and those are guided by physical abilities and limitations.  Like, if you only have two hands then don’t try to find a solution that requires three.  Or more simply, gravity pulls down.

That might all seem too abstract for the average data scientist, but here’s where it gets interesting.  The two schools of human learning are gradually converging on an understanding that both types, model-based and model-free act together with one type sometimes more dominant than the other.

 

Back to Model Building

If you are asked to build or evaluate a ‘high stakes model’ here are some considerations that Burr and Keeling say you should keep in mind.

Model-based systems seem superior at the outset but the requirement to constantly adapt to a changing environment places the system in a loop of sensing (gaining information about the environment), modeling (building the richly reconstructive representation), planning (course of action to reach the goal) and action.  The SMPA loop is computationally intensive and results in either a time cost or an accuracy cost that need to balance against how fast the environment is changing.

The model-free systems based on simpler heuristics are faster to train in a changing environment but may not transfer learning well outside of a narrow domain.  However, a model-free “moral agent is not required to internalize the norms of society in order to ensure their behavior meets certain moral standards, and can potentially make do with a simplified model of the world (or maybe even a set of well-tuned heuristics) when certain institutions act as regulative constraints.”

What is needed is a mechanism that interprets or negotiates between the two approaches.  This mechanism has not yet been defined but the authors point to the need for the two models to operate in parallel (presumably where resources and time allow).  This is the challenge for the data scientist model builder.

 

Back to Bias and Transparency

The larger question for transparency in these high-impact situations is recognizing the limitations of each approach and being prepared to use examples of why the model’s recommendation was reached and by extension, why we should believe it is the ‘right action’.

The authors summarize the current state of conflict between these two approaches.  The way we actually arrive at moral decisions frequently has more to do with heuristics than an ideal model.  For example, we are much more likely to act to repair or challenge an unfair act if the person impacted is close to us rather than on the other side of the world.  Yet in reality the moral/ethical decision should require the right action in both cases.  So designing an AI that relies mostly on model-free heuristics may cause it to consistently make suboptimal moral decisions.

Also, systems that emphasize model-free heuristics are less likely to reach acceptable solutions when the circumstances are novel.

However, model-based ‘ideal reasoner’ systems that don’t recognize the imperfect heuristic methods we actually use in moral decision making may appear biased to many.  For example an ‘in group/out group’ bias is easy to spot in many such decisions.  A universally consistent solution may seem biased against the ‘in group’, although it is actually eliminating bias against the ‘out group’.

When evaluating moral decision making, we rely heavily on examples for explanations of why a decision was reached.  So it goes largely without saying that the higher the stakes of the model, the more transparency is required. 

The unsolved challenge that we are addressing here is how future data scientists can build these dual-personality models and balance their outcomes so that the greatest possible number of people agree that the result is “right action”.

 

 

Other articles by Bill Vorhies

 

About the author:  Bill is Contributing Editor for Data Science Central.  Bill is also President & Chief Data Scientist at Data-Magnum and has practiced as a data scientist since 2001.  His articles have been read more than 2.1 million times.

[email protected] or [email protected]


Credit: Data Science Central By: William Vorhies

Previous Post

Google's TensorFlow Lite Model Maker adapts state-of-the-art models for on-device AI

Next Post

How MSSPs Can Overcome Coronavirus Quarantine Challenges

Related Posts

How Machine Learning Discretely Assists Data Scientists
Data Science

How Machine Learning Discretely Assists Data Scientists

February 24, 2021
A Plethora of Machine Learning Articles: Part 1
Data Science

A Plethora of Machine Learning Articles: Part 1

February 24, 2021
What are Data Pipelines ?
Data Science

AI Chatbot Platforms: The Best in the Market and Why to Consider

February 24, 2021
Modernizing Data Dashboards. – Data Science Central
Data Science

Modernizing Data Dashboards. – Data Science Central

February 24, 2021
4 ways Cryptocurrency is Benefiting the Fintech Industry
Data Science

4 ways Cryptocurrency is Benefiting the Fintech Industry

February 23, 2021
Next Post
How MSSPs Can Overcome Coronavirus Quarantine Challenges

How MSSPs Can Overcome Coronavirus Quarantine Challenges

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

Asimov’s Three Laws Of Robotics And AI Autonomous Cars 
Artificial Intelligence

Asimov’s Three Laws Of Robotics And AI Autonomous Cars 

February 26, 2021
Something’s Fishy — New Funding To Tackle Illegal Activities At Sea Using Machine Learning And Data Analytics
Machine Learning

Something’s Fishy — New Funding To Tackle Illegal Activities At Sea Using Machine Learning And Data Analytics

February 26, 2021
Role of Image Annotation in Applying Machine Learning for Precision Agriculture | by ANOLYTICS
Neural Networks

Role of Image Annotation in Applying Machine Learning for Precision Agriculture | by ANOLYTICS

February 26, 2021
60+ free martech sessions. The agenda is live!
Digital Marketing

60+ free martech sessions. The agenda is live!

February 26, 2021
SolarWinds cybersecurity spending tops $3 million in Q4, sees $20 million to $25 million in 2021
Internet Security

SolarWinds cybersecurity spending tops $3 million in Q4, sees $20 million to $25 million in 2021

February 26, 2021
Chinese Hackers Using Firefox Extension to Spy On Tibetan Organizations
Internet Privacy

Chinese Hackers Using Firefox Extension to Spy On Tibetan Organizations

February 25, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • Asimov’s Three Laws Of Robotics And AI Autonomous Cars  February 26, 2021
  • Something’s Fishy — New Funding To Tackle Illegal Activities At Sea Using Machine Learning And Data Analytics February 26, 2021
  • Role of Image Annotation in Applying Machine Learning for Precision Agriculture | by ANOLYTICS February 26, 2021
  • 60+ free martech sessions. The agenda is live! February 26, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates