Friday, April 23, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Data Science

Simpson’s Paradox, the Bane of Clinical Trials

July 14, 2020
in Data Science
Simpson’s Paradox, the Bane of Clinical Trials
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

(This article is now a chapter of my github proto-book Bayesuvius)
Simpson’s paradox is a recurring nightmare for all statisticians overseeing a clinical trial for a medicine. It is possible that if they leave out a certain “confounding” variable from a study, the study’s conclusion on whether a medicine is effective or not, might be, without measuring that confounding variable, the opposite of what it would have been had that variable been measured. Statisticians have to enlist expert knowledge to assure themselves that no influential variables are left out.

Judea Pearl considers Simpson’s Paradox a fundamental problem which is greatly clarified by his theory of causal Bayesian networks (See references below).

You might also like

IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future

5 Ways Tech Is Making Insurance More Efficient

Strategies for a successful Voice of the Customer program

Here is a simple example of Simpson’s Paradox (Or Simpson’s curse, or Simpson’s weed).

Some patients of both male and female genders are given a medicine or a placebo in a double blind study. Some recover from their ailment and others don’t. Let

r= recovered? No=0, Yes=1

t= took medicine? No=0, Yes=1

g= gender? Female=0, Male=1

The situation can be modeled by the Bayesian Network (bnet) in Fig.1

Figure 1

For this bnet, one hasE_{underline{g}|t}

P(r,g,t)=P(r|g,t)P(g|t)P(t)

Therefore,
P(r=1|t)= sum_g P(r=1|t, g)P(g|t)=E_{underline{g}|t} P(r=1|t, underline{g})

whereE_{underline{g}|t} is a conditional expected value (a kind of weighted average).

Supposeq_0, q_1 are non-negative real numbers. For the vectorvec{q}=(q_0, q_1):

Define a positive outcome (or success orq_t increasing with t ifq_0 leq q_1.

Define a negative outcome (or failure orq_t decreasing with t ifq_0 geq q_1.

simpson-vecs

Figure 2

It is possible (see Fig. 2 for a graphical explanation of how) to find perverse cases in whichP(r=1|t, g=0) andP(r=1|t, g=1) increase with t butP(r=1|t) decreases with t. So it is possible to conclude that the medicine is a success for each of the two g populations considered separately, yet the medicine is a failure when both populations are “amalgamated”. The lesson is that a “trend reversal” is possible upon amalgamation. The sign of the outcomes is not necessarily preserved when we do a weighted average of typeE_{underline{g}|t}.E_{underline{g}|t} is an expected value on the “confounding” random variable g conditioned on the root random variable t.

simpson-b

Figure 3

Footnote: Sometimes the bnet Fig.3 is used to explain Simpson’s paradox instead of the bnet Fig.1. But those two bnets are equivalent because

P(g|t)P(t)=P(g,t)=P(t|g)P(g)

References


Credit: Data Science Central By: Robert R. Tucci

Previous Post

AI/Machine Learning Market – Revolutionary Trends 2025 – Cole of Duty

Next Post

Password managers: Is it OK to use your browser’s built-in password management tools?

Related Posts

IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future
Data Science

IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future

April 23, 2021
5 Ways Tech Is Making Insurance More Efficient
Data Science

5 Ways Tech Is Making Insurance More Efficient

April 23, 2021
Strategies for a successful Voice of the Customer program
Data Science

Strategies for a successful Voice of the Customer program

April 23, 2021
How Python Interpreter Works? – Data Science Central
Data Science

How Python Interpreter Works? – Data Science Central

April 23, 2021
What Does The Future Hold For the Companies Developing Mobile Apps
Data Science

What Does The Future Hold For the Companies Developing Mobile Apps

April 22, 2021
Next Post
Password managers: Is it OK to use your browser’s built-in password management tools?

Password managers: Is it OK to use your browser’s built-in password management tools?

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

Edge Computing, The Cloud, And AI Autonomous Vehicles 
Artificial Intelligence

Edge Computing, The Cloud, And AI Autonomous Vehicles 

April 23, 2021
Data Science and Machine-Learning Platforms Market 2020 | Latest Trends, Demand, Growth, Opportunities & Outlook Till 2027 | Top Key Players: SAS, Alteryx, IBM, RapidM – KSU
Machine Learning

Data Science and Machine-Learning Platforms Market 2020 | Latest Trends, Demand, Growth, Opportunities & Outlook Till 2027 | Top Key Players: SAS, Alteryx, IBM, RapidM – KSU

April 23, 2021
King Island connectivity upgrade to include 110km radio link across Bass Strait
Internet Security

King Island connectivity upgrade to include 110km radio link across Bass Strait

April 23, 2021
IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future
Data Science

IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future

April 23, 2021
Your Doctor’s Assistant is AI 
Artificial Intelligence

Your Doctor’s Assistant is AI 

April 23, 2021
Machine learning model generates realistic seismic waveforms
Machine Learning

Machine learning model generates realistic seismic waveforms

April 23, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • Edge Computing, The Cloud, And AI Autonomous Vehicles  April 23, 2021
  • Data Science and Machine-Learning Platforms Market 2020 | Latest Trends, Demand, Growth, Opportunities & Outlook Till 2027 | Top Key Players: SAS, Alteryx, IBM, RapidM – KSU April 23, 2021
  • King Island connectivity upgrade to include 110km radio link across Bass Strait April 23, 2021
  • IoT in Telecommunications: Challenges, Opportunities, Benefits & The Future April 23, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates