Simpson’s Paradox, the Bane of Clinical Trials

(This article is now a chapter of my github proto-book Bayesuvius)
Simpson’s paradox is a recurring nightmare for all statisticians overseeing a clinical trial for a medicine. It is possible that if they leave out a certain “confounding” variable from a study, the study’s conclusion on whether a medicine is effective or not, might be, without measuring that confounding variable, the opposite of what it would have been had that variable been measured. Statisticians have to enlist expert knowledge to assure themselves that no influential variables are left out.

Judea Pearl considers Simpson’s Paradox a fundamental problem which is greatly clarified by his theory of causal Bayesian networks (See references below).

Data modernization: The key to tomorrow’s highly competitive insurance industry

The Essential Role of Automation in End-User Computing

Information Communication Technology Ensures a Safe Social Media Ecosphere

Here is a simple example of Simpson’s Paradox (Or Simpson’s curse, or Simpson’s weed).

Some patients of both male and female genders are given a medicine or a placebo in a double blind study. Some recover from their ailment and others don’t. Let

r= recovered? No=0, Yes=1

t= took medicine? No=0, Yes=1

g= gender? Female=0, Male=1

The situation can be modeled by the Bayesian Network (bnet) in Fig.1

Figure 1

For this bnet, one has $E_{underline{g}|t}$

$P(r,g,t)=P(r|g,t)P(g|t)P(t)$

Therefore,
$P(r=1|t)= sum_g P(r=1|t, g)P(g|t)=E_{underline{g}|t} P(r=1|t, underline{g})$

where $E_{underline{g}|t}$ is a conditional expected value (a kind of weighted average).

Suppose $q_0, q_1$ are non-negative real numbers. For the vector $vec{q}=(q_0, q_1)$ :

Define a positive outcome (or success or $q_t$ increasing with t if $q_0 leq q_1$ .

Define a negative outcome (or failure or $q_t$ decreasing with t if $q_0 geq q_1$ .

simpson-vecs

Figure 2

It is possible (see Fig. 2 for a graphical explanation of how) to find perverse cases in which $P(r=1|t, g=0)$ and $P(r=1|t, g=1)$ increase with t but $P(r=1|t)$ decreases with t. So it is possible to conclude that the medicine is a success for each of the two g populations considered separately, yet the medicine is a failure when both populations are “amalgamated”. The lesson is that a “trend reversal” is possible upon amalgamation. The sign of the outcomes is not necessarily preserved when we do a weighted average of type $E_{underline{g}|t}$ . $E_{underline{g}|t}$ is an expected value on the “confounding” random variable g conditioned on the root random variable t.