
Simpson's Paradox - mmaia
http://vudlab.com/simpsons/
======
tlb
The Omitted Variable Problem is part of my mental framework that causes me to
not believe most epidemiological studies, especially ones that confirm a
popular belief.

The almost universally omitted variable is health-consciousness. Some people
are health-conscious and some aren't. People who are health-conscious do a
whole bunch of things, some of which help (like exercise, sleep well, eat
moderately). They also do things that are widely believed to be good for you,
like eating broccoli.

So if you do a study, you'll find that people who eat lots of broccoli are
healthier. You'll be able to confirm pretty much any widely believed health
folk wisdom, unless it's something quite harmful, as long as you omit health-
consciousness as a variable.

~~~
rthomas6
Surely some studies account for this? If you look for people who do X and
people who don't and just analyze their lives, yes this problem is likely to
exist. But if you take two randomized samples of a the population and say to
group A, "do X," and to group B "don't do X," you have an effective control
group. At least I think so. Don't some dietary studies even provide the
participants with custom food regimens to try to eliminate extra dietary
differences between study participants?

~~~
defen
You're describing a randomized controlled trial and not an epidemiological
study.

~~~
dbecker
You are mistakenly confounding that epidemiological and observational.

Epidemiology studies can be randomized or observational.

~~~
defen
Interesting...what would be an example of a randomized epidemiological study?

------
tel
Simpson's Paradox is scary if you've not seen it before, but you should not
stop here and instead proceed _immediately_ on to omitted variable bias and
the conversation here
([http://normaldeviate.wordpress.com/2013/06/20/simpsons-
parad...](http://normaldeviate.wordpress.com/2013/06/20/simpsons-paradox-
explained/)).

In short, Simpson's paradox occurs because probability distributions and
causal claims _are distinct things_ which behave differently. It's nothing
more than a particular, insidious example of correlation not implying
causation. It's perfectly possible for a probability distribution to have a
"contradictory" shape, but perfectly impossible for logical statements about
the world to be contradictory.

The resolution is that you shouldn't let your probability distributions turn
into logical statements without analyzing your causal assumptions. This will
lead you to whether or not excluding a variable is omitted variable bias (and
whether including an improper one will lead to included variable bias, which
is rarely recognized).

------
teamonkey
The point of an interactive illustration like this is that it should make it
more intuitive and easier to understand than the core concept, not more
confusing.

What is the meaning of the green and purple lines on the graph, why do they
have different gradients and why can't I adjust them? Why does the Simpson's
Paradox apply sometimes and not others? Why are there so many bars and donut
charts? What does the gray circle around the donuts mean? Information
overload.

~~~
publicfig
I feel as thought both graphs with purple and green lines are accurately
explained. Were you trying to skim through the article or were you still left
confused after reading it? The top one is explained in the accompanying text
and the bottom one is explained with a labeled x and y axis and relies on the
information provided above.

~~~
teamonkey
If the article describes the subject adequately, what is the point of the
graphic? I found the interactive graphic more confusing than the description.

Compare and contrast to the clarity of their central limit theorem
demonstation [1] vs its wikipedia page [2].

[1] [http://blog.vctr.me/posts/central-limit-
theorem.html](http://blog.vctr.me/posts/central-limit-theorem.html)

[2]
[http://en.wikipedia.org/wiki/Central_limit_theorem](http://en.wikipedia.org/wiki/Central_limit_theorem)

~~~
publicfig
By that, I meant that the article described what the graph was representing,
not that the article described the information presented adequately. Also, I
absolutely disagree with the assumption you made that graphs shouldn't be
present if the information is presented in the article. It's good to provide
visual demonstrations when applicable in order to help clarify subjects.

------
sp332
Reminds me of Anscombe's Quartet: four datasets that have the same mean,
variance, correlation, and linear regression, but are really very different.
[http://en.wikipedia.org/wiki/Anscombe%27s_quartet](http://en.wikipedia.org/wiki/Anscombe%27s_quartet)

~~~
lewis500
maybe we'll do that one next!

------
jasonwatkinspdx
Judea Pearl has formalized a resolution to Simpson's Paradox:

[http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.34....](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.34.8955&rep=rep1&type=pdf)

An oversimplification of his idea would be to say that given assumptions about
the causal independence of variables, it becomes clear which way you should
group the data. Although it's not always possible to make these independence
assumptions, _much_ of the time they are obvious and uncontroversial.

Taking the Berkeley gender bias case as an example: We know it's possible that
biological gender influences which department graduates apply to, but that
it's impossible that the department a graduate applies to influences their
biological gender. This fact alone tells us that we need to look at the data
by department rather than in aggregate, resolving the paradox.

~~~
gweinberg
Well, it certainly makes more sense to look at the by department data rather
than the aggregate data. But I would say that the fact that we see such large
disparities in which departments are applied to is already enough to refute
the idea that male and female applicants are pretty much the same, and that
therefore it would be pretty reckless to conclude any sex discrimination based
on the difference in acceptance rates.

~~~
tel
And Pearl's point is that the "real world" logic you just applied is both
important and not statistical—you need to augment your analysis with these
causal assumptions in order to translate probability into meaningful causal
statements.

------
alexPetrov
Reading up on Simpson's Paradox again made me realize something: women
retrieving custody more often than men appears to me to be a perfect example
of Simpson's Paradox. Overall, women get custody more often than men, this is
true, but if you consider only the cases where men actually asked for or
attempted to retrieve custody, this is no longer true[0].

[0]:
[http://www.villainouscompany.com/vcblog/archives/2012/04/chi...](http://www.villainouscompany.com/vcblog/archives/2012/04/child_supportcu.html)

~~~
Tichy
How to interpret that, though? Seems likely to me that who asks for custody
would be influenced by their chances of receiving it. How many of those cases
are "contested custody"?

Then that article places the claim "Additional evidence, however, indicates
that women may be less able to afford the lawyers and experts needed in
contested custody cases (see “Family Law Overview”) and that, in contested
cases, different and stricter standards are applied to mothers." without
providing any data. So it ends up being a propaganda piece, which makes it not
very trustworthy.

------
wunderlust
FWIW, Simpson's paradox is a "veridical" paradox, not a "vertical" paradox as
noted in the article. Apparently veridical isn't yet accepted by spelling
checkers. (As I'm writing, Chrome offers the single suggestion "vertical" for
my "veridical".)

------
dsego
For me this was far better explained in the following text (also posted on HN
not so long ago): [http://www.michaelnielsen.org/ddi/if-correlation-doesnt-
impl...](http://www.michaelnielsen.org/ddi/if-correlation-doesnt-imply-
causation-then-what-does/)

------
trains
It's nice visually but the text-based information could be more thorough -
which would result in less links at the bottom.The more info contained on the
page the better!

------
tensafefrogs
Anyone interested in this stuff should definitely watch the lectures from the
Stats 110 class from Harvard that's up on iTunes U (and perhaps other places).

Lecture 6 talks about this paradox and if I recall correctly he might even
talk about this exact case:

[https://itunes.apple.com/us/course/statistics-110-probabilit...](https://itunes.apple.com/us/course/statistics-110-probability/id502492375)

------
maxk42
That is some sexy javascript.

~~~
ZoF
My first thought as well

------
martimoose
I had a hard time understanding the last graphic, because I was reading on an
iPad, and the sliders don't work. I thought it was all static, and couldn't
match the numbers in the graph with the previously mentioned numbers, as the
defaults of the sliders do not give a simpson's paradox.

------
cfontes
Really interesting text, thanks.

But I think this last statement

"or Texas schools to waste money copying Wisconsin."

was meant to be the other way around

~~~
yen223
I may have read it wrong, but I think the statement is correct. Texas
minorities outperformed Wisconsin's, but the stats made it look like
Wisconsin's minorities did better.

~~~
penrod
To elaborate: Black students in Texas outperform black students in Wisconsin,
hispanic students in Texas outperform hispanic students in Wisconsin, and
white students in Texas outperform white students in Wisconsin.

Overall test scores are higher in Wisconsin only because white students in
both states are the highest scoring group, and Wisconsin schools have a higher
proportion of white students than Texas schools.

To the extend that any of this can be copied, it would probably be better for
Wisconsin to improve the test scores of its ethnic groups to Texan levels,
rather than for Texas to emulate Wisconsin's lilly-whiteness.

------
xerophtye
Those interactive charts and graphs!! (My first encounter with vudlab and am
loving it!)

