Simpson’s paradox in practice

What each thing is

1. Success rate: how many cases went well, divided by the total number of cases. 9 successes in 10 surgeries is 90%.
2. Total: the result with all cases put together, without splitting by type.
3. Group: a slice of the data separated by some characteristic. Here the groups are the simple and the complicated surgeries.
4. Weighted average: an average where each group weighs according to how many cases it has. Each doctor’s total is a weighted average of their groups.
5. Simpson’s paradox: when a difference between two sides appears, disappears or flips once you split the data into groups. "Paradox" here means it looks like a contradiction, while the arithmetic is right. The name comes from the statistician E. H. Simpson, who wrote about the effect in 1951.
6. Bias: unequal treatment between people or groups.
7. Spreadsheet: a cell is each little box; cell A1 is in column A and row 1, at the top left corner. A formula is a calculation that starts with an equals sign (=) and that the spreadsheet works out by itself.

The example of the two doctors

Across all surgeries, doctor B has a 74.5% success rate and doctor A has 35.5%. Picking B looks obvious.

Splitting by type of surgery:

• On the simple ones, A has 90% (9 out of 10) and B has 80% (80 out of 100).
• On the complicated ones, A has 30% (30 out of 100) and B has 20% (2 out of 10).

A wins on both types and loses on the total. That happens because, of the 110 complicated surgeries, 100 went to A, and of the 110 simple ones, 100 went to B. Each doctor’s total is the average of their own cases, and 100 of A’s 110 cases are the hard ones. That is why their average drops.

The total, step by step:

• Doctor A: (9 + 30) successes in (10 + 100) surgeries = 39 out of 110 = 35.5%.
• Doctor B: (80 + 2) successes in (100 + 10) surgeries = 82 out of 110 = 74.5%.

The ready-made spreadsheet

1. Open a new spreadsheet in Excel or Google Sheets.
2. Click cell A1 and paste the data below. Each line becomes a row, and each column lands in its own cell:

Médico	Tipo	Casos	Sucessos
A	Simples	10	9
A	Complicadas	100	30
B	Simples	100	80
B	Complicadas	10	2

If everything landed in a single cell, select column A and use Data, Text to Columns (in Excel) or Data, Split text to columns (in Google Sheets), choosing tab as the separator. Or type the data by hand, one value per cell.

3. In cell E1, write Taxa (rate). In cell E2, paste the formula below. It divides the successes by the cases on that row:

=D2/C2

4. Carry the formula down: click E2, hold the little square at the bottom right corner of the cell and drag it to E5. Or copy E2 and paste it into E3, E4 and E5. Then select E2 to E5 and click the percentage button (%) on the toolbar.
5. Now each doctor’s total. In cell G1, write Médico (doctor). In G2, write the letter A (the doctor’s name). In G3, write the letter B. In H1, write Taxa no total (rate on the total). In cell H2, paste the formula below. It adds up that doctor’s successes and divides by the sum of their cases:

=SOMASE($A$2:$A$5;G2;$D$2:$D$5)/SOMASE($A$2:$A$5;G2;$C$2:$C$5)

6. Carry the H2 formula into H3, the same way as in step 4, and format both as a percentage. To see one decimal place (35.5% instead of 35%), click the increase decimal places button once.
7. Compare. In column E, A wins on the simple ones (90% against 80%) and on the complicated ones (30% against 20%). In column H, A loses on the total (35.5% against 74.5%). That is the paradox.

The formula in step 5 is written for a spreadsheet in Portuguese. In an English spreadsheet the function is SUMIF and the separators are commas: =SUMIF($A$2:$A$5,G2,$D$2:$D$5)/SUMIF($A$2:$A$5,G2,$C$2:$C$5). If that one gives an error, use the SOMASE version in the block above.

Test it yourself

Give doctor B the same mix of cases as A and watch the paradox disappear. The rate of each type stays the same; only the quantity changes:

• On B’s row with simple surgeries, change C4 to 10 and D4 to 8 (still 80%).
• On B’s row with complicated surgeries, change C5 to 100 and D5 to 20 (still 20%).

Now B’s total drops to 25.5% (28 out of 110), and A, with 35.5%, comes out ahead. Each doctor’s rate on each type of surgery did not change. The mix of cases did. Afterwards, put the original values back (C4 100, D4 80, C5 10, D5 2) to keep the example.

It happened for real: Berkeley

A 1975 study about graduate admissions at the University of California, Berkeley, in the United States, showed the same effect. On the total, men were admitted more often than women. But looking at each department separately, the authors found no bias against women. The difference came from where each person applied: women applied more to departments with a lower admission rate, that is, where it was harder to get in. The admission rate is the number of admitted people divided by the number of applicants.

The study is by Bickel, Hammel and O’Connell, published in the journal Science.

Summary to keep

1. A total can hide what happens inside each group.
2. The total is an average that weighs each group by how many cases it has.
3. In the example, doctor A took more hard cases and their average dropped, even with a higher rate on each type.
4. Splitting helps you see, but the decision depends on understanding the cause.

Sources

Stanford Encyclopedia of Philosophy, Simpson's Paradox. Bickel, Hammel and O’Connell (1975), Sex Bias in Graduate Admissions: Data from Berkeley, Science 187(4175), pp. 398 to 404.

Educational and independent content.