!['And ... My batting average of predicting the [unborn child's] sex is 97% accurate!'](https://nebusresearch.files.wordpress.com/2013/02/rex-morgan-7-february-2013.gif?w=840)
The comic strip Rex Morgan, MD, put up an interesting bit of nonsense in its current ridiculous story. (Rex and June are investigating a condo where nobody’s been paying rent; the residents haven’t because everyone living there is strippers who’re raising money for a cancer-stricken compatriot; the details are dopier, and much more slowly told, than this makes it sound.) But on the 7th this month it put up one of those things that caught me. Never mind the claim that Delores here (the cancer-stricken woman) puts up about being able to sense pregnancy. She claims she can predict the sex of the unborn child with 97 percent accuracy.
Is that plausible? Well, she may be just making the number up, since putting a decimal or a percentage into a number carries connotations of “only a fool would dare question me” similar to those of holding a clipboard and glaring at it while walking purposefully around. If she’s doing ordinary human-style rounding off, that could mean that she’s guessed five of six pregnancies correctly. I could believe a person thinking that makes her 97 percent accurate, but I wouldn’t be convinced by the claim and I doubt you would either.
So here’s a little recreational puzzle for you: how many pregnancies would Delores have to have predicted, and how many called accurately, for the claim of 97 percent accuracy to be hard to dismiss? How many until it isn’t clearly just luck or a small sample size?
nebusresearch 
My first concern would be selective memory: does she forget her failures more easily than her successes?
But assuming the “97%” figure is an accurate description of her record: failure rate of 3% (approx. 1/30) implies that she’s made at least 30-odd predictions. The chances of getting that success rate by luck on 30 predictions is very small: it works out at 31/2^30, or about 1 in 35 million.
Which is pretty small… but looking at it another way, if you took everybody in the USA and asked them to predict the sex of 30 babies, even if they all just used a coin-toss, you’d expect around ten people to get that 97% success rate just by luck. Expand to the rest of the world and there’d be 200 people with a record as good as Delores.
But we’ve only been considering the task of predicting a baby’s sex. There are many other things that people like to predict intuitively: lottery results, whether a patient will live or die, etc etc etc. Throw those into the mix, allow people to forget about the things that they weren’t able to predict, and you can find thousands of people with a 29/30 record on *something* just by chance.
(I don’t think detecting pregnancy or even a baby’s sex automatically falls into the realm of the supernatural – it’s conceivable* that there might be biochemical hints that somebody can pick up on. But I’d want a lot of convincing before I discard my null hypothesis here.)
*sorry
LikeLike
Well, considering my probability knowledge is tolerably rusty…
In a reduced, simplistic method, to claim 97% accuracy you’d need at least 34 pregnancies, of which 33 are predicted correctly and one incorrectly. With less pregnancies than that, either you have 100% accuracy, or you have greater than 3% inaccuracy.
…but that’s obviously just the start, and we’ll have much further to go here. :)
LikeLike
My probability theory is also rusty but I believe that with 34 pregnancies, your confidence interval on the average is about 17% (via √n/n) so there’s a lower bound of 80%. To get confidence within 1% you need √n/n < .01 so √n < .01n, or n > 10000.
So she’d have to have correctly predicted around 9700 pregnancies out of 10000, and that’s a LOT of babies.
LikeLike
(of course my probability theory is ridiculously rough and I’m sure I could work out a more accurate number using Bayes theory or something)
LikeLike
I meant to comment on this earlier and only just got around to it: the √n/n formula is not the right one here.
For a single binomial event with Pr(1)=p and Pr(0)=1-p, the variance is p(1-p).
In this case, if the true accuracy of her predictions is 97%, then variance for a single trial (counting success as ‘1’ and failure as ‘0’) is 0.97*0.03 = 0.03. For 34 trials, the variance in total # of successes is 34*0.03 = 0.99 and so the standard deviation in (total successes) is sqrt(0.99) or pretty close to 1.
(Easier way to get the same result: instead of counting successes, count failures, and since the probability of failure on any one trial is pretty small, you can approximate it as a Poisson distribution. In a Poisson distribution with expected value n, the variance is equal to n, so the standard deviation is sqrt(n). )
In this case, it’s easier to calculate confidence intervals by applying a binomial formula:
If the true expected success rate is p, then the chance of getting 34 successes from 34 trials is p^34, and the chance of getting 33 successes (i.e. 1 failure) is 34*p^33*(1-p)^1
Adding those two together and experimenting with possible values of p:
If p=0.8, then the probability of getting at least 33/34 successes is about 0.5%.
At p=0.85, that rises to 0.24%, and at 0.865 i.e. 86.5%, it rises to 5%.
So if we take a frequentist approach and use a cutoff of 0.05, then a success rate of 33/34 would lead us to conclude that Delores’ expected success rate (i.e. what we can expect to see in future trials) is at least 86.5%. A Bayesian approach might give a different and more reliable answer, but it requires a good estimate of the prior distribution function, which we don’t have.
LikeLike