Proving Something With One Month’s Counting

One week, it seems, isn’t enough to tell the difference conclusively between the first bidder on Contestants Row having a 25 percent chance of winning — winning one out of four times — or a 17 percent chance of winning — winning one out of six times. But we’re not limited to watching just the one week of The Price Is Right, at least in principle. Some more episodes might help us, and we can test how many episodes are needed to be confident that we can tell the difference. I won’t be clever about this. I have a tool — Octave — which makes it very easy to figure out whether it’s plausible for something which happens 1/4 of the time to turn up only 1/6 of the time in a set number of attempts, and I’ll just keep trying larger numbers of attempts until I’m satisfied. Sometimes the easiest way to solve a problem is to keep trying numbers until something works.

In two weeks (or any ten episodes, really, as talked about above), with 60 items up for bids, a 25 percent chance of winning suggests the first bidder should win 15 times. A 17 percent chance of winning would be a touch over 10 wins. The chance of 10 or fewer successes out of 60 attempts, with a 25 percent chance of success each time, is about 8.6 percent, still none too compelling.

Here we might turn to despair: 6,000 episodes — about 35 years of production — weren’t enough to give perfectly unambiguous answers about whether there were fewer clean sweeps than we expected. There were too few at the 5 percent significance level, but not too few at the 1 percent significance level. Do we really expect to do better with only 60 shows?

Sure we do. This may be more convincing than trying to prove it rigorously: With the clean sweeps, we were trying to tell the difference between something which happened once every 1000 attempts versus something happening once every 6000 attempts. With only 6000 attempts in total, we didn’t have many examples. One or two occurrences more or less completely wipes out the difference. This time, we are trying to tell the difference between something happening once every six times and something happening once every four times. We can see enough occurrences that one or two flukes can’t obscure that difference.

With three weeks of sampling, 90 items up for bids — sadly, more than the CBS online archive keeps, so I can’t spend an afternoon watching episodes and making my tally — the chance of 15 or fewer wins (as opposed to the 22.5 expected) comes to 3.9 percent. Three weeks might let us distinguish between a 1/4 chance of the first bidder winning and a 1/6 chance of the first bidder winning, at least to a 5 percent significance level.

On four weeks, 120 items up for bids, the difference between the 25 percent probability’s 30 wins for the first bidder and the 17 percent probability’s 20 wins could be distinguished with a probability of about 1.9 percent. The sampling needed is looking more practical.

Five weeks of episodes, with 150 items up for bids, would give us an expected 37.5 wins for the first bidder if that seat should win 1/4 of the time, and 22.5 wins if the first bidder should win 1/6 of the time. The chance of 22.5 or fewer wins with the first bidder having that 1/4 chance is a mere 0.96 percent. With five weeks we could expect to say whether the first bidder has a chance closer to 25 or to 17 percent at the five percent and maybe even the one percent level of significance.

I should mention what I haven’t demonstrated: suppose the chance of the first seat winning is actually 1/5, or 20 percent. Then these five weeks of episodes won’t tell the difference between that and 1/4, or 25 percent; nor will they tell the difference between that and 1/6, or 17 percent. To tell the difference between smaller differences, I need more episodes, more data. That’s probably not surprising. It takes more evidence to tell apart two things that are very alike.

But here’s something neat that I have demonstrated: if we watched the items up for bids carefully for five weeks, tallying how often the first bidder wins, we could reasonably expect to say whether the first bidder wins about 1/6 or about 1/4 of the time. With the assumptions that every item up for bids is an independent event, making the next first bidder no more or less likely to win, we could conclude the chance of any one episode having all six wins from the same seat was either 1/1000 or 1/6000, and therefore, whether over 35 years of hourlong episodes of The Price Is Right, a clean sweep be expected once or should be expected six times.