The 3 Rules of Statistics (Starting with the Super Bowl)

Photo Credit: Paul Grand

If you’ve bought into any of the recent Super Bowl hype, you’re aware an unlikely competitor-for-attention has been making headlines: Commercial Real Estate.

Roger Staubach, hall of fame quarterback, real estate tycoon, and presenter of this year’s Lombardi Trophy is now famously quoted, “You can mark my word: the Packers will prevail.”

And it’s all based on a deterministic statistical analysis of office-building vacancy rates!  Nearly two thirds of the time, NFL teams based in cities with the highest percentage of empty office space have won the Super Bowl.

If you’re thinking, wait, something’s fishy, first take a look at some headlines by major news periodicals:

The Wall Street Journal: Super Bowl Winners Born of Hard Work, Office Vacancy

Star-Telegram: Maybe it evens out at Super Bowl: Picking winner by looking for loser

Bizmology: Super Bowl winner predicted by analyzing commercial real estate vacancy rates

Are you any more convinced?

Staubach goes on to say, “I have nothing but the utmost respect and admiration for the Steelers and the city of Pittsburgh, but the numbers don’t lie. The Lombardi Trophy is going back to Title Town USA.”

But the numbers do lie, maybe not in their analysis, but certainly in their conclusion.  When you cut out the fat, the hype, and the commentary, you’re left with the argument: Office Vacancy rates have a significant effect on the outcome of the Super Bowl.

It’s plain wrong.  A football game is comprised of an infinite number of momentary decisions subject to countless deterministic factors (preparation, athleticism, wind, sound, focus, timing, sweat, match-up, missed cues, spotted-balls, foot placement, chance, even the very unlikely effect of office vacancy rates).

Fortunately, as these periodicals are merely playing into the hype of Roger Staubach’s analysis, and do not conclude that a real significant relationship between vacancy rates and Super Bowl winners exists, fans aren’t running to dump their life-savings on Roger Staubach’s assertation, “The Packers will prevail.”

Roger Staubach’s quirky analysis underlines a very serious disease of statistics.  Statistically speaking, there’s always a winner.  Outliers exist in most statistical data; it could have been bird migration, tea consumption, or UPS daily delivery statistics that better hindsight-predicted Super Bowl winners in the ten years of Staubach’s study.  It’s an absolute certainty a city characteristic exists that predicts the last ten Super Bowl victors with one hundred percent accuracy.

Those hind-sight predictions most often have zero indication of future success.

I often analyze risk and probability through Monte Carlo Simulation, where I run thousands, tens-of-thousands, even a hundred thousand iterations of possible outcomes, and the structure of the statistical output is always the same: a sinking tail and a rising head.  In other words, there are always extreme data points; but since the boundaries are self-created, I can grasp their existence, and may pass them off as near-entirely insignificant.

Our reality is comprised of an immeasurable number of iterations; the boundaries aren’t always clear, and we’re rarely privy to a significant understanding.  But since data-mining makes it possible to isolate the outliers, and statistical misrepresentation gives the outliers credence, we find ourselves believing the very fundamental statistical trait forming the outlier, pure chance.

A statistic should never stand alone, and must be accompanied by logic and experimental repeatability; the same way an astronomer theorizes about a star then actively looks for the star-foe to his conclusion.  Statistics without reference are often more dangerous than plain uncertainty.

So without further ado, follow these three simple rules of statistics and never be fooled again.

1)      Never use statistics (unless you’ve got no choice)

2)      Never trust anybody that uses statistics (unless you’ve got no choice)

And most important…

3)      Statistics are clues, not answers, never treat statistics as facts, merely starting points

Join us by subscribing for free email updates and follow us on Twitter.