Statistical thinking II

There are some key statistical concepts that everybody should be familiar with. Here is a non-comprehensive list of some of my favourites.

There are few concepts more fundamental and understandable than averages. They are calculated by adding up a set of numbers (say annual wages of people in a population) and then dividing by the number of numbers you added up (to give an average wage). The calculation is easily done but the interpretation is slightly more tricky. We often fall into the trap of thinking that the average means the usual or the typical. This is not necessarily the case. The average wage of a population comprising 50% rich people and 50% poor people would lie in between the two but represent nobody in the population. In that situation, the average would actually be an extreme and provide information about nobody. You might say this is a pathological example, but this happens regularly. Cities are full of rich areas that border poor areas. See here for an illustration from Brazil’s favelas. This happens in real life.

Another way averages are misinterpreted is the notion that being below average is a bad thing. This may be the case but the construction of the average (in a normally distributed population) means that half of the population will lie below the average. Everyone wants to be above average, but by definition half will be below. There are numerous examples of people misunderstanding averages but one of my favourites is the exchange below featuring Michael Gove giving evidence to an education committee in January 2012 

Gove

There is a clear related problem with ranking. Every year we produce tables of the best universities to help students choose the best institutions to which to apply. Just recently the REF results were published and while the official report presented the universities in alphabetical order, everyone else ranked them. Even if we all agree that the methods used to form these rankings are valid and appropriate, then there is still the problem that differences in rankings may not represent a real difference. Consider a situation where we asked a group of people to flip a coin 100 times each and count the number of heads. We would expect that each person to count about 50 heads but we would be surprised if everyone got exactly 50 heads: we’d expect some to get slightly more than 50, some slightly fewer. None of them would be performing better than anyone else, but it would still be possible to rank them according to how “good” they are at getting heads.

This clearly leads on to another related problem. Regression to the mean. Suppose we uncritically chose the top ten “best” coin flippers to flip coins for us, hoping that they will continue to flip more heads than tails. Since these flippers performed well entirely by chance, in the future they will likely perform no better than anyone else and will score closer to the average of 50 heads than they did first time around. They will appear to have gotten worse, even though in reality nothing has changed. This happens in reverse too. Say for example, we ranked hospitals by performance on some metric and provided extra funds for the bottom 10%. Chances are that the bottom 10% will comprise a combination of poor performers and those that were simply unlucky. Those that are unlucky would probably have improved next year by chance anyway, but a naïve interpretation would attribute the improvement to the funds received. Similarly, if a clinical trial of a hypertension drug measured blood pressure in potential participants and only included those with really, really high blood pressures chances are good that some of those people were just particularly stressed or had a particularly poor diet the preceding week. These individuals will improve regardless of what is done to them, but an uncontrolled analysis might attribute the improvement in blood pressure to the drug. Or say, a government gets voted out because the economy is in the worst state it has been in for a number of years. Lo and behold, a new administration almost immediately identifies the green shoots of recovery and claims the credit.

Statistical reasoning has a role to play in our everyday decisions whether we understand the rules behind them or not. It is better to be able to reason our way through these things.

 

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s