The statistical problems with sample size

Imagine for a moment that some agency, in an attempt to establish the average Body Mass Index for Americans, used players in the NFL as the original “sample”. After carefully establishing the range and the mean for BMI in the entire league, would it be reasonable to look at the rest of the American population and declare that the human species was shrinking

The absurdity of the proposal is staggering.

However, just such a scenario has been played out for years with the Scholastic Achievement Test when newspapers decry our nation’s “plummeting” SAT scores.

The validity of this standardized test for anything other than determining the socio-economic status of those sitting for the exam might be the topic of a future article. Cost is listed as one of the major impediments to sitting for the exam.

Today, however, alarmist headlines are inexcusable when, for decades, the SAT was normed against and administered to America’s academic elite, the upper quartile of high school students.

We are moving, albeit at a glacially slow pace, toward a society that discourages the concept of disposable children. As we come closer to furnishing all children the opportunity to compete against their peers, we are inexorably increasing the sample size.

Statisticians will confirm that the “average” score will invariably fall as the sample size increases. Except in the case of a totally randomized sample, it is foolish to expect otherwise. The SAT has yet to be normed on a random sample across the spectrum of all American students.

Yet, such realities do not hinder ideologues from using such erroneous data analysis as political fodder to question the effectiveness of our public schools.

Diane Ravitch clearly illustrates in Reign of Error how student achievement and academic performance have been steadily improving for decades according to a host of of alternative measures. More students know more about more subjects, and at earlier ages, than ever before in our history.

Moving forward will require that irrelevant soundbites and erroneous headlines not drive the debate around educational egalitarianism.


[This Commentary appeared originally in the now defunct Prince George’s Gazette on 10/23/2014.] 

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s