Bluegrass Beacon: Kentuckians should beware of education data ‘snow jobs’

By JIM WATERS
Bluegrass Institute for Public Policy Solutions

Sunday, Aug. 25, 2012, 1 p.m. – Law enforcement officials again are cautioning elderly Kentuckians about financial scams, warning: “if it sounds too good to be true, it probably is.”

A similar disclaimer could also be applied to the continual inundation of rosy claims concerning the performance of our young people in Kentucky’s public education system.

The Kentucky Department of Education wasted no time in praising a new Harvard University report claiming Kentucky students have made some of the best progress in the nation during the past 20 years on federal reading, math and science tests.

But do the conclusions of this study, derived from the National Assessment of Educational Progress (NAEP), the “Nation’s Report Card,” deserve the rush by state education officials to gush over its findings with a press release only one day after its release?

I doubt it, especially considering the failure we’ve seen in the commonwealth to close black-white student achievement gaps and significantly move the needle when it comes to high school graduation and college-remediation rates.

At the very least, a dose of healthy skepticism is needed regarding claims that Kentucky is fifth-best among 41 states from a report whose stated purpose is to “determine the extent of the United States’ progress toward closure of the international education gap.”

The more I find out about the Harvard study, the more the glow of Kentucky’s performance loses its luster. I’m now uncertain, at best.

• · The study completely ignores the fact that Kentucky schools exclude large numbers of students with learning disabilities from NAEP testing.

Since these students tend to struggle academically and generally produce lower test scores, how can researchers not consider the negative impact such exclusions could have on overall educational performance?

The Harvard report claims that Kentucky eighth-graders are tops in the nation when it comes to reading. Beware of such claims – especially considering the Bluegrass State led the nation by excluding a full 7 percent (compared to an average nationwide exclusion rate of only 3 percent) of its entire raw sample of eighth-graders – including both disabled and non-disabled students –which the NAEP wanted to test.

• · The Harvard report only offers a state-to-state comparison of students’ overall average scores with no consideration of those states’ different racial demographics.

This especially matters for Kentucky, where a whopping 84 percent of our students are white compared with, say, a state like California where around three in four students are non-white and many don’t speak English as their primary language.

Lump all students together and, sure, Kentucky shows a great performance. But the artificial advantage evaporates quickly once you break down the scores by race.

An independent analysis of changes in NAEP fourth-grade math scores for each reporting state’s black students between 1992 and 2011 shows that Kentucky’s blacks, who comprise around 10 percent of the Bluegrass State’s public school students, got left behind – way behind. The Harvard report is silent on that issue.

Because it gives little consideration to demographics, it also fails to report that, according to the NAEP scores, only 13 percent of Kentucky black eighth-graders are proficient readers. There’s also no mention that only 39 percent of the commonwealth’s white eighth-graders reached proficiency on the 2011 NAEP reading test.

After 22 years of education reform in Kentucky, what kind of progress do these facts show?

All of this didn’t stop KDE spokesperson Lisa Gross from telling a Glasgow Daily Times reporter that she didn’t “see how demographic changes affect the validity of the report.”

Of course, a data scam – like any other kind of snow job – has a much better chance of succeeding in an atmosphere of denial.

Jim Waters is acting president of the Bluegrass Institute, Kentucky’s free-market think tank. Reach him at jwaters@freedomkentucky.com. Read previously published columns at www.freedomkentucky.org/bluegrassbeacon.