I found this slide deck on the web and thought it hilarious, because if there’s one thing you see a lot of in public education politics, it’s standardized test scores and statistics.
I guess the reason I found it so funny is because, well, it’s actually trying to be funny, but also because it gave me one of those, “Oh yeah, DUH!” epiphanies. Like when someone explains something simple that you’ve always known, but didn’t grok at a fundamental level until someone else spoke it and the words actually hit your ears. Hey, it happens!
In this case, it was the slides on how biases in your data corrupt any statistical analysis.
More to the point, if you sample a large population but the sample isn’t sufficiently randomized or representative of the total, you can’t run stats on the sample and say anything at all about the total population.
In the case of a “choice” school like BCS, where parents choose to attend and the socio-economic and demographic profile of the BCS student body is obviously different (and diverging) from that of LASD, it makes no sense whatsoever to compare standardized test scores for the charter sample to those of the total district. To compare the two without adjusting for the bias in the sample would just be… bad science.
But we hear all the time that this or that charter school outperforms nearby district schools, so where’s the disconnect between the theory and the practice? Well, ironically enough, I say the disconnect is math illiteracy among ordinary parents and voters.
You see, if voters (and county school board members) really understood that “choice” programs by definition enroll non-random samples of students, maybe they’d be in a better position to call BS on bad science claims like “highest achieving charter school in CA” and “scored higher than the district on standardized tests again.”
If your “education reform” rests on math illiteracy, you shouldn’t be too proud.
Ironically, the better job we do educating everyday folks on problems with this kind of bad science, the less likely voters will be to fall for such statistical lies, cheats and manipulations being passed off for real progress in public education via “reforms.”
The smarter voters get, the less they’ll fall for this “school choice” hoax.