There’s a particular frustration I feel when I read this sort of discussion. In an interview with Noam Chomsky I once listened to, I recall him saying something along the lines of, “There’s no point in telling truth to power. They already know the truth, and they don’t care. They’re not the people you need to talk to.”
I think that there are two real goals in education reform: crushing any potential resistance from working class students (which is to say, most students in public schools, and a lot in charter and private schools), and crushing any potential resistance from teachers. The latter is significant and I think too often overlooked: teachers are a large body of educated, unionized workers, in relatively good communication with each other.
So I think it’s worth noticing how much of the absurdity of testing and educational metrics is aimed at discrediting teachers, who are blamed for failing to achieve impossible goals with useless tools.
The Bill Gates foundation had a similar incident to the kidney study. After studies revealed that the best performing schools were small schools, they spent a huge amount of money helping school boards divide large schools into multiple smaller ones.
A few years later someone realized this was simply a statistical artifact that small schools will be have a higher variance in their scores due to a smaller population. After a while the Gates foundation stopped supporting the practice and apparently eventually acknowledged the math mistake.
My wife, a science teacher, wanted to analyze the standardized test scores for the entire 6th grade where she taught (given 3 times a year). She wanted to see the results of an unofficial experiment she had been running for years. Because of lazy counselors, she was the person who made class assignments. For years she had been manipulating class assignments to get the worst students assigned to her. At one time she had 26 students with IEP’s in one class. 11 more than are allowed to be in a class by law. About 15-20 in her other classes. She had several average and gifted student too.
So you’d think her kids would learn slower than those of the other teachers, right? Dead wrong. Her kids progressed an average of 2 years in her class. Progression was higher the lower the student initial score was. Lowest progression was with her top kids, but at least they progressed.
Progression averaged 1 year or less for all of the other teachers. Worse, the advanced students declined.
Comparing students with similar initial scores between classes, my wife’s student’s still doubled the improvement of similar students in other classes.
The conclusion is that the proportion of slow learners does not necessarily correlate with lower average improvement (and can be a negative correlation). At a gross level, it is possible to tell if a teacher is very effective, ineffective, or average.
Other conclusions were that standardized testing can be used to find areas of weak understanding and work on those areas with students. Not by teaching to the test, but by doing hands on projects and open ended questions that result in depth and breadth of knowledge that make students flexible to handle any kind of testing.
Testing is a tool. Used incorrectly, it’s worthless. Used correctly, it can be powerful. Unfortunately, it’s almost always used incorrectly.
Just as a followup: When my wife presented the results (she still had to hide that she manipulated class assignments), she was reassigned to teach advanced science. She taught 6th, 7th and 8th grade (now just 7th and 8th). She still uses test results to adjust her teaching. Over the course of 3 years of teaching the same students, students like those that were declining before, her students go up “7” years. So it’s not a fluke limited to slower students.
I use 7 in quotes because they don’t get 7 years smarter. But their scores do improve the amount expected over 7 years. Though being able to hold a conversation with an 8th grader about the relative merits of different nuclear power types, or the relative merits of genetic manipulation, you get the impression they can hold their own against many college freshmen.
