To someone who went through school without a single standardized test and then went on to study computational linguistics this sounds really scary. Do people really grade texts according to some standardized measure of complexity?
2 Likes
pretty much, yes. it used to be that the tests were graded by human graders according to a semi-strict rubric which varied by district. these rubrics were typically a combination of factors involving grammar (correctness and complexity), vocabulary, and persuasiveness.
now, however, we have trained NLP-based statistical classifiers according to these past grades. so, yes, increasingly, these tests are graded according to a âstandardizedâ measure of complexity, albeit a very complicated one defined only by the trained classifier, which is always (or at least almost always) proprietary.
itâs pretty awful if you bother thinking about it. as a demonstration, an MIT professor engineered a trivial text generator which maximizes these statistical tests while making no sense whatsoever. 
here is a link: http://www.bostonglobe.com/opinion/2014/03/13/the-man-who-killed-sat-essay/L9v3dbPXewKq8oAvOUqONM/story.html
btw, how the heck did you go through school without a standardized test? home-schooling?
1 Like
Having built the occasional NLP-based statistical classifier I am seriously unconvinced. Sure, you may be able to derive a crude classifier that works ok much of the time, but quite often you will screw someone who wrote a good text that doesnât fit some hidden pattern in the original âgoodâ sample. Itâs just irresponsible in that context.
2 Likes
German schooling. All our tests were just created and graded by our own teachers for one class.
uh, yeah, did you finish reading my reply? the MIT professor maxed it out just by spewing âgoodâ words he found in the promotional material. :-/
it also stands to reason that if someone writes a very good essay while somehow avoiding whatever signals the system was trained to detect, that they will get a poor score.
i agree that itâs irresponsible, but thatâs never stopped America before.
2 Likes
Yes, but as I understood it, that was with human readers. If thatâs the standard, then perhaps machines really arenât any worse, but that doesnât really make it less horrible.
this is my mistake. the SAT is operated by ETS, and ETS also offers an algorithmic essay scoring service; however, ETS does not algorithmically score the SAT.
algorithmic essay scoring is, however, in wide use.
you are correct. humans churning through a stack of essays with brain-dead metrics can do just as badly as a machine. this is a better article: http://www.nytimes.com/2012/04/23/education/robo-readers-used-to-grade-test-essays.html
1 Like
That is what I suspected⌠but I still canât quite wrap my mind around it.
I guess there is a cultural element in how poor people prioritize their spending, and I donât know what the prices in the American cell phone industry are.
And I have to admit I know little about âpovertyâ in the US - Iâm sure itâs worse than what âpovertyâ means in European high-tax countries, but on the other hand poverty is always a relative termâŚ
A 10 EUR/month entry-level cell phone plan is equivalent to cheap food for at most two days. Or rent for a small apartment for about half a day. Or 7.4 liters of gas (including Austrian tax). Or five trips on public transport (inside town).
How can people not afford this and still survive?
Or is a cellphone still considered a luxury item, while some other things are considered essential?
I think there is a time element here as well.
I am 80 and when I went to school teachers were responsible for setting the tests, marking them and then discussing with the students how they could improve.
The quality of the marking depended on the quality of the teachers which, in my particular case, very low.
I have heard about standardized testing and wondered about the results.
Then I read a book, Freakonomics where it suggested that some teachers altered the way they marked in order to show better results presumably because that would reflect well on them.
It could, perhaps, be argued, that standardized testing encourages cheating.
I do not know the answer. My education was very limited.
Gareth Powell
sorgai.com
They arenât crazy nanotech man high-tech or anything; but the history of pencils is actually littered with a surprising amount of tinkering to get them right (see what I refrained from doing there?). Lead hasâŚunfortunate effectsâŚon the developing literary genius, as well as giving a soft and relatively faint line. High quality natural graphite in sufficient bulk and purity was tapped out relatively early. The tinkering with carbon/ceramic mixtures that could be cheaply and repeatably produced with the appropriate compromise properties between snapping, smearing, tearing paper, etc. was pretty involved.
When Henry David Thoreau wasnât busy being a parasitic slacker and writing about self reliance he actually did some work in the field as part of the family businessâŚ
There are definitely people choosing between food and phone at the end of the month in the States. My dadâs phone is only active about half the time, on for a month or two, then off for a month or two. I put off getting a cell phone until after college and have had to consider dropping it when I was broke and bill pruning. The cell service I have now in the Netherlands is cheaper than what I was paying back in the States, and the food is way cheaper.
This topic was automatically closed after 5 days. New replies are no longer allowed.
