I did this job for a couple of months.
It wasn’t nearly as bad or as hard as the article makes out, but perhaps that’s because I was working on Japanese content. Presumably, the more applicants there are, the more competition for available work, and the stricter the evaluations. Anyway, I thought the pay was reasonable for the work involved, although it was pretty boring - I blitzed through the evaluations, alternating between evaluating and exercising in order to stay awake.
It was only ever an interim job for me though, and I wouldn’t want to do it long-term. Some of the testing was pretty arbitrary. For instance, I remember one task: A user searched for “J-pop boy bands”, and three videos were search results. Watching the videos they were all K-pop boy bands. I figured the results were “bad”, but not “worthless”, because there is crossover in popularity between J-pop and K-pop and boy bands are boy bands. So I gave each result 0.5~1 out of 5. The evaluation told me I’d made a serious mistake and should have rated each video as 0 out of 5. It’s weird trying to apply “objective” evaluation to seemingly subjective criteria.
