Originally published at: Video by teenager on TikTok disrupts 4,600 survey studies | Boing Boing
…
4 Likes
Demographics screening department: because why should young women get a voice.
16 Likes
Sort of like polls done during election periods that only sample people who answer a landline telephone with no caller ID between the hours of 2PM and 7PM.
24 Likes
This begs the question, how skewed are the demographics of the set of people who fill out online surveys for money? I sure don’t do that.
16 Likes
Exactly! and furthermore, all web polls are fundamental sampling errors (or “survey bias”) in that you have to have visited the web page, very likely for another reason, to take the poll. (“Our poll conducted on our Guns-n-Ammo page indicate strong support for no laws restricting guns!”) For instance, (i confess) i’m fond of DailyKos, which conducts ad-hoc polls all the time, you can guess how all those polls trend just given that much information. …accurate polling in a highly connected world is all but impossible.
9 Likes
Or like how all network television was mostly terrible for 50 years because Nielsen rating boxes were placed in a handful of midwestern white peoples’ houses.
12 Likes
A smart researcher might use surveys like this as a way to get a quick and cheap temperature check on attitudes they want to explore further, in the same way that a wildlife biologist might go on daily observational hikes to get the lay of the land and look out for signs that their systematic models are incomplete.
But I don’t have a lot of confidence that all researchers are smart or responsible.
8 Likes
8 Likes
I have a lot of confidence that many of them are not at all smart. They rarely include the option of a “don’t know” or “not applicable” box on their 1-10 scoring ranges and so constantly get skewed data when I choose 5s all the way through (or sometimes 1s). They don’t even realise this IS skewing their data.
6 Likes
Ha! In my 20s I was broke as could be in the big city so I spent a good amount of time trying to find side hustles. One of those ratings companies promised me $25/mo to wear a pager-like device that would pick up some sort of silent encoding in recorded music and report it for ratings purposes. It took three months to pay me and the cheque bounced.
Another one of those companies promised me money and grocery gift cards if I scanned the bar codes on my grocery shopping. After a couple of months of getting paid in envelopes full of coupons for products I didn’t want at grocery stores I didn’t shop at, I just stopped scanning. Six months later I got a lawyers’ letter demanding I return the scanner or they’d sue.
17 Likes
Doesn’t everybody lie in surveys anyway?
5 Likes
Do you want an honest answer?
7 Likes
Based on the research that has been done on what people say vs what they Google for, yep.
2 Likes
I stop reading every psychology paper that makes the rounds in popular media as soon as I get to the point where they say the data is self-reported. Between that and rampant p-hacking, no wonder the entire field has a massive replication crisis.
7 Likes
Why should any woman get a voice?
Gods, all the misogyny on this planet is so fucking exhausting.
2 Likes
Many do. And the company paying for a survey might be marketing to a particular group and only want to survey that group. When I was a uni student, a statistics lecturer suggested that if we ever saw someone doing a survey at a shopping centre we should repeatedly walk past them until they asked us to complete the survey, so we’d see first-hand how surveys were conducted. It was surprising how many terminated the survey when I gave my occupation as uni student. I hear that these days, with paid online surveys, people will create multiple accounts with different ages, so that if a survey knocks them back based on their age, they can flip to a different account with a different age and see if that gets them paid.
Sometimes we’d see someone carrying out a survey on campus. If they didn’t look like a scientologist, chances were that it was a student in a 2nd year STAT unit about survey design, and they were field testing their own survey. And apparently 3rd year STAT students who had already done the unit would try to break their survey. They’d send in one person to do the survey and find out what the questions were. Then they’d pick a memorable question and agree on a particularly silly answer to give and they’d all go and do the survey giving that answer. So the survey might show that a particularly large proportion of uni students were members of the Judean People’s Front.
3 Likes
Heh, yeah, been there…
1 Like
This topic was automatically closed after 5 days. New replies are no longer allowed.
