Online symptom checkers are inaccurate


[Read the post]


well duh. even doctors mess up though at less of a rate if they are familiar with your personal medical history. it is a guessing game of well they have non specific symptoms ABCandD, that matches up with conditions XYandZ but could be a lot of things and based on what I know most likely X, but can’t completely rule out Z, lets try for X first and see if that works.


Huh, so I probably don’t have testicular cancer and an ovarian cyst?
It’s probably Lupus.


It gives WebMD a 51% success rate in correct diagnoses after 3 questions. That’s pretty darned good if you think about it – these aren’t complicated heuristics they’re using, just simple decision trees.

For the sake of comparison it might be interesting to list the success rates of actual MDs.


Which company manufactures the best keyboard with a button to instantly create specific keyboard button images?


When will the Rob Beschizza button be available on the iPad keyboard setup?


What I’d be curious to know is how much of the inaccuracy is down to an actually flawed decision tree, how much is down to the fact that parts of the decision tree can’t really be included in an online quiz(you can ask the user if their throat is sore; but the ‘does lab indicate strep?’ step is harder to do at home), and how much is down to users either inaccurately characterizing their symptoms or focusing more on symptoms that are more visible/painful rather than subtler but potentially more useful ones.

A flawed decision tree would be the most tractable problem, in principle, though potentially quite a slog in detail.

Having to omit steps you can’t self-report at home is less tractable; but you could presumably improve the situation by telling the user that you are at a point where more data are needed; and what needs to be clarified.

Weaknesses in user self-report would probably be the least helpful outcome; but the most interesting; both from the perspective of learning more about human cognitive biases and (ideally) in being able to craft better questions to get the data you actually need without ‘leading’ the user, simply confirming their preexisting assumptions, or inadvertently biasing your results. That is probably a fairly hairy problem; but not one unknown in polling, questionnaires, and testing(though sometimes one used specifically for better-targeted evil, rather than better accuracy).


Is he worth robbing?



I for one would prefer the keys to have little e-ink displays. They could be made like RFID/NFC tags - a little chip with a coil, attached to a display. E-ink takes only a little energy to change and none to stay. The keys made this way would then need just an unique ID each (to address), and a coil or a few coils under the keyboard to power and command them when change is needed.


Friend of mine is a medical librarian at Loyola. She says it’s scary how often students and doctors use google. She’s had to run classes on how to more accurately use google, if they decide to.


Don’t forget, Google is only as good as the person using it. If you are getting all your med information from and not cross-checking it with other more reliable sources, then yes, you’re probably going to mis-diagnose yourself frequently. I have self-diagnosed a broken toe and swollen glands recently, mainly to determine whether a trip to the dr. was necessary. Incidentally, I didn’t get significantly better info from the clinic when I went there about the sore throat and swollen glands; they were able to rule out strep, but determining if it was mono (the next option on the decision tree) would have required a blood test and mono has no treatment. I just wanted to find out if I was contagious, and for how long.


Weirdly enough googling my symptoms probably saved life. I have Graves’ disease, but presented with puking and weight loss not Marty Feldman eyes. None of my docs thought to take a simple thyroid test until I googled it and graves came up as most likely. By that time I was down to 106 pounds of emaciated near death.


Mine was pretty accurate, it said that I had eyeball-covered puppies and pagodas growing from my extremities, which was surprisingly close.


Having previously visited the ER for a friends broken toe who got told ‘yep it is broken, want some pain killers?’ when I whacked a tree root quite hard with my foot while playing paintball I just said yeah I know what the ER doc will say so left my boots on and got on with the afternoon. Hurt like hell for awhile though.


I’d be curious to see a study on the accuracy rates of doctors when it comes to different populations. They say that it’s “believed” that doctors have an 85% to 90% success rate (by who? what terrible weasel words!) but I wouldn’t be surprised if doctors ended up being less reliable than symptom checks for some groups.


I assume @shaddack has made his own x-ray machine to check things like that.


I got the tube and the scintillator plates. The high voltage part is on the todo list.



Ayup. I’m currently on my second case of cracked ribs. Even now that I have Medicaid there’s nothing modern medicine can do except wrap my chest in a big Ace bandage, which I’m capable of doing myself.