Dr. Watson, Please Report to the Health Care System

by Andrew McAfee on March 7, 2011

Physician, professor, and novelist Abraham Verghese wrote a recent op-ed in the New York Times in reaction to the news that IBM is working with the universities of Columbia and Maryland to adapt Watson, the Jeopardy! champion supercomputer, to the task of medical diagnosis. Verghese is perceptive and eloquent about the power of a potential ‘Dr. Watson:’

Watson might help me digest the sheer volume of data that is in the electronic medical record and might see trends in the data that speak of an impending disaster. And since Watson is constantly trolling the Web, he would perhaps bring to my attention a case report published the previous night in a Swedish journal describing a new interaction between two of the drugs my patient is taking.

Better still, if Watson could harness data from all the patients in our hospital and in every other hospital in America, we might be alerted to mini-epidemics taking shape. For example, Watson might recognize that the kidney failure in our patient is linked to kidney failure in a patient in Buffalo and another in San Antonio; all three patients, he might inform me, were taking a “natural” weight loss supplement that contained a Chinese herb, aristolochia, that has been associated with more than 100 cases of kidney failure.

Let me expand this scenario, and try to convey why we should all look forward to the day when Dr. Watson starts seeing patients.

Watson works by applying lots of algorithms to a huge body of data. For the Jeopardy! competitions, the data were hundreds of millions of documents on every conceivable topic. Dr. Watson would be much more specialized; its data would be a digital version of humankind’s medical knowledge. As Verghese points out, this knowledge would be constantly updated as new medical facts and findings come in. All of these data would be of varying levels of quality and reliability, of course, but this is not an insurmountable problem —  statistical algorithms are quite good at separating signal from noise.

When I went to Dr. Watson as a patient, I’d contribute my own data: the contents of my personal health record, any recent test results, and a description of my symptoms. Current speech recognition and natural language processing technologies are so good that I could give this description over the phone.

Dr. Watson would then go to work. It would approach the problem (“What condition(s) does Andy have?”) from many directions at once, turning loose an army of algorithms. Each of them would examine the data I’d supplied, compare it to previous cases and patterns detectable in the accumulated body of medical knowledge, and reach a diagnosis. To oversimplify a bit, the supercomputer would give my case to a host of non-identical virtual diagnosticians, each of which would return with its answer about what was wrong with me. The real-world equivalent of this would be to give my case to hundreds of differently-trained and -oriented doctors, then collect all their answers.

If a large majority of these answers were consistent — if most of the doctors thought I had the same condition — I’d have pretty high confidence in this consensus diagnosis. In just the same way, if most of Dr. Watson’s algorithms converge to the same answer, the computer has high confidence that it knows what’s wrong with me. As those of us who watched the Jeopardy! competition learned, when Watson was confident it was usually also correct.

I can also easily imagine Dr. Watson asking for more information: “Andy, have you traveled abroad in the past month? If so, where?” “Go get a blood test, please, and have the lab upload the results.” “Let’s do an MRI. I’ve scheduled you for one tomorrow”

Once it has all the additional information it thinks it needs, Dr. Watson can arrive at a diagnosis almost instantaneously. And it’s worth stressing the strengths of this diagnosis:

  • It’s based on all available medical knowledge. Human doctors can’t possibly hold this much information in their heads, or keep up it as it changes over time. Dr. Watson knows it all and never overlooks or forgets anything.
  • It’s accurate. If Dr. Watson is as good at medical questions as the current Watson is at game show questions, it will be an excellent diagnostician indeed.
  • It’s consistent. Given the same inputs, Dr. Watson will always output the same diagnosis. Inconsistency is a surprisingly large and common flaw among human medical professionals, even experienced ones. And Dr. Watson is always available and never annoyed, sick, nervous, hungover, upset, in the middle of a divorce, sleep-deprived, and so on.
  • It has very low marginal cost. It’ll be very expensive to build and train Dr. Watson, but once it’s up and running the cost of doing one more diagnosis with it is essentially zero, unless it orders tests.
  • It can be offered anywhere in the world. If a person has access to a computer or mobile phone, Dr. Watson is on call for them.

What’s more, Dr. Watson knows what it knows, and what it doesn’t know. If a case has it stumped — if it doesn’t have high confidence in its own diagnosis — it can call in the humans to take over. No pride or ego will get in the way of its doing so.

We find ourselves now at a truly amazing point in time. All the elements needed to build Dr. Watson are in place and pretty thoroughly tested. It’ll be a great challenge to turn Watson the world’s best Jeopardy! player into Dr. Watson the expert diagnostician, but geeks both medical and digital love a challenge. I predict they’ll succeed with this one.

If they do, wouldn’t you rather get diagnosed by Dr. Watson instead of a human physician? I’m not using this question as a rhetorical device; I’m asking it in all seriousness. In his op-ed, Verghese frets a good bit about the digitization of health care delivery, but I’m eager for technology to improve this inefficient, error-filled, and pre-industrial industry. I am blown away by the skill, knowledge, and dedication of just about all the health care workers I’ve come in contact with, but the adjectives in the previous sentence are all accurate.

If and when Dr. Watson gets as good at diagnosis as Watson is at Jeopardy! I want it as my primary care physician (even though I’m thrilled with my current PCP). I think I’ll get better medical care as a result.

Do you agree? If not, why not? Leave a comment, please, and let us know. I’ll discuss responses in my next post here.

  • http://twitter.com/paglandier PA Glandier

    Hi Andy,

    I agree the time is quite exciting and could be avalaible very soon ! Such a system could help all over the world and drastic reduce drastically the price to acces healthcare.
    However you don’t raise any doubt : Who would run such a system ? Private companies ? How can you ensure security ? How about responsibilities ?

    Anyway this looks really good !

  • http://karthickhariharan.blogspot.com Karthick Hariharan

    With all due respect for Dr.Watson, I am still a firm believer of Human intuition and incomprehensible ability of humans to solve even complex situations. Our Knowledge is certainly not limited to data. Of course, it can help us arrive at better conclusions. The veracity of the data available for Dr.Watson must be strongly checked.

    I strongly support Dr.Watson to be used for data analytics but certainly not in health care prognostications and personal diagnosis. For 99.99% diseases, medicines, tablets and treatments are not the cure. Do not make it worse with Dr.Watson. Let’s keep it human.

  • http://twitter.com/ainsleybraun Ainsley K. Braun

    I completely agree. With where the system is currently headed with the new health care bill allowing anyone to get health care, we either need a drastic increase in doctors or something like Watson to take away some of the patient burden. We currently don’t have enough doctors to support an immediate surgein demand.

  • Jbordeaux

    Couple of thoughts: I don’t believe Watson is actually “trolling the Web,” the version on Jeopardy worked from a fixed corpus, and I believe there is a good reason for this – one that presents a hurdle for anyone trying to help Watson ‘learn’ in real time.

    Also, to the point below, who would run such a system? Put another way, who would be accountable for bad associations? If Dr. Watson is assisting a human, that shows real promise. But the system that placed Toronto in the U.S. should not be our primary (or sole) anything…

  • http://twitter.com/humbertomoreira Humberto Moreira

    Professor,

    This is a very interesting scenario, but there are several forces working against public acceptance of this:

    First is the “gut” preference for human intuition even in the face of better statistical outcomes from machine decisions. We can see this exemplified very clearly in the analogous debate over pilot-free cockpits.

    http://www.longbets.org/4

    Even though cockpits are already highly automated and the technology to enable completely pilot-free commercial flights is close to fruition, there is a strong consumer aversion to this idea. The prospect of passengers frantically trying to enter a cockpit to reboot a frozen computer which is flying the aircraft into a mountain is too unnerving to a generation accustomed to seeing recurring “blue screens of death” on their own machines.

    A second limitation is that to codify Medical Watson would require casting a very direct light on some tough ethical choices. For example, let’s say Watson is running a diagnosis and decides there’s a 1% chance of a life-threatening ailment in a patient. Proper diagnosis would require an expensive test. Does Watson weigh the expected value of the reduction in lifespan/quality against the cost of the test? Who sets the parameters as to when the procedure is justified, and who gets to see those parameters?

    These difficult allocation decisions have always been made, but the current decision rights are spread over a number of actors, and at an individual level, doctors continue to have some discretion. Having a machine weigh those variables brings them to the forefront.

    The scenario of several non-identical virtual doctors is a really interesting one, and it sets up a final wrinkle. It’s well-known that in many cases these kind of “ensemble” machine learning methods that combine the results of several models (“doctors”), can produce better results than single models. However, imagine a scenario where 10 “doctors” voted against the diagnosis, and 2 “doctors” voted for it. It later turns out that the minority opinion was correct. On a purely objective level, the methods can be understood, but it’d be very hard to rationalize why the 2 “doctors” weren’t taken into account.

    Along these lines, there have been a number of studies on the positive effect of heartfelt apologies from doctors. In many cases, this can lead to good outcomes.

    http://www.nytimes.com/2008/05/18/us/18apology.html

    Not only would a be unable to apologize, but its bedside manner would be a daunting UI/UX challenge.

    Repeatedly, studies have shown how strongly doctors’ behavior and bedside manner impacts patients’ evaluation of their healthcare experience. In most cases, patients seem to be asking for more human behavior, not less.

    http://www.mayoclinicproceedings.com/content/81/3/338.full

    This doesn’t mean that machine learning won’t be a vital part of the healthcare industry – it already is, and as online health sites have proven, we love to pre-diagnose ourselves and would surely value improved data on the back end, but in terms of primary care physicians we’ll probably still prefer a human face for some time (even if the actual diagnosis is taking place on their iPad)…

  • http://www.juliuscampbell.com/datawhisperer Julius Campbell

    I’m almost with you on this one. I would rather not have Dr. Watson give me a diagnosis instead of a human doctor. I would rather get my diagnosis from a human doctor consulting with Dr. Watson. You have written before that decision makers place too much faith in intuition and I agree with you. Human intuition is amazing, but it is nearly impossible to apply it consistently. You have also written that the best decisions (using chess as an example) come from the combination of human plus computer teams:

    http://blogs.hbr.org/hbr/mcafee/2010/02/like-a-lot-of-people.html

    I look forward to getting a diagnosis from a human-Watson team instead of just Watson alone.

  • http://massimorossello.myopenid.com/ Massimo

    I think that Verghese has been more realistic than you, you went too far. You assume that about medical issues there is ONE truth, which is true for Jeopardy! questions by design, but not for human-related issues.
    So, finding correlations between hospital cases can be done, giving a diagnosis (except for the trivial ones) cannot.
    Dr. Watson will not replace Dr. House…

  • http://it.bride.md/ ragazze ucraine

    What an excellent blog! And since Watson is constantly trolling the Web, he would perhaps bring to my attention a case report published the previous night in a Swedish journal describing a new interaction between two of the drugs my patient is taking.

  • Jesse

    To assume that physical diagnosis is unnecessary for delivering healthcare in the USA is a huge mistake.  Yes, you dont need a physical exam for EVERY possible ailment, but you certainly need it for a good 80% of medical encounters.  To suggest that Watson can singlehandedly take over healthcare with no human involvement needed is a joke.

    Consider this scenario:

    Mother brings a 2 month old child with a history of vomiting.  2 possible diagnoses are gastroenteritis and pyloric stenosis.  Watson cant tell the difference between these two diagnoses without a physical exam., even if he has access to every medical database ever conceived.  Pyloric stenosis requires immediate surgery whereas gastroenteritis requires either IV fluids or watchful waiting, depending on severity of symptoms.  

    Another problem that Watson will have is that he cant triage very well.  There’s a lot to be said for a set of human eyes looking at a patient without doing anything else to determine how “sick” they are.  For example, a 12 month old child may have a fever which is no big deal.  But what if he’s lethargic?  That is a very different scenario that requires a lumbar puncture and admission to the hospital for meningitis.  Watson cant easily tell if the child is lethargic by just talking to the mother; he needs to “see” the patient to know.  My sense is that Watson would send every one of these kids to an ER/hospital if the mom uses the word “lethargic” to describe him which is total overkill.  A human looking at the child and deciding whether he’s really “lethargic” or not would avoid the massive ER costs that Watson would incur.

    In terms of Watson being able to piece together epidemiologic trends from across the country, I’m skeptical.  99% of this data is kept sequestered at the hospital/clinic level, its not available in any kind of massive database that Watson would have access to, and its certainly not on the internet.  So before you can even release Watson on this task, you’d have to patch together the myriad databases out there into one cohesive framework.

  • Dr. Rick

    Dr. Watson will inevitably suffer the the worst flaw any human doctor can have:  insensitivity to the non-verbal cues that distinguish between an organic disease state and a psychosomatic condition. Perhaps one-fourth or more of visits to PCPs (and even subspecialists) are totally or substantially psychic in origin. Dr. Watson will take the patient at his/her word, literally, and will mis-diagnose a sizable fraction of human conditions. Mrs. Smith will come in with headaches and fatigue and may be convinced she has “food allergies” and Dr. Watson can’t possibly perceive the stress in her descriptions and body language. Granted, some human doctors lack this ability too, but ALL Watsons will. No good physician takes the patient at his/her word all the time. At best, Watson could be a valued assistant.

  • http://thesoftshops.com/microsoft-office-2010.html Office 2010 pro

    I think so?health is more important than everything.

  • bookworm worm

    A wrong diagnosis compensation claim is something you are entitled to make if your physician has incorrectly misdiagnosed an illness or disease. 

  • Anonymous

    aa

  • stand up comedy

    contoh majas http://goo.gl/bjsrQZ

  • Anonymous

    Dr Watson has a few fundamental problems, which some commenters have alluded to:

    1) Dr Watson will always be sub-optimal at accounting for patient bias. It will fail to input data into the system correctly

    2) Dr Watson cannot prescribe

    I’m reading Tyler Cowen’s Average is Over right now. He spends a considerable portion of the book outlining why “freestyle play” (humans synthesizing multiple computer judgements) is the most effective way to play chess. I find the same is true of me personally as a white collar startup founder. I think it’s reasonable to suggest that Dr Watson will be most effective in conjunction with a physician

    With 2 notable exceptions:

    1) Traditional radiology. It’s very obvious that this will be computerized. Computers can report findings and recommend orders for primary care physicians to carry out.

    2) Anesthesia. After sedation, anesthesia is a simple negative feedback loop. JNJ has recently received approval for a device called Sedasys that automatically monitors and administers anesthesia intra-operatively. I can see a day where a large percentage of cases go anesthesia-personell free.

Previous post:

Next post: