NPR, as a rule, shies away from reporters using anonymous sources. But what happens when a highly credible on-air guest cites an anonymous source?
Last month, Dan Ariely, a behavioral economics professor, talked with All Things Considered host Robert Siegel, about how incredibly loyal, almost irrationally so, people are to their dentists – more so than with other medical professions.
Ariely studies irrational behavior.
He wrote the New York Times bestseller, “Predictably Irrational: The Hidden Forces that Shape our Decisions.” His research is published “in leading psychology, economics, and business journals,” says his bio. He also holds the James B. Duke Professor of Behavioral Economics chair at Duke University.
So when he appeared on NPR’s air, there was every reason to trust him.
Ariely offered information certain to unnerve listeners and anger dentists ¬– information based on a fact that he cannot back up.
If two dentists were asked to identify cavities from the same X-ray of the same tooth, Ariely said they would agree only 50 percent of the time.
Ariely cited Delta Dental insurance as his source. However, Delta spokesman Chris Pyle said there is no data that could lead to that conclusion.
Here is what Ariely said:
Prof. ARIELY: And we asked both dentists to find cavities. And the question is, what would be the match? How many cavities will they find, both people would find in the same teeth?
SIEGEL: And I'd really hope it would be somewhere up around 95-plus percent.
Prof. ARIELY: That's right. It turns out what Delta Dental tells us is that the probability of this happening is about 50 percent.
SIEGEL: Fifty percent?
Prof. ARIELY: Fifty percent, right. It's really, really low. It's amazingly low. Now, these are not cavities that the dentist finds by poking in and kind of actually measuring one. It's from X-rays. Now, why is it so low? It's not that one dentist find cavities and one doesn't. They both find cavities, just find them in different teeth. (Soundbite of laughter)
Prof. ARIELY: And here is what happens. Imagine you're a dentist, and you see a patient, and you really want to find a cavity because you get paid more if you find cavities and you can fix them. And the patient is already on the chair. He's already prepped. You might give them the treatment right now, really good marginal income for you. How is this motivation to find cavities - will influence your ability?
Now, you look at an X-ray - which is a little fuzzy and unclear, and there are shadows and all kinds of things are happening. What happens is this unclarity of the X-ray helps, in some sense, the dentist to interpret noise as signals, and find cavities where there aren't really any.
SIEGEL: And fill them?
Prof. ARIELY: And fill them, and drill them, expand them. I don't think they ever tell their patients, hey, I thought it was a cavity but turns out it was just a mistake. (Soundbite of laughter)
Not surprisingly, the American Dental Association took great offense at Ariely’s statement. As did many dentists and some listeners.
“The individual being interviewed was making all manner of claims regarding how dentists are just making more money based on misdiagnosing information on X-rays,” wrote Don Bassham, a dentist from Walla Walla, WA. “Filling teeth that didn’t need to be filled and not ever telling the patient about a mistake in their diagnosis. The strong implication in the whole report was that dentists are just out to empty your wallet and blatantly abuse the trust patients have placed in their dentist.”
Delta Dental is the largest network of dentists in the country, with more than 130,000 dentists, said company spokesman Pyle. He called Ariely the day after the story aired. Pyle acknowledges that medicine is not an exact science and diagnoses are open to interpretation.
“But according to Dr. Ariely, he was basing his statement on a conversation he said he had with someone at Delta Dental,” said Pyle. “But he cannot cite Delta Dental in making that claim because we don’t collect any data like that which would come to such a conclusion.”
So what happened?
Ariely said he got that 50 percent figure from a Delta source who told him about “some internal analysis they have done and they told me the results. But they didn’t give me the raw data. It’s just something they told me.”
Ariely did not provide the name of the Delta medical officer, whom Ariely said was not interested in talking with me.
“The ADA was angry with me and they are still angry,” said Ariely. “Delta Dental was very upset that I used their name and I shouldn’t have used their name. That’s the reality.”
Ariely told me he happened upon that figure when he was conducting research analyzing 20 years of raw data on Delta claims. He wanted to know: Was there a relationship between the length of time one sees a dentist and the kind of treatment chosen?
“As patients were longer with a particular dentist,” Ariely told me, “the final treatment tended to be the more expensive treatment even if it doesn’t benefit the patient.”
To back up that finding, he used data Delta provided him.
But Ariely did not see or analyze any data that would lead to a conclusion that dentists would agree only 50 percent of the time based on studying an X-ray.
Unless, the professor could provide a specific source or study that listeners could independently check, he shouldn’t have said that on the air. He said this has been an “expensive but good lesson.”
But what is NPR’s responsibility?
This is a tough one. NPR can’t re-report and check out every thing that an on-air guest says. The difficulty is compounded when the interview is live and the host asking the questions might not have access to the information necessary to challenge a false or exaggerated statement.
In this case, the interview with Ariely was taped ahead of time and edited for air – but no one thought it necessary to challenge his undocumented statement.
ATC executive director Christopher Turpin said NPR had no reason to question Ariely, given his credentials as a tenured professor and an expert on how irrational human beings are.
“It’s a delicate game,” said Turpin. “We’d have to be a very different and much larger news organization if we went back and investigated everything before air. We do take things on trust. If someone says something that is really questionable, then we have an affirmative obligation to check it out. But there was no legitimate reason to question Ariely’s statement.”
Siegel might have asked the basic question that journalists always need to ask: How do you know that? And he could have pushed Ariely to back up his claim.
I’m not sure, though, that this particular incident could have been avoided. ATC did read Delta’s statement denying Ariely’s claim the day after the story aired.
Nonetheless, Ariely’s unsubstantiated assertion unfairly hurt the reputation of many honest dentists and planted a seed of distrust with patients.
At the very least, ATC should correct the online story to make it clear that Ariely cannot back up -- with a specific study or named source --his accusation that if you ask two dentists to identify cavities from the same X-ray, they agree only half the time.
ATC has other pre-taped segments with Ariely, and those should be double-checked before they are aired. There's no doubt that Ariely is both entertaining and informative about how irrational we humans are -- but he also must be right.