Incredible credentials
Why experts who focus on individuals aren’t necessarily experts when it comes to new population patterns
When a dodgy health-related claim appears in the media, it will often come with credentials. For example, a cardiologist might make misleading claims about COVID vaccines and heart attacks. Or a psychiatrist might make weakly evidenced claims about the magnitude of smartphones’ impact on mental health.
From podcasts to opinion pieces, the implication is that the specialist knows what they’re talking about, because they regularly see patients affected by these conditions. This means they should be well qualified to comment on potential causes as well, right?
Not necessarily. I’ve previously written about how population health typically focuses on averages rather than individuals. A clinical trial can tell us the average effect of an intervention on an outcome like death, and a cohort study can help us estimate the probability of a particular disease outcome. But in general, these studies won’t tell us with confidence whether that intervention will work for a specific individual, or whether a specific individual will have a given outcome without the intervention.
The challenge can also run in the opposite direction. Someone may be very good at treating individual patients, but not so good at interpreting counterintuitive new patterns in population-level vaccination trends. Just because a cardiac surgeon can save lives during surgery doesn’t mean they are well-placed to untangle the quirks of vaccine data. Put simply, heart surgery isn’t good training for conducting epidemiological analysis, just like you don’t want an epidemiologist doing heart surgery.
This doesn’t mean people can’t specialise in more than one thing, of course. There are lots of great clinician scientists out there, whose work spans both individual-level treatment and population-level research. But they will have built expertise in each area they’re specialising in. It’s also why collaboration is so important in disease research, to bring together different perspectives – from clinical practice and virology to immunology and epidemiology.
It’s particularly tempting to dabble in wider topics during media interviews. It’s flattering being asked for your opinion by a journalist, especially if you have lots of them. But I think it’s important to defer to others if it’s something that you really aren’t best placed to talk about – especially if it’s a topic that is often open to misinterpretation.
For example, in early November 2020, I was invited to join a BBC Question Time panel. (I’d previously been lined up earlier in the autumn, only to be dropped at the last minute, but that’s a story for another time.) The astonishingly good Pfizer-BioNTech trial results had come out on 9th November, so there was going to be a dedicated Q&A part of the show on the details of the new types of vaccines. My response: for a topic of this importance, they really needed to get a vaccinologist on, who could properly explain the different technologies involved. (Recent misleading claims about mRNA vaccines from the incoming US administration suggest that correctly understanding the benefits and limitations of these technologies remains a challenge.)
Whenever we see an appeal to credentials in the media, it’s useful to consider whether the expertise is as aligned as it might seem. Sometimes the most dramatic claims come from ‘experts’ who work on a particular aspect of a health issue – just not the part that matters when it comes to investigating patterns in wider data.
Small edit: although cardiologists perform keyhole procedures, now clarify that heart surgery is done by cardiac surgeons.
My new book Proof: The Uncertain Science of Certainty is available now.
And more on the counterintuitive patterns that can occur in population-level vaccine data:
Three vaccine quirks that aren’t what they seem
Last month, a new study estimated that COVID vaccines had directly saved around 1.6 million lives between late 2020 and March 2023. In reality, this number is likely to be even higher, because the study only accounted for the direct effect of the vaccine (i.e. protecting from death if infected) rather than any indirect effects (i.e. reducing transmissio…
Cover image: Ian Keefe via Unsplash
The other issue is confusing individual interventions and population interventions. For me, the great example is, would I be better off if I wore a bike helmet? And, would the population be better off if we legally mandated the wearing of bike helmets? It is common to see neurosurgeons being asked about their opinion on the second question, which is a long way outside their knowledge.
At one point in my public health career I was involved with "commissioning" - decisions about how much resource should go into different health specialties or services. I quickly realised that specialists all overestimated the prevalence of the conditions they treat. Not surprising, really - their clinics are full! But they mostly had a very handle on how common the condition actually was. Tertiary specialists - who were referred all the serious cases from a wide catchment area - were particularly susceptible to this...... Paediatricians, for example, lobbied hard (and, eventually, successfully) for the introduction of non-group-specific meningococci vaccine. About 10% of children who get ill with meningococcal disease die; and survivors are often severely affected, with eg amputation, deafness, or brain damage. It was very distressing to tell parents of the death or damage to their children. But the proportion of the population that gets I'll with meningococcal disease is very small. And, oddly, the economic "cost" to a society is quite limited; whereas rolling out a very expensive new vaccine across the entire young population is extremely expensive. With limited resources, could the limited pot of healthcare funding be spent better elsewhere? In the end it was the cumulative cost of treating damaged survivors for the rest of their lives that swung the decision. drugsincontext.com/vaccination-against-meningitis-b-is-it-worth-it ...... Similar examples are everywhere, whether you're looking at erectile dysfunction services (possibly less well funded than they should be because they're embarrassing), or the always-quote hip replacements...... Another factor is "service-led demand", or lack of demand. If services for a particular condition are so limited that there is no point in seeking a specialist appointment, GPs don't refer: so there is no demand, and therefore no perceived need to fund more of those services......