The invitation from the London Review of Books to review Ben Goldacre’s Bad Pharma™ reads:
“We were unsure, at first, what a review could add that isn’t already in the book – scrappy summaries and bits of praise are not for us. The book is of sufficient importance that the main thing is to get someone who knows what they’re talking about to present the material confidently.. frame the discussion”.
My head said it was inconceivable that the LRB wouldn’t take a review, even if it was at odds with the invitation to praise Bad Pharma. But my gut told me the inconceivable was about to take flesh.
Sure enough LRB turned down the review. Because, they said, their readers would be baffled by it – piquant given that LRB specializes in complex reviews of the esoteric and the obscure.
Now it is unfair to say that if Ben Goldacre didn’t exist, Andrew Witty, the CEO of GlaxoSmithKline, might have had to invent him. But it needs something like this to flag up how perilous our position is and how paradoxical – seemingly beyond the capacity of the editors of LRB to follow. This rejected review will puzzle some, and perhaps annoy those it doesn’t puzzle because it leaves answers for another day.
The review
Ben Goldacre made his name with an earlier book Bad Science. Bad Pharma is equally well written. It will be accessible to anyone interested in health and its politics. It strikes the right note of outrage. It tackles the problems pharmaceutical companies pose to medical care and Evidence Based Medicine. The risk is that the solutions it proposes will aggravate rather than resolve our problems.
Data gone missing
The central problem it targets in the opening section is the clinical trial data missing in company vaults. The story is bang up to date, covering Roche’s withholding of key data on Tamiflu, leading governments worldwide to hand over billions of dollars of your money to stockpile Tamiflu in recent years. The more of this data academics like Peter Doshi and Tom Jefferson have got hold of the more questionable this decision to buy the drug has looked.
Regulatory capture
The next section deals with what is widely called regulatory capture. The regulatory agencies that approve drugs, such as the US Food and Drugs Agency (FDA) contain many bright and good people. But somehow the system fails patients and doctors, and simply can’t be trusted to ensure that medications on the market are safe and effective.
RCTs & EBM
The third section introduces randomized controlled trials (RCTs) which form the basis for evidence based medicine (EBM). RCTs and EBM sound as good as motherhood and apple pie to most people. After outlining how RCTs and EBM should work, Ben offers a compendium of ways in which companies game RCTs including selected reporting, selected patients, and selected comparators, in trials that are too short, too long or too small, that yield outcomes that are surrogate, composite, incomplete, or stem from subgroup analysis, with clinicians misled by talk of dramatic relative risk reductions when the absolute risk reduction is close to non-existent.
The big picture – RCTs are critical to good medicine and everything would be fine if only companies didn’t “game” them.
Marketing
The final section entitled “Marketing” contains the juiciest details and the reddest meat. It outlines how advertising to patients works, how companies recruit celebrities to endorse products, sell diseases to sell medication, create patient groups, and employs PR agencies to get their message out and bury their critics.
It covers drug adverts in medical journals, the tune to which journals profit from drug company articles, the corrupting power of trinkets and free lunches, how sales reps track the personalities of doctors, and how companies like IMS Health sell data on the scripts that doctors write so companies know exactly what pitch will work with your doctor.
Ben’s searchlight switches to ghostwriting and how companies now run the continuing education of doctors. This leads to the knotty issue of conflict of interest. He draws complex distinctions between situations and behaviors, and the merits of greater or lesser puritanism, and to his credit is not an extreme puritan.
The wrap
The book ends with a scene, a question and a plea.
- The scene is Ben being told at a party that Andrew Witty, GSK’s current CEO, is a good guy, who bangs the table in his efforts to ensure things are done better.
- The question is, how will industry respond to the challenge he poses?
- The plea is to any of us who have any other ideas on how to solve all this, to let him know.
Framing the issues
Many readers of Bad Pharma will know much of what is here from books such as Marcia Angell’s The Trouble with Drug Companies or Jerome Kassirer’s On the Take. Where is the problem in re-iterating it eloquently? The central problem is the premium Ben puts on controlled trials not found in other books.
Marketing 101
Ben’s Marketing chapter shows how doctors just don’t get it. It confuses sales for marketing. Marketing means understanding doctors better than they understand themselves, and shaping their thinking so that the drug becomes the unquestioned answer to any problem a doctor faces. There is no better way to do this than with clinical trials in eminent journals.
Bad Pharma cites all the research showing that companies get the results they pay for but ignores the research showing that on all quality metrics, whether adequacy of trial design, transparency of procedures, or declaration of conflicting interests, industry trials routinely come out better than academic studies.
Ben cites studies about how doctors don’t believe they can be influenced by sales trinkets. Doctors say they follow the evidence. Ben believes they are influenced by the gifts (sales), and would prefer them to be influenced by the evidence. Andrew believes that swayed by books like Bad Pharma, doctors are increasingly influenced by the evidence (marketing). Andrew uses gifts as decoys.
Bad Pharma spots the sneaky little sales gnats and swallows the marketing camel, and while the advocates of evidence based medicine do just this, the only outcome can be growing perplexity at worsening indigestion and ever shriller calls for more regulation and policing of conflicting interests – which pleases Andrew.
Regulation 101
Bad Pharma calls for more regulations but the more regulations there are the stronger industry becomes. The Chinese sage Chuang Tzu caught the dilemma beautifully in 323 BC:
“For security against robbers who snatch purses, rifle luggage, and crack safes, one must fasten all property with ropes, lock it with locks, bolt it with bolts. This is elementary good sense. But when a strong thief comes along he picks up the whole lot, puts it on his back, and goes on his way with only one fear; that ropes, locks and bolts may give way.”
The problem for healthcare worldwide goes back to a set of regulations – the 1962 amendments to the US Food and Drugs Act put in place following Thalidomide. These were later adopted everywhere else. They had three critical components to them:
1. Product patents supersize profits and risks
The first component involved maintaining the US system of product patents on drugs rather than the German system of process patents (Dancing in the Dark). Congressional staffers in 1962 had demonstrated that process patents led to more innovation in pharmaceutical development and produced cheaper drugs than product patents but instead the Germans were persuaded to change to product patents.
Universal product patents laid the basis for the development of blockbuster drugs pioneered by Glaxo two decades later. Blockbusters are drugs that earn a billion dollars or more per year for a company. They are not life-saving compounds. They are highly marketable products that are literally worth more than their weight in gold. This distorts company priorities. It puts a premium on hyping the benefits and concealing the hazards of a few unimportant drugs like: Prozac for depression, Lipitor for cholesterol lowering, and Fosamax for osteoporosis, all of which we could have easily managed without.
But companies can now go out of business if a significant hazard of their blockbuster comes to light. So company ingenuity is funneled into hiding hazards. Bad Pharma is silent on the role of patents in creating this perverse incentive. By putting the premium it does on clinical trials, it plays straight into pharma’s hands.
2. Prescription-only status addicts doctors
A second set of bolts put in place in 1962 was prescription-only status for medicines (Dance of the Sugar Plum Fairies). The idea that drugs should be prescription-only fits Ben’s worldview – no matter how health literate you are doctors have spent more time being educated in these things than you and they’re here to manage you.
But prescription-only was a mechanism designed to control addicts and it makes doctors rather than their patients the consumers of medicines. And pharmaceutical companies bring more money to bear on influencing this small band of consumers – $50,000 per annum per doctor – than other companies bring to bear on influencing an entire population. When they slip on the Ring of Prescription Privileges, doctors become visible to Eye of Sauron and shrivel.
The hazards of thalidomide came to light in Germany because it was over-the-counter there and doctors didn’t make a living out it. Ditto with tobacco and alcohol.
In contrast, the significant hazards of prescription drugs now take 15 – 20 years to be accepted. The combination of trials and prescription-only status has fashioned doctors into a risk-laundering service for companies. Bad Pharma mentions nothing about this.
3. The capture of medicine
Controlled trials were the third set of ropes and bolts put in place in 1962, and Andrew is praying they don’t give way (Dance to the Music of Time).
Louis Lasagna was the Ben Goldacre of his day. He was the first Professor of Clinical Pharmacology, the author of the version of the Hippocratic Oath that many doctors take on qualifying, and one of the earliest exponents of controlled trials. In 1962, almost by accident, he slipped the use of controlled trials as a test for efficacy into the FDA amendments and set us on the road to tragedy.
Regulations up till 1962 had been about safety. Forcing companies to show a drug worked for something was intended as an additional safety step. Eliminating drugs that don’t work seemed the perfect way to “first do no harm”. Except that’s not what happened.
All of a sudden this new scientific tool, whose uses and quirks are still not fully understood, became a tickbox for bureaucrats – if there are some positive studies regulators can let a drug on the market and no-one can criticize them afterwards no matter what goes wrong.
This is the point at which Bad Pharma makes a category mistake. It treats a box ticking exercise as though this could inform the practice of medicine.
Regulators deal with both food and drugs. All they want is a way, in the case of food, of determining if this yellow slab that could be lard colored to look like butter meets criteria for butter. If it does they can let companies label their product as butter. It’s not the regulators job to decide if this is good butter, what butter might be used for, whether it’s healthy for you – that’s up to consumer groups and physicians.
For drugs, the criterion is two positive trials. These regulations were not designed to have anything to do with the practice of medicine, they regulate adverts. The FDA in the US, along with regulators in the UK and elsewhere, are ticking boxes perfectly well – and as the law stands cannot do otherwise. It is hardly surprising that pharma have found many ways around the regulations. Consumer groups and physicians step up to the plate when it comes to butter but not when it comes to medicines. Who’s captured?
Introducing trials into the regulatory apparatus has created a mess. What’s been captured is the production of evidence and the more books like Bad Pharma fetish clinical trials the more captured medicine becomes.
The limitations of RCTs
RCTs can give the wrong answer
RCTs can be very helpful when they disprove claims that a drug works. This can rein in the unscrupulous huckster, the credulous physician and protect the vulnerable patient.
But RCTs can also be unhelpful. For many conditions they simply do not produce a meaningful outcome. For instance, where superficially similar problems can be caused by both drug and illness, such as suicidality on antidepressants, RCTs may perversely show that drugs that clearly cause a problem seemingly don’t cause it. Within the mental health domain it’s debatable if RCTs can show anything “works”. Using the clinical trial procedures used to bring Prozac on the market, alcohol could have been shown to be as effective an antidepressant as and safer than Prozac (Shadow Dance).
Rather than being used as initially intended – to first do no harm – trials have given rise to an efficacy fetish. As a result we are now all put on an increasing number of drugs that have been approved by regulators and there seems to be no way to limit the number of drugs we end up on. End up on because these drugs that work have not been shown to save lives. The result is that drug induced death is now the third leading cause of death. This is a tragedy of mythic proportions.
RCTs tell us almost nothing about cause and effect. They discover nothing. They likely block the discovery of many treatments. But for the fact that Barry Marshall in 1980 took a laboratory rather than a clinical trials approach to showing ulcers were caused by h pylori and could be cured by antibiotics, Glaxo would have used RCTs to bury the evidence that antibiotics cure ulcers to defend the very first blockbuster drug, their H2 antagonist for ulcers, Zantac.
More generally in a 400 page book about treating illnesses and drugs, Bad Pharma has only 3 pages (101-104) that come close to dealing with biology, even though biology is the primary driver of the superficial associations that controlled trials throw up.
Disease mongering
Efficacy for something is another component of the tragedy. This element restricted companies to selling medicines to doctors for diseases only. In 1962, no-one foresaw that if constrained in this way companies would convert what had been a series of vicissitudes of everyday life and normal variation in terms of beauty and functionality into a set of diseases. And so we have all become depressed, osteoporotic, with hypercholesterolemia where otherwise we might have had burn-out, aging bones that could be managed by exercise, and a diet-related issue that is only significant against a background of more important cardiac risk factors.
Companies now speak the language of medicine to an extent that doctors fail to appreciate. Not only have vicissitudes been transformed into illnesses, but acute illnesses have become chronic and the moral imperative to treat that is brought to tuberculosis has been co-opted to sell products for restless legs and fibromyalgia. In 1962, the drive was to temper patient enthusiasm for treatment with medical skepticism. Fifty years later skeptical patients try in vain to temper the zeal of doctors following the supposed evidence embodied in the latest guideline based on published trials.
Fixing a bad system
Bad Pharma describes the superficial features of the problem but it fails to analyze how we got here. It just assumes Pharma is Bad and that everything else follows from this.
This analysis in contrast appeals to an historical moment – things that might have been otherwise. Some might argue the insertion of controlled trials into drug regulation was an accident waiting to happen. Others might say this is an unparalleled demonstration of the perils of interfering with the market. Whatever frame you opt for, the result is the strangest market on earth in which mega-corporations spouting a rhetoric of market freedom operate in a totalitarian way to control the least free market ever seen.
Ben locates missing data as the key problem and access to the data as the solution. This is a hot issue in Europe with the British Medical Journal leading a campaign for access to clinical trial data.
Is Andrew feeling nervous? Probably not. Companies have been managing this scenario for years. When recent calls for access to trial data emerged (GlaxoSmithKline and Access to Data), GSK immediately offered access to their data once the right academic board and governance is in place. With control of the analytic approach to the data, GSK can achieve both the result they want from “independent” analysis and the kudos of transparency all in one go.
Andrew has patient groups lined up – not paid for by companies as Bad Pharma would have you believe – to recast the argument in terms of academic dilettantes versus patient rights to privacy and confidentiality. For most casual readers there can only be one winner to this argument. Andrew has politicians on board with the need to manage the communication of risk in the public domain – and in a supreme irony on this issue the primers for politicians cite Ben’s Bad Science as their number one text.
It’s like pitting a major league outfit against a set of minnows. If the minnows win they are likely to make a bad situation worse. Good intentions aren’t enough. Unless the good guys understand the system they are up against they can make things a lot worse – just ask Lou Lasagna. Access to trial data on GSK’s terms will make the days of Bad Pharma look halcyon. The good guys don’t think there could be any adverse effects to good clinical trials. If the outcomes aren’t right, like a religious belief it will always be because the trials weren’t good enough, not because there are so many things for which trials aren’t the answer.
“If the outcomes aren’t right, it will always be because the trials weren’t good enough, not because there are so many things for which trials aren’t the answer.”
Missing data
Missing data are critical but Bad Pharma misses the critical data. Andrew probably purrs at Ben’s dismissal of adverse event reports, at his missing company perversities around hiding adverse event data that are more astonishing than ghostwriting. For instance companies in the US now encourage patients and doctors to report adverse events directly to FDA rather than to the company. Enlightened? Very – companies have a duty to follow up adverse event reports and to establish causality whereas FDA doesn’t. Diverting reports to FDA saves money, reduces legal liability and relegates adverse event reports to the status of anecdotes (See American Woman and American Woman 2).
The ultimate perversity is when in the face of hundreds of compelling reports of a treatment induced problem, when employees using standard causality algorithms have said their drug caused this problem, companies will under oath deny there is any link between the problem and their drug (Burn in Hell). Thanks to EBM, they can get away with this. How? Well built into RCTs is the notion that the only findings that count are those that are statistically significant. And because RCTs are designed to see whether one event is statistically significant or not – whether the drug works – they cannot properly look at what else the drug might be doing and so the safety of drugs which was in 1962 the primary goal of regulation has been eclipsed.
The fetishing of clinical trials means it’s now difficult for doctors or patients to believe the evidence of their own eyes. Restoring this ability – which clinical trials almost by definition attempt to undermine – is more important to the practice of medicine than access to data about “butter”.
“The plural of clinical trials is not medicine.”
The individual patient once sat at the top of the EBM hierarchy – but no longer – following a relentless repetition of mantras like “the plural of anecdotes is not data”. Well the plural of clinical trials is not medicine.
The candidate of hope
At times you have to stand back and admire how good Andrew is at his job. Is he really banging the table just to get GSK to be a bit more like what Ben would like? Ben calls for more industry whistleblowers but most of what he cites comes from industry whistleblowers in the US and very little from doctors anywhere. If I was one of the many ex-pharma employees whose whistleblowing cases have not worked out, I would feel bitter and ask just where have doctors been in all this?
The problems could be settled without trying to make Pharma good. Doctors could but don’t take the simple step of banning industry “trials” from their journals. Because the New York Times takes the checking of the primary sources of an article more seriously than the New England Journal of Medicine does, we might all be safer if clinical trials were published in the NYT rather than the NEJM – which is a damming indictment not of pharma but of doctors.
Bad Pharma is relentlessly British in its focus. Ben wants the Royal Colleges to stand up and make a difference. Within the terms of the debate he outlines, there is almost nothing Britain can do. The one thing that might rattle Andrew would be an exhortation to doctors and patients to start believing the evidence of their own eyes again – but this Ben cannot say.
Instead Andrew offered controlled access to clinical trial data. In response as the BMJ noted “[Witty] was singled out for praise by one of the industry’s harshest critics. Ben Goldacre called it a “cartwheel moment”.
The BMJ issue in which this comment features had a three-page spread on Andrew Witty featuring his image on its front cover in political poster format, resembling no-one more than Barack Obama, with a single word Hope underneath. Making the case for GSK, as Not so Bad Pharma, he says “the 100,000 people who work for GSK are just like you, right? I’m sure everybody who reads the BMJ has friends who work for drug companies… They’re normal people. Many of them are doctors”.
Professional suicide
EBM like Freud has been wonderful in bringing the biases of both doctors and patients to light. But just as the Catholic Church has learnt there comes a point where routine dismissal of what people say to you because some of it is mistaken leads to a crisis, medicine needs to realize that EBM has led us into a comparable crisis.
Doctors need to realize their profession is at stake (Professional Suicide, Model Doctors). If clinical judgment and discretion no longer matter, nurses and others will be far cheaper prescribers and can more readily be ordered to stick to guidelines than doctors. Anyone can get you on a drug; it takes an expert to get you off. How does an expert know when or how to do this? Not from clinical trials.
Bad Pharma™ hypes the RCT and EBM brand, and denies the existence of any side effects. The only other place in medicine where this is done to a comparable extent is with blockbuster™ drugs. A Freud might think the™ revealing.
Lou Lasagna stands as an extraordinary symbol of how misguided this approach might be. He pushed for an inclusion of placebo-controlled trials in the 1962 Food and Drugs Act in the belief that establishing the efficacy of a drug would make a contribution to the safety of all us. But as of 1962, only one drug prior to marketing had been demonstrated in a placebo-controlled trial to be safe and effective – thalidomide.
“Anyone can get you on a drug; it takes an expert to get you off”
Interestingly I ended up having to do it by myself, more or less! As once doctors get you on a drug they can expect you to take it for life. In my case initially I was told I would be on Ciprager (Citalopram) for the rest of my life, which I took for panic attacks. And when I became unwell on that and developed what I now call “SSRI-induced Bipolar Disorder” I was told that I would be on Lithium for the rest of my life. But that was the day I decided to come off Lithium and haven’t looked back (2010) (In jumps of 200mg ~ it needs to be done slowly. I probably came off too fast and became depressed due to the withdrawal symptoms).
And I managed to get off most of the rest of the over-prescribed drugs (eg Seroquel & Zyprexa ~ both of which have lawsuits against the makers) by the end of 2011. Not before I get labelled “non-compliant” and “a’la carte” by GPs. {I was told I was non-compliant because I had stopped Lithium for 2 days as I had a medical complaint and was worried it could become toxic in my system, based on previous advice from a Psychiatrist. And I also refused to move from 150mg to 200mg to Seroquel as I felt that I had become angry on 200mg}
The only thing that stands between me and RECOVERY at the moment is lack of SUPPORT. So many people have turned their back on me in the last 5 years. And also the key ingredient to RECOVERY and major aspect of my health ~ Sleep. So at the moment I do daily Yoga and Meditation and I am also working on Diet. As well as exercise.
Anne, I know exactly what you mean both professionally and personally.
I have undergone 10 surgeries in my life and have been on numerous drugs, many of which were given to me with no mind from the physician of either how or when i would taper off, no instruction, no support.
I have personally thought that the argument that is given regarding current medical practice (ie. first do no harm) should start with support, not with trial, it is compassion that should give the foundation, not hurrying to get to the next patient with no humane oversight.
The worst of it, for me, and many others is ‘death by diagnosis’. I was told that i was essentially screwed, that i there was nothing that could be done (yet another series of surgeries), and that there were no conservative options. I became suicidal (as the surgeries have a poor batting avg with more than 50% worse off after than before), and obsessive about the condition.
I am fortunate to have found good medical care, but it is still very difficult.
I wish you all the best in your recovery, and if you need an ear email me, there are many in our shoes who would benefit from a friend who’s been down or going down the same road.
(Tymothys@gmail.com)
Hi David,
Bad Pharma is an attempt to give a straight explanation, for the general public, of various well-documented shortcomings in trial design, trial reporting, and evidence dissemination.
You believe that the very notion of randomised trials is broken and wrong. I think it might be useful for your readers if you could post the feedback you’ve had on your ideas from people like Donald Irvine, and Iain Chalmers, who I know has looked at them in some detail.
Ben
Why do you suppose the LRB refused to publish?
If you say I think RCTs are broken, you have the review wrong and perhaps the emails to and from me to you, Iain Chalmers and Donald Irvine wrong. I don’t think RCTs are broken. I think they are far less useful than is commonly made out and in many cases can be seriously misleading – as laid out in a draft article sent to you and Iain and others to critique.
The first point to orient anyone who might want to read the emails is that they arose because I have taken the debate to you, Donald Irvine, Iain Chalmers and others. I sent a link to a lecture I gave in Cardiff, Time to Abandon Evidence Based Medicine, which can be accessed on http://wp.rxisk.org/videos/
The full email exchange is below – starting with Donald Irvine saying Like all fads in medicine, EBM was seen by its devotees when it was introduced as the be all and end all of good clinical practice. Iain Chalmers lists areas of agreement and disagreement. But neither he nor anyone else picks up the issues in this review. They stick to the clinical trials issues and avoid the frame outlined here.
I go along with Andrew Witty that there is as high a proportion of decent people working in GSK as there are in medicine or in the BMJ. The idea of Bad Pharma (its not Bad RCT) in this sense doesn’t ring true.
I think we have a bad system and unless EBM addresses this, it runs the risk of playing straight into Pharma’s hands. A key problem with the system is the use of RCTs for regulatory purposes. This is what hasn’t been addressed.
It’s been difficult to address the issues when I offer to come and debate and no-one responds. I have sent the article mentioned above to a journal that is usually quick to handle things suggesting Iain and you as reviewers or respondents and where other articles have been turned round within a week, its been months without reply in this case. Its difficult to address the issues when even LRB chooses not to publish a review.
Difficult when Iain ends an otherwise friendly note with: I have spent two days reading, considering and responding to your request for comments on the material on which you have asked me to comment. I hope that you will understand that I will not be able to invest any more time discussing our disagreements.
I haven’t included here the birth defect material mentioned in these emails. I think this is one of the greatest scandals of our time. I had hoped that Iain with his background in this area and you and others might use your considerable influence to draw attention to the scandal – if you want a real live example of the consequences of bad trials and ghostwriting etc you can do no better than this.
On the broken system theme, I’m going to develop this further in posts over the next few weeks starting with April Fool on April 1st.
David
From: Donald Irvine
Date: 22 November 2012 08:05
Subject: RE: Patient centred Care
To: David Healy
Dear Professor Healey,
First, I must apologize for the late reply. I have been somewhat pushed of late, but that is no excuse.
Like all fads in medicine, EBM was seen by its devotees when it was introduced as the be all and end all of good clinical practice. Now it has found its place across a spectrum of elements such as clinical competence and the achievement of good outcomes, which together constitute good clinical practice. That is surely right.
However, as you know from your own experience, for patients that is only part of the story. For them, care matters just as much, a fact that too many health professionals seem to overlook.
I wonder that you want to battle with the EBM devotees. Most sensible people have the sense of proportion I allude to above. But if you do decide to go this way, the RSM would be as a possible suitable forum.
Lastly, have a look at the attached report from the UK cardiac surgeons. There is a group which are determined to give patient-centred care. It is an holistic concept, as Fred Hafferty and I point out in our chapter.
Donald Irvine
From: Iain Chalmers
Date: 1 December 2012 14:58
Subject: RE: EBM?
To: David Healy Cc: Ben Goldacre et al
Dear David
I watched your lecture in bed late yesterday evening. I made a few notes, but not as many as I might have done had I not been lying on my back! As you have invited my thoughts, here are some for you to consider based on my scribbles.
Almost all of the actual data you presented concerned ‘commercially important’, anti-depressant drugs. You clearly know more than most people about that field. My research background is in pregnancy, however, during which drugs tend to be avoided and those that are used are of no commercial interest (magnesium sulphate for severe pre-eclampsia; corticosteroids prior to preterm delivery, for example) . So extrapolating from the field with which you are most familiar to the field with which I used to be familiar doesn’t really work. Even so, I was puzzled by a comment that you made twice: that clinicians advise women that they must continue taking anti-depressants during pregnancy and that this advice doubles the chances of their giving birth to babies with congenital abnormalities. Neither of these claims rings true to me, but maybe you are drawing on data with which I am not familiar (I left the pregnancy field more than 20 years ago), so please send me the references to evidence about the frequency of that advice, and the association between the advice and congenital abnormalities. If it is true, it is very important.
In several places you point out that estimates of differences that are not statistically significant do not (and can not) justify claims of ‘no difference’. Unfortunately this remains a common problem, despite repeated efforts (by me among others) to reduce it.
Your position on clinical trials confuses me. On the one hand you say that clinical trials are “terribly useful”, and that you’re “a great believer in clinical trials”; yet you offer slide at 47 minutes 19 seconds, entitled “Clinical trials are not the answer”
The title of the slide is a straw man. Who says clinical trials are the (my emphasis) answer? You use 5 points to illustrate why they aren’t the answer.
1. If both the illness and the drug cause the problem. Agreed, as you have shown in a field in which an alleged side effect of a treatment (suicidal ideation) is also a manifestation of the condition for which anti-depressant drugs are prescribed.
2. If the clinical population is heterogeneous. Sometimes yes, sometimes no, as illustrated by lots of examples – for example, trials of low dose aspirin or cognitive behavioural therapy.
3. If you don’t know what you’re doing. It was unclear to me what you mean by this. Do you mean giving a treatment when you don’t know how it works, as with ECT in severe depression, or magnesium sulphate for eclampsia, for example?
4. Cannot provide patient-level answers. As randomised n-of-1 trials are top of the draft of the EBM Group’s evidence hierarchy published several years ago, this is not true. And if you’re referring to using data from groups to provide prediction of treatment effects in individual patients then you should give examples of how you overcome this unavoidable challenge by preferring non-randomised groups to randomized groups as a basis for the guess.
5. Do not reveal cause and effect. Judging cause and effect will always end up with guesswork, so that isn’t a problem specific to clinical trials.
I’m afraid the slide led me to conclude that you haven’t thought sufficiently carefully about whether conclusions that may be warranted on the basis of your very deep knowledge of anti-depressant drugs apply as widely as you appear to assume. This was reinforced by your suggestion that trials that need large numbers mean that treatments don’t work, and that, instead, ‘we’ve got to believe the evidence of our own eyes’. So you wouldn’t want transexamic acid if you get knocked off your bike and started haemorrhaging? Even something as basic as when trials were introduced appears to have escaped you. You refer to ‘1959 as being in an era before RCTs were introduced’, which you go on to date as ‘1962.’ And you refer to Cardiff in your covering message as ‘the birthplace of EBM’. How so?
So, on the basis of this, you conclude that EBM has failed us and the Cochrane Collaboration is a marketing arm of the pharmaceutical industry!? I’m afraid you’re going to have to come up with some more persuasive evidence than that offered in your lecture. But I could tell that you must have had fun preparing the talk, as did the writing group of Clinicians for the Restoration of Autonomous Practice, whose masterpiece you may have read in the much maligned BMJ some years ago
Tally Ho!
Iain
From: David Healy
Date: 1 December 2012 15:42
Subject: Re: EBM?
To: Iain Chalmers Cc: Ben Goldacre et al
Iain
These comments are exactly the kind of response that’s helpful. The reason to send it to you was to get as thorough a critique as possible.
To respond to your points – it looks like we have a common enemy in statistical significance testing.
On the Antidepressants and Birth Defects front – see attached Healy Mangin Mintzes. I was the lead expert in a series of SSRIs and birth defects cases that so far have led GSK to hand over more than $1 Billion in settlements. There is a real concern about the contribution antidepressants may be making to children being born with learning disabilities.
Pharmaceutical companies have been able to shut down all information about this while OTIS now lists the antidepressants as among the most commonly prescribed drugs in pregnancy and in a recent Maternal Services Study, the mean rate of AD scripts in the UK among pregnant women was 10% with Edinburgh leading at 15%.
Attached a draft article on some of the points made in the video that may help clarify some of the points (DBM and Clinical Judgment). In simple form, the argument goes like this: unless we are dealing in hard outcomes such as lives saved, we are dealing with effects. When RCTs discuss the results of the assay on the primary effect they can be wonderfully helpful. But to stray beyond this – even in describing the effect on the primary effect as a benefit risks problems. Everything of course risks problems but the call is for a recognition of a need to re-assert purist interpretations of RCT – something I expect no-one within EBM to argue with but you all de facto risk being co-opted by Pharma with a broad brush endorsement of RCTs as offering gold standard evidence.
If we are dealing with effects, and the effect in question is one that the disease can mimic in some way, then RCTs are in principle incapable of giving a clear answer, and moreover are incapable of even giving an estimate as to the frequency of drug induced problems. They need a supplementary judgement.
You say I’m dealing with a highly commercial area and one group of drug antidepressants and you point to other treatments. I suspect that most other commercial drugs fall into exactly the category I’m talking about and most treatments people are taking – from the statins through to asthma drugs, pain-killers etc.
At the end of the day this is an empirical question.
You raise the hugely important point of the trials that do more than prove a drug does nothing. I agree with this and this is an area which is particularly difficult to pin down. My concern is this – RCTs were introduced as a contribution to the safety of drugs in the belief that drugs without efficacy cannot be safe. The problem we have is that efficacy has been fetished – in terms of clinical practice we need to restore the focus on safety.
The point about the date of introduction of RCTs is a red herring – in that I’ve written extensively on the introduction of RCTs – what’s being said here is that at this point RCTs were not a feature of the AD story – they weren’t a regulatory requirement either.
Re cause and effect – I think we need to resurrect the importance of laboratory work in this domain. I am at one with anyone within EBM who is skeptical about biological claims, but at the end of the day the reason a drug produces a particular effect is established at the physiological level – no amount of RCTs can decide whether the statins produce their key effects by reducing cholesterol levels or by an anti-inflammatory effect.
Consider a treatment for back-ache. Drug A is superior to drug B – on average. Unless we understand the physiology of back problems, however, we don’t know if drug A rather B is the right drug for you – or if you prefer we can’t do the trial that would identify the population for which drug B is particularly effective.
Re Cardiff – this is of course where Archie Cochrane began to elaborate many of his ideas
David
From: Iain Chalmers
Date: 29 December 2012 17:47
Subject: Your Cardiff talk, response to my comments on it, and articles about SSRI’s.
To: David Healy Cc: Ben Goldacre et al
Dear David
At the beginning of the month you sent me and others a videoed talk to watch and comment on. I responded promptly and you responded to some of the points I had raised and sent me over 60 pages of reading material. Two weeks later you wrote observing that you had heard nothing from me or the others to whom you had written, and that the possible explanations were that we think:
1. You’re wrong in which case it wd be good hear just where
2. Out of control and response is not likely to get anywhere
3 Right and we’re mulling over the implications
I responded saying that there were other possible explanations, and that my explanation was that your request would have to take it’s turn after I had dealt with prior commitments. I believe Donald Irvine and Brian Haynes may have given you feedback. I have found much to agree with in what you have sent me, some to disagree with, and some simply confusing.
What do I agree with?
As I said in my earlier message, claims of ‘no effect/no difference’ based on confidence intervals including unity are illegitimate (see attached audit of this mistake in abstracts of Cochrane reviews). However, even you sometimes imply that it is possible to show ‘no difference’. In the Healy, Mangin and Mintzes paper (on p 5) you state that a study with a confidence interval of 0.5 to 2.3 “suggested no effect”. And in paras 150 and 153 (most of the text in these two paras is identical) in you Paxil paper, you state that CBT and IPT as “as effective” as drug treatment for mild to moderate depressions (I would really liked to have seen the evidence on which you based this very important claim, but you didn’t present or refer to any).
I learned a great deal from the Paxil paper from your review and interpretation of data which have become available quite a long time after I left the perinatal field. Thanks for introducing me to this evidence. In brief, I agree with your position. The assertions made by drug companies and their friends are shocking. Gideon Koren’s trial putting pregnant women on prophylactic antidepressant drugs is obscene, but given the role he played in the Nancy Oliveiri affair, I suppose it shouldn’t have come as a complete surprise.
What don’t I agree with?
You state (para 90, Paxil paper) – in bold type (!) – that “no significant adverse effect of a drug…has been demonstrated by means of a controlled trial”. You must live in a very different world from the one I live in. Where do you want the list to start? Prophylactic anti-arrhythmics in MI, steroids in acute traumatic brain injury, antibiotics in preterm labour with unruptured membranes, etc, etc.
Controlled trials are done to address uncertainties about the relative merits of alterative treatments, and new treatments turn out more or less half of the time to be inferior to existing, standard treatments (see attached). In that sense controlled trials identify ‘adverse effects’ half the time.
And you state (para 89, Paxil paper) – again in bold type – that “no Birth Defect has ever been established on the basis of a controlled trial.” What about genital abnormalities in men exposed as fetuses to diethylstilboestrol? (see attached); and cerebral palsy after fetal exposure to antibiotics given to women in preterm labour with unruptured membranes? There may be other examples.
You state that “The only known direct toxic effects on the fetus come from infections such as rubella, drugs such as thalidomide or roaccuatne, or heavy alcohol intake.” (para 142, Paxil paper). I am not a sure as you appear to be that environmental factors are innocuous. Ones that come to mind are radiation and chemicals. I am certainly aware of recorded increases in the incidence of malformed infants after exposure to munitions in Falluja (Iraq) and Gaza.
The features of all of these statements that I am unlikely ever to accept is their maximalist, ‘brook no disagreement’ nature. My strong impression is that you are prepared to generalise from your deep but essentially narrow experience in the field in which you have been a pioneer and to which you have made extremely important contributions.
What I find confusing?
As I mentioned in my 1 December message, your position on clinical trials confuses me. On the one hand you say in your talk that clinical trials are “terribly useful”, and that you’re “a great believer in clinical trials”; yet you offer slide at 47 minutes 19 seconds, entitled “Clinical trials are not the answer”. The title of the slide is a straw man. Who says clinical trials are the (my emphasis) answer? You use 5 points to illustrate why they aren’t the answer. I’ve commented on all of the points previously, but I reiterate the points here because I don’t think you addressed some of them satisfactorily.
1. If both the illness and the drug cause the problem. Agreed, as you have shown in a field in which an alleged side effect of a treatment (suicidal ideation) is also a manifestation of the condition for which anti-depressant drugs are prescribed.
2. If the clinical population is heterogeneous. Sometimes yes, sometimes no, as illustrated by lots of examples – for example, trials of low dose aspirin or cognitive behavioural therapy.
3. If you don’t know what you’re doing. It was unclear to me what you mean by this.Do you mean giving a treatment when you don’t know how it works, as with ECT in severe depression, or magnesium sulphate for eclampsia, for example?
4. Cannot provide patient-level answers. So what can, apart from randomised n-of-1 trials (which are at the top of the draft of the EBM Group’s evidence hierarchy published several years ago)? If you’re referring to using data from groups to provide prediction of treatment effects in individual patients then you should give examples of how you overcome this unavoidable challenge by preferring non-randomised groups to randomized groups as a basis for the guess.
5. Do not reveal cause and effect. Judging cause and effect will always end up with guesswork, so that isn’t a problem specific to clinical trials.
You suggest in your talk that trials that need large numbers mean that treatments don’t work, and that, instead, ‘we’ve got to believe the evidence of our own eyes’. So you wouldn’t want tranexamic acid if you get knocked off your bike and started haemorrhaging?
In para 222 of the Paxil paper, you criticise inadequacies of a paper by Wisner et al. comparing untreated with treated antenatal depression which “claims to find no problems”, but you point out that it has a great many problems: “First it is not randomised…and that only a randomized trial is likely to offer protection against these confounding factors.” I’m also confused by your position on observational studies. In the Persaud and Healy paper you refer (p 33) to the unreliability and potential invalidity of epidemiological evidence compared with case studies.” And I was interested in that paper on Debendox that you made no reference to Elbourne et al. Debendox revisited. Br J Obstet Gynaecol 1985;92:780-5, which reported analyses of two large cohort studies done by my colleagues at the National Perinatal Epidemiology Unit. Could it have been, I wonder, that the confidence intervals were 0.12-3.34 and 0.09-1.47, thus suggesting (on the basis of your commentary on Figures 4-6 in the Paxil paper) that Debendox is more likely to prevent than to cause congenital abnormalities?
I remain confused by the reference in your lecture to ‘1959 as being in an era before RCTs were introduced’, which you go on to date as ‘1962.’ In your response you dismiss my query as a red herring and that you have “written extensively on the introduction of RCTs”. Maybe what you written will clarify what you meant.
I have spent two days reading, considering and responding to your request for comments on the material on which you have asked me to comment. I hope that you will understand that I will not be able to invest any more time discussing our disagreements.
Best wishes for your work.
Iain
Dear Ben Goldacre,
Please find attached a recent critique of your writings about ”lifestyle and complementary” medicine. It highlights some of the errors in critical reasoning you make, including about RCTs and EBM.
http://objectiveskeptic20.blogspot.co.uk/
David, your views are interesting and alarming. Your assault on the idea of EBM is iconoclastic. However, you seem to avoid annotating your arguments with ideas other than your own. Every linked reference I clicked on was to material you had produced yourself. And some of your more sweeping statements are completely unreferenced. How do you justify this?
Gill
There are many assaults on EBM. This isn’t one of them. This is a review about the regulatory system and where I think Ben and others are making a mistake. It has led to a capture of EBM, making EBM part of the problem rather than the answer. This cannot be solved within EBM – it needs engagement with the wider politics of healthcare.
Very few blogs comes with references – the links in this one where to multiple earlier posts where I have raised these issues and in some cases developed aspects of them in more detail. For references you could try Pharmageddon.
David and commenters
I found the post and the comments interesting and thought-provoking.
The LRB’s decision not to publish the review is a puzzle, but one not worth trying to solve.
More puzzling are the adversarial nature of the debate, the frequency with which EBM is framed as an entity that is fixed like some bureaucrat’s book of regulations, and the context-free way in which RCTs are discussed.
I think we are all practicing EBM to the best of our abilties – which leaves lots of room for improvement (in us and in the current understanding of EBM).
Would it not be more productive if we identified problems and then discussed the most plausible ways of tackling them? For example, the way in which evidence from RCTs can be used to tick regulators’ boxes IF the application of evidence is not appropriately critically appraised. A placebo controlled RCT can provide evidence on one class of question about effects, but the question would not be relevant if we want to choose between alternative non-placebo treatments.
Should we not be trying to improve our critical appraisal skills – and those of healthcare professionals and policy makers? Critical appraisal as it is currently taught is looking increasingly naive.
Michael – Yes and No. Yes it would be constructive to look at critical appraisal skills. But here’s a challenge to critical appraisal. Over a decade ago on the basis of compelling N=1 studies, I took a public position that antidepressants can cause suicide. I lost a job because of it. But I was right and it later turns out that all the RCTs when combined show that antidepressants in fact appear to lead to a net loss of life. This loss of life has gone on because GSK and other companies have been able to deploy the argument that controlled trials trump all. They don’t – you can get a controlled trial result showing drugs that cause suicide are linked to less suicidal acts than placebo. So one question for EBM devotees is how comfortable do they feel letting pharma get away with this? A second is can they rise to the challenge and pinpoint what they are doing wrong?
A decent critique of RCTs is one thing, but it is a bad mistake to just have an internal debate. While we neglect the bigger context, a debate about RCTs just strengthens pharma’s hand. I hope to pick up the wider issues in 4 more posts through April – April Fool, The Tragedy of Lou Lasagna, The Empire of Humbug and Brand Fascism – the last post will try to map out possible answers to the problems. Would be great to get your input then especially.
Just in the interests of making the debate more public: Is anyone organizing to put Andrew Witty’s offer of “access” to GSK’s data to the test? It could be a valuable bit of guerilla theater, at least, and maybe more. What he’s offering is basically “full disclosure of everything I think it’s appropriate for you to know,” which is no disclosure at all. I’m amazed that any reasonably sharp journalist or scientist would do cartwheels in response. Yet maybe this is a project the cartwheelers and the skeptics could take on together. Including the really impressive crew gathered here.
Just identify some real issues around GSK drugs and public health, and draw up proposals. You could pull in experts on bipolar disorder, asthma, cancer, osteoporosis, even Restless Legs Syndrome. Organize into very credible, kosher-looking teams: maybe two well-respected investigative reporters plus one medical expert to help them interpret what they find. Or one “controversial” medical expert like yourself, plus two colleagues who are just as qualified but less radioactive.
Then put on your best Sunday clothes and go sit down at GSK’s lunch counter, and politely but very publicly demand your access – just like those black college students did in the American South fifty years ago. Those kids knew that a grilled-cheese sandwich and a place to sit down were not the whole prize by a long shot. They also knew they would more likely be hauled off to jail than served, and were acting on behalf of mothers, fathers and cousins with even less access than they had. Their main objective was to force Greensboro, North Carolina and the U.S.A. to show their true colors in public, and use that to try and spark a movement.
I understand that even if you get the access you won’t find “the truth” there, because studies designed by GSK won’t even have asked the right questions. But anything you win in a fair fight will be valuable. It was a good day when the real details of Study 329, on paroxetine in kids, saw the light of day. That was not irrelevant, and it will not make Sir Andrew Witty cheer if somebody tries to do it again. Couldn’t we at least make him put up or shut up?
The current bought-and-paid-for RCTs are almost worse than no research at all. Still, I can’t go so far as to see the fight for real, independent trials as just a big distraction that can only play into PhARMA’s hands. What about the Womens’ Health Initiative? That was the biggest advance in the fight against breast cancer in years. It’s got to have saved the lives of several thousand women to date, and who knows, I may be one of them – before the WHI, I’d assumed I would someday take hormones to prevent heart disease, Alzheimer’s and all the other things the WHI showed us hormones did NOT prevent. Drug companies HATE this particular RCT; they’ve done nothing but snipe at it since it came out. Including playing the violins for “clinical judgment” all of a sudden.
So sure, RCT’s are not the be-all and end-all. There are many problems they just can’t solve, and they are easy to rig in ways that are hard to spot from the outside. But the very concept is not some kind of trap, is it? What bugs me about Ben Goldacre is not his muckraking over corrupt trials; that’s okay if not perfect. It’s his naivete (or whatever!) in taking GSK’s voluntary compliance scam for something real.
One of the things I have wondered about is how can you have evidence based medicine without an underlying mathematical and conceptual framework for discussing evidence of different types.
With RCTs we are told that anacdote is bad but a logician will happily accept a counter example (a single example) as a demonstration that a theory is invalid. There are of course evidential reasoning frameworks build on top of logic, probability and possibility theory but those researching EBM don’t seem to have looked at this area.
There are clearly different types of evidence. A RCT is not an equivalence class with evidence but it is one mechanism for providing evidence. Maybe I am nieve in thinking that biological theories that explore cause and effect would provide important sources of evidence as do anacdotes (particularly as counter examples). With EBM as it is we could have an absurd situation where a trial happens to suggest that one outcome works (maybe my a low probability of chance) when the biological theory suggests that that shouldn’t happen. EBM would recognise the trial rather than conflicting evidence which needs more investigation and better understanding,
As we look at an individual trial there is a lot of emphisis on the results but it is an experiment and hence we should have a specification for the parameters of the trail. What does the trial actually say and what are generalisations that are being drawn from the trial. In many of the papers I have read there is very little distinction and certainly no attempt at providing a formalised or even semi formal specification of results vs generalisations. Without this it is impossible to do an proper meta analysis of results or even really know do the results apply to this real world situation.
There also seems to be an important issue around measurement and how this is used in statistics but seems to be ignored and washed under the carpet. I read one paper recently where the justification for changing a protocol was based on quoting the mean and standard deviation of data where the distribution of the data did not justify the use of those summary statistics. Worse still a lot of psychiatry papers seem to rely on questionaires – however, they forget this, and abstractly treat them as scales without understanding or thinking about the underlying semantics. So we have papers quoting means and standard deviations for scales that combine multiple different factors into a single number even though this comination has an arbitary weighting of different factors and these factors (although having some correlation) vary independently. In this case the mode is the only viable statistic.
The point here is that in introducing abstraction layers into the RCT and measurement system without any formal specification and validation of the properties errors are being introduced into the results of the RCT. Sometimes these errors would be blindingly obvious if it were not for the scale abstraction. I’m sure there are other issues, for example diagnosis and disease classification, where errors creap in due to the introduction of false or badly specified abstractions.
The underlying point is that without a strong coneptual and mathematical framework for dealing with evidence we will never have evidence based medicine. The danger is where the RCT story is seen as a equivalent to evidence and since no one wants to question the use of evidence the RCT story will be unchallenged,
The philosopher Nancy Cartwright has written some very insightful analyses of RCTs, and the problem of inferring effectiveness from efficacy. One of the points she makes is that statistical significance is not enough to get us from one to another; non-statistical theory is needed. While I have only come across her work recently, and consequently risk oversimplification of her views, I have little doubt that she is doing some very serious thinking and raising some very serious questions.
Nancy Cartwright’s work is very interesting. There are a lot of excellent analyses of RCTs and EBM. But the point here and in posts over the next few weeks are that when it comes to healthcare we are making a category mistake by confusing regulation with science. Hopefully this point will be a little clearer by the end of the sequence of posts
“Why do you suppose the LRB refused to publish?”
————–
Because your review is rambling, overly ideological, three times too long, and hard to follow?
Dirk – It wasn’t overlong. They gave 3,500 words. Tell me what you think the ideology is? Not sure I can spot it. Hard to follow…
I over-reacted, I admit it. But your stance on SSRIs is always a red flag for me. Calling Prozac an “unimportant” drug, etc. But that is an old argument and this is not the time or the place. Guess I thought your piece was too long because it ‘s so rare for mags to run long ones; most book reviewers would die for 3,500 words. 😉
Given the controversies – and huge actual and potential profitability – it’s odd that Ben Goldacre didn’t take look at vaccines or vaccine safety in his critique of Bad Pharma. A cynic might imagine that Sir Andrew had invented Ben as the consummate diversionary tactician – to take the heat off vaccines.
Vaccine Pandemrix, swine flu, has induced narcolepsy in children.
Narcolepsy is a crippling sleep disorder, a neurological condition which interferes with the brain’s mechanisms to control wakefulness. Victims also suffer from cataplexy, a sudden and frightening relaxation of the body’s muscles which causes people to collapse or pass out without warning.
The Department of Health says “The decision to recommend that children got this vaccine was based on *evidence available at the time*. We keep all *emerging evidence* under review and it has now been stopped in those less than 20 years of age.”
The children now take Ritalin and Modafinil and Xyrem (£13,000 a year, not always available) to try to kickstart their systems.
The families of children with narcolepsy from Pandemrix are considering bringing legal action against the Department of Health.
Pandemrix is manufactured by Glaxosmithkline, whose CEO, Sir Andrew Witty has decided to join Alltrials, the latest controversial petition designed to bring comfort that *all clinical trials* are accurate, above-board and not misleading to anyone who took drugs, which were subsequently manufactured, on the basis of these trials and who took them because they were, without any shadow of a doubt, safe.
Anecdotal data is fast becoming the *measure of safety*, so despite signing Alltrials myself, I have little confidence in any weight it will bring to highlight *past* discrepancies.
What would be more useful than Alltrials, would be Regulators of Health insisting on Allresults being in their inbox before any new woefully underwonder drugs are released. And, if Regulators are not up to the job of extrapolating fact from fiction then they should find somebody who is………………..
Thanks David, I will keep my suggestions until you have completed this set of posts.
I will expand on my too brief comments on EBM and critical appraisal, and discuss why “evidence” is so slippery a concept (and thus easy to use in box-ticking evasions of responsibility), and how decision-making always involves both evidence and values. I may also discuss why doctors and their professional associations and regulators) have failed to turn a spotlight on the willful blindness to industry’s deliberate misuse of evidence.
Hello David,
This is one of those instances where a correction is likely to be most welcomed by the corrected.
Enjoy:
The correction original quotation is “the plural of anecdote is data”.
I will not bore you with the details but that is also evidentially correct and it is based on the identical mechanism we use for proof in science [and in rechallenge and in dechallenge – which you know all about].
So where is the correct quote recorded?
‘Raymond Wolfinger’s brilliant aphorism “the plural of anecdote is data” never inspired a better or more skilled researcher’ Nelson W. Polsby PS, Vol. 17, No. 4. (Autumn, 1984), pp. 778-781. Pg. > 779.
This is confirmed in an exchange of emails between Wolfinger and Fred Shapiro, the Editor of The Yale Book of Quotations, The Oxford Dictionary of American Legal Quotations, and others in a linguistlist.org listserv post by Shapiro – you can click on the listserv link and look it up yourself:-
http://listserv.linguistlist.org/cgi-bin/wa?A2=ind0407a&L=ads-l&P=8874
Subject: Re: “Plural of anecdote is data” (Ray Wolfinger)
From: Fred Shapiro
Reply-To: American Dialect Society
Date: Tue, 6 Jul 2004 23:21:27 -0400
Content-Type: TEXT/PLAIN
Parts/Attachments:
Parts/Attachments TEXT/PLAIN (34 lines)
On Tue, 6 Jul 2004 [log in to unmask] wrote:
> Nelson W. Polsby PS, Vol. 17, No. 4. (Autumn, 1984), pp. 778-781. Pg.
> 779: Raymond Wolfinger’s brilliant aphorism “the plural of anecdote is
> data” never inspired a better or more skilled researcher.
I e-mailed Wolfinger last year and got the following response from him:
“I said ‘The plural of anecdote is data’ some time in the 1969-70 academic
year while teaching a graduate seminar at Stanford. The occasion was a
student’s dismissal of a simple factual statement–by another student or
me–as a mere anecdote. The quotation was my rejoinder.
Since then I have missed few opportunities to quote myself. The only
appearance in print that I can remember is Nelson Polsby’s accurate
quotation and attribution in an article in PS: Political Science and
Politics in 1993; I believe it was in the first issue of the year.”
I also e-mailed Polsby, who didn’t know of any early printed occurrences.
What is interesting about this saying is that it seems to have morphed
into its opposite — “Data is not the plural of anecdote” — in some
people’s minds. Mark Mandel used it in this opposite sense in a private
e-mail to me, for example.
Fred Shapiro
————————————————————————–
Fred R. Shapiro Editor
Associate Librarian for Collections and YALE DICTIONARY OF QUOTATIONS
Access and Lecturer in Legal Research Yale University Press,
Yale Law School forthcoming
e-mail: [log in to unmask] http://quotationdictionary.com
————————————————————————–
Clifford
Thanks for this. By the strangest of coincidences, Dee Mangin had tracked down the origins of this quote also and it is built into the next post in the sequence April Fool which was posted yesterday.
The only thing to comment on in Fred Shapiro’s email is “in some people’s minds”. It has become a standard retort in pharmacotherapeutic circles especially those that have any links to industry that The Plural of Anecdote is not Data or The Plural of Anecdote is not Evidence. So much has this been the case that I suspect many people’s jaws will drop at the idea of having to readjust their mindset by so much.
There are a number of words that need reclaiming. Anecdote is high on the list – an alternative in this case might be Narrative Based Medicine – but Big Data is another option (See April Fool).
David
My submission to the Leveson Inquiry about Goldacre may add context.
http://www.ageofautism.com/2012/12/leveson-inquiry-submission.html
Last time I looked Andrew Witty was ‘lead non-executive member’ of the Department of Business, and Goldacre had recently completed a report for the Cabinet Office, so high level political patronage is clearly an issue.
John Stone
David, this is a really thought-provoking article, thank you. Your historical review provides a helpful depth to my own rumination on EBM, methodology and causal inference. My take is from an even broader angle, I think — life is uncertain, and scientific inquiry on the one hand attempts to bracket such uncertainty, and on the other magnifies it. To apply half-baked data (and pretty much all RCTs are half-baked data) to real humans is hubris. RCTs are the best we have today in terms of reaching internal validity (note, I did not say causality) of an intervention, but have virtually no relevance in the real world. Observational data, while being potentially generalizable, suffer from many other threats to validity. As you point out, we have very little mechanistic data, though I believe that understanding what happens between single cells or even in a cellular matrix in a lab is a far cry from what occurs in the whole organism.
To me the issue is that we have colluded to obscure these questions with our conviction that we have found the answer that fits everything — EBM! What both you and Ben Goldacre offer are interesting glimpses at various problems with pinning our hopes on a single way of looking at things. In my view the debate needs to ask many many broad questions like the one you have asked. Does EBM give us what we need? When does it? When doesn’t it? Why? I am afraid that there are no simple and catchy questions or answers here, and reducing it to a single problem is misguided. A satisfactory solution will not appear unless we have the courage to ask difficult questions even at the risk of losing our celebrity.
My general rule as regards moderating is that everyone should be allowed a comment – but ad hominem attacks aren’t helpful. John Stone’s comments above are very detailed and his suggestion that he has been denied the opportunity to raise issues makes it very difficult for me not to let the comment stand.
There have been other comments where people have claimed that they have been denied access to other outlets such as BMJ however that I have not posted pro temp. There have also been comments like those on a previous post from Arthur Topham which were followed up with a comment about conspiracies that seem to me likely to derail the conversation from the issues.
For the record, ad hominem arguments rarely shed light on anything. Aside from Robert Gibbons, there are few people listed in these columns who have more links to, or prior consultancies with industry than I.
Andrew Wakefield notwithstanding, most of what we know of the hazards of treatments comes people working in or with close links to industry.
Arguments about conspiracies also rarely shed light. It is not surprising that Andrew Witty has close links to government and advisory bodies. The way the system is set up makes this entirely logical and his input to government while shaped by his position is likely to be in good faith.
Talk about the system is not talk about a conspiracy. A system is a set of legal and bureacratic arrangements that are there in the broad light of day for anyone to inspect. They have been put in place by people with good intentions hoping to get as close to a Win-Win as they can.
The challenge for those of us injured or disadvantaged by the system is to point how to improve on the current arrangements and if a sufficient number of parties stand to gain there is a chance of change.
This holds true for Masons, Communists, Germans, Jews, the British establishment, Free-market devotees, or whoever. Life cannot be lived without banking regulatory health and other systems in place – human wit can modify these – and sometimes with the best of intentions can make things worse as the Tragedy of Lou Lasagna and subsequent posts will attempt to show.
David
Let me make it clear that I am talking about the system, and I dislike ad hominem. In the case of Ben Goldacre I have consistently tried to engage him in the public arena over the way he has represented the science, but also it seems to me that consistently – and certainly on this question – he simply represents the people who disagree with him as being defective (bad people), rather than addressing the problems. He is also a very influential voice within the media, and I do not think it is less right to ask questions than if he was a politician or a captain of industry. I would been much happier if he had acknowledged the questions about the rather inconsequential epidemiological studies he cited in support of MMR safety, or the right of individual children to have their symptoms investigated, not least if they have had an adverse reaction to vaccines (or are they simply exempted from scientific investigation?) But I don’t think this is essentially different from asking why it comes about that LRB did not publish your review of Goldacre’s book (which in itself is very bothering).
Leaving Goldacre aside, I don’t think it is surprising that pharmaceutical companies will ruthlessly defend their products or governments their policies, and you may have trouble getting heard if your child has been hurt. Vaccines are, of course, different and tend invert the usual ethical issues in medical treatment. The ideology is totalitarian, and in this country the policy mostly totally oblivious – most years no one gets compensated and no one ever gets within a mile of suing the manufacturer, which means there is zero sanction on them to get it right.
I think we need to be clear that what we don’t want is a perfected system for steam-rollering the public over medical issues. If I mention that Witty and Goldacre get heard in government it is by contrast with the fact that ordinary people don’t, and in the view of the “science” lobby shouldn’t. Their experiences don’t count, and this is another instance.
I am a medical anthropologist interested in the issues debated here, especially from the perspective of those who have been harmed by exposure to statin drugs. I can offer a concrete example of what David is talking about with regard to the reification of the RCT and the Cochrane Collaboration.
The 2013 update Cochrane Review of Statins for Primary Prevention (Taylor 2013) offers an important example of how the hegemonic role of the RCT can steer analysis away from harms and to a sacralised devotion to “The Trial” as the only legitimate form of evidence. The review contains the statement that the authors have examined “the totality of evidence” (page 15) as the basis for their recommendations that statins are effective and safe for primary prevention use—a very controversial position—not shared by many other independent analysts. (See inter alia Therapeutics Initiative #77; Ray 2010 and other articles in Archives of Internal Medicine June 28 2010; Redberg in various debates with Blumenthal, etc) Most recently there is a debate in France contesting the cholesterolization of heart disease and the statinization of the cure with the publication of Philippe Even’s book “La vérité sur le cholésterol” with discussions in Nouvel Observateur and Le Monde.)
But the key point here is why did the Cochrane authors call the meta-analytic process they undertook “the totality of evidence?” They could have said that they had assessed a group of clinical trials that met a variety of relevant criteria and that that evidence led to the conclusion they reached. But from a narrative/cultural perspective we see that they believe that what they are doing is all there is to do. It is “the totality.” This phrasing, I assume, cannot be understood to be accidental, but rather an indicator that the clinical trial meta-analysis model is ALL the evidence that one needs to understand benefit of a drug intervention and more to the point, drug harm.
But it also seems to mean that for some reason they are not working within the same data transparency model that Goldacre advocates to find the harm data (or overly exuberant assessments of benefit) within the trials they are assessing. This I cannot explain. Why is data transparency not an issue for these authors? Other Cochrane stalwarts are doing everything they can with regard to Roche and the release of Tamiflu data (e.g. Doshi and Jefferson) but the Heart Group seems okay with not pushing the trialists (these are all industry funded trials by the way) i.e. the Collaboration known as the Cholesterol Treatment Trialists (CTT) to release the Serious Adverse Event data in several trials. Many other researchers have asked for SAE data and been rebuffed (including Criqui and Golomb 2004; Walsh and Pignone2004; Petretta 2008 among others.) It is not clear why this particular Cochrane group seems to have accepted the industry position that their own (by CTT) analysis of primary data was sufficient without at least engaging in a debate about making those data publicly available.
I think it would actually be helpful in building an overall understanding of statin SAEs to get this information. So I am not opposed to Goldacre’s free-the-data perspective but really don’t know if this is where efforts to understand statin harm should be concentrated.
If David is right then this could wind up being a giant waste of time.
This is a crucial strategic question, especially for those who have been harmed by statins (and their caregivers) and whose doctors refuse to accept “the evidence of their own eyes.” Will freeing the data actually help? This is where the differences in perspective between Healy and Goldacre will be played out ….and this is where the limitations of the model of “the data” as an independent, free-standing source of truth will be tested. How quickly will truth be converted into “truthiness” by industry PR narrative structures that will find a way to controversialize, dismiss or reshape “the data” that points to harm? (See for example WHI trial as discussed above by Joanna.)
So to conclude, the Cochrane analysts have produced a product which is exactly aligned with the industry position as David predicted. And not because they are in COI position, one has to assume, but because they see no other framework as meaningful. They considered very little—a nod towards the big data work of Hippisley-Cox—outside the clinical trial, and from my perspective they didn’t ask enough tough questions about concealed data. But mostly its seems they wound up with a position that serves industry interests because their devotion to the reified, sacralised RCT led them to “the totality of evidence.”
*-*-*
I have just started to read a new book by medical anthropologist Joseph Dumit called “Drugs for Life: How Pharmaceutical Companies Define Our Health” which I think will be an important contribution to the discussion on this site.
David – regarding your comments to John, this article puts it in a slightly different way:
http://onlinelibrary.wiley.com/doi/10.1111/eci.12074/full
Perhaps Harriet says it in another way too, that what a person considers truth is defined by their own myopia.
You say, “Talk about the system is not talk about a conspiracy. A system is a set of legal and bureacratic arrangements that are there in the broad light of day for anyone to inspect. They have been put in place by people with good intentions hoping to get as close to a Win-Win as they can. ”
The bit about good intentions is an assumption, IMO. I know too many medical system whistleblowers to believe that.
People like Ben Goldacre, and his supporters, never write anything that doesn’t result in a win/win, because what ultimately matters to them is maximisation of the financial gravy train. They all try to run with the hares and hunt with the hounds, to maximum results from “both” sides, so will do their best not to bite either side – except where the gains from the system outweigh the opinions of the other audience.
You also say, “The challenge for those of us injured or disadvantaged by the system is to point how to improve on the current arrangements and if a sufficient number of parties stand to gain there is a chance of change.”
It certainly is a challenge. And often it’s futile, because any gain to us, which would also be more than a significant loss to them is defended to the accountant’s hilt. What matters to them, is “loss of reputation and goodwill”. Any truth which might result in that, will be ruthlessly fought as either fraudulence (even when it’s not) or defamation (even when it’s not).
Furthermore it’s not easy to take on what is obviously a lie, when the public has been more than hooked in for over fifty years. Particularly, for instance, if it’s a very large company, and a very popular baby OTC product. http://www.beyondconformity.co.nz/BlogRetrieve.aspx?PostID=60258&A=SearchResult&SearchID=5436410&ObjectID=60258&ObjectType=55
Been there done that. I also believe that the reason that the company ran a mile was that the documentation from the medical literature, which most doctors have no idea about, was so thorough that they saw the writing on the wall. So they blustered then really fast quickly flicked the product to an ignorant company.
The problem is that to this day, doctors and people do not understand that the evidence against this sort of thing, are out there in the open, and often in the medical literature.
The result is that anyone selling products like the example above, is that the manufacturers make two killings. Financially by selling the initial products, and a second financial windfall selling the medical procedures and drugs used to give the person a better quality of life. Even better, they often do that, while leaving the person totally ignorant about the fact that their product was responsible for the damage in the first place.
This is the sort of WIN/WIN that drug companies love, and Ben Goldacre is the ultimate pawn, because the bottom line is that he will write enough “bad” to keep the people asleep – thinking that someone is on the case, yet on the other hand, write enough “good” so that people don’t ever see the real issues.
That’s why he, and his books are so dangerous. They tell a version of “truth” but the things he doesn’t tell about (sins of omission) – if he wrote about them, would completely recolour what he does write about, so as to tell a completely different story.
And that ultimately is the basis of both deception and “conspiracy” inflicted on the public, which underpins the whole medical system, including the FDA, CDC and manufacturing revolving door policy. What might be euphemistically called the “Delicate Fabric of Collaboration” http://www.ncbi.nlm.nih.gov/pubmed/9411380 which in reality is a very sick joke.
The legal and bureaucratic systems which underpin this, are designed like a colander, and the companies make maximum use of bureaucratic loop-holes which often make the rules and regulations redundant.
That’s the worst thing that people don’t realise because they believe that the regulatory “industry” has their best interests at heart, when that’s the furthest thing from their minds.
As previously said in the comments section by Ben Golacre, “Bad Pharma is an attempt to give a straight explanation, for the general public, of various well-documented shortcomings in trial design, trial reporting, and evidence dissemination.”
In this regard, I think Dr. Goldacre has done a great job in engaging the public at large with a very informative and engaging book covering very difficult scientific issues related to the medicines ordinary people take everyday.
I don’t think that anyone would seriously think Goldacer’s Bad Pharma to be an academic treatise on these issues.
On the other hand Dr. Healy has written many books, covering these same issues, that are rightly considered academic works, although as such they may not eventually be as widely read by ordinary folk.
In the end it is about public engagement in the demand for safer & effective medicines. The industry side is well coalesced – time for the advocacy side to do the same.
I agree with you from a patient’s standpoint. Sometimes you need someone who will inspire regular folks about the issue. Us regular folks are not going to be able to understand the minutia of all points of the argument but they will get the point of the bigger picture. I did. Getting more regular folks to look closer only puts pressure on the political aspect of this. That helps. I also do think having been duped by big pharma with Paxil and all the health issues that have shown up afterwards. I don’t need a research paper to confirm that. I already get it.
I now don’t trust physicians and drug companies and big food. That is the long and short term consequence of this. We spend 1K a month through employer for healthcare and do not use it. I would like to be able to pick a cheaper Obama plan through my employer rather than the two they have. I can not justify the high cost of healthcare to the quality of care I have gotten. It’s like a run away train and I have gotten off. I get solicited from our healthcare to use it and since once they get you in the door they can charge for other stuff. So I have limited their profits as much as I can. We take zero pills and supplements. My medicine cabinet is bare too.
What I was doing is following research papers to see if I could find answers. But Ben educated me to the pitfalls of the lack of information I can glean from. That is big too. Ben doesn’t have to answer it all for me. The fact he has clued me into what I may be missing is a lot and enough for me. If he just does this, that is really wonderful for everyone. No one can do it all and you have to specialize and keep the message clear and simple for regular folks to grasp and run with it.
My husband is a educator and teaches critical thinking and I educate family members too. These drug companies may have won the battle but I do think folks who have been burned by these drugs and physicians will remember and move more and more away from traditional medicine.
Hense all these chiropractors et al selling their snake oil approaches in this every widening void for regular folks for health care. They are gaining ground since folks are fed up with the current healthcare offerings. Not great but if they are smart enough to check these guys out then they will be smart enough to figure that doesn’t work either and keep searching. The change is beginning albeit slowly.
I still see a need for emergency care but that is about it. All those specialists I saw was a joke and they actually made me worse or had zero answers. Seriously are they serious? We may be simple folks but we know when something is working or not or getting worse. Sheesh.
I think this problem not new. We have know for many many decades how poorly we approached medicine. Vaccines and antibiotics was a big trade off with health. The focus was that path and chronic illness is the norm now.
I do not see a big change in my life time and I am 56. But I do see a shift in it and we all know it takes a long time for big changes to happen. I do think Paxil does more than we realize and I doubt the studies have been conducted or ever will be conducted to prove to me many chronic illnesses piled on due to this drug.
The HMOs who used this drug to keep patients out of their waiting rooms bet on short term profits and long term they are losing business in my area. They are begging us to come back and over my dead body. What is different now is the fact we can communicate all this online the word is spreading faster and faster and regular folks have access to others who are in the same boat. That I believe will be the game changer here. Guys like Ben are using it very effectively.
There are millions and millions of folks who have been harmed or family member or friend harmed and these drugs companies are going to have to spend all those profits just to try to get us to listen again. It will be futile for sure.
I am a retired doctor who for the past ten years has been delivering a post graduate clinical research training program in Canada, and I would like to pick up on Harriet Rosenberg’s “totality of evidence”.
The ICH-E8 defined the purpose of the RCT as ‘Therapeutic Confirmatory’; however there seems to be an ongoing controversy about exactly what it is that the RCT might best be suited for, as opposed to what it currently seems to be best at doing. There are few shared views on either point, added to which there is a significant – indolent – variance among the medical specialties as to whether the RCT is of any use at all as a testing device. And of course within a given specialty, exactly where it might best be applied. Seemingly psychopharmacology, for several reasons, may not appear ideally suited for RCTs (Whitaker and recent posts).
Too many bosses use the RCT for too many purposes, many of whom appear to operate in less than transparent environments where RCTs are both covertly and overtly manipulated, distorted and hijacked in the horse race to the finish line.
My own concern goes a bit deeper. What is meant by Harriet Rosenberg’s “totality of evidence” (Cochrane and statins), and Cochrane’s seemingly “… sacralised devotion to ‘The Trial’ as the only legitimate form of evidence”? Is the word “evidence” in fact masquerading for “efficiency” or more precisely for “efficacy”? If it is we’ve got a problem.
The trials that Cochrane chose for review will have met a diagnostic set of tick marks, but regardless of the selection criteria, the concern must surely be that, rather than the review merely “sacralising” its own review method as the ultimate yardstick, that it should do so by means of a preoccupation with demonstrating efficacy. If this is the case it looks as if the hegemonic role of the RCT is indeed being endorsed by Cochrane; ergo one may conclude that the systematic review rudder is steering the Cochrane ship round in circles while it attempts to evaluate increasingly questionable measures of efficacy. So much for efficacy; where does that leave safety?
Safety, it seems, cannot be estimated from RCTs. The RCT cannot be trusted for detecting or documenting SAEs and with few exceptions (Goldacre, arguably) the RCT should, rather than be seen as providing “serious evidence”, vide supra, be seen simply as a conditional seal of approval for a drug; a “half-truth” as it were, on efficacy only. The RCT would be relegated to an efficacy test with poor sensitivity AND poor specificity, for the purpose of providing some reasonable assurance that a drug is probably safe for marketing. If this sounds familiar it’s because this is what we have now: new drugs being released for what is in fact test marketing, an untenable form of commercialization where there are no tests and no scientific testing of marketed drugs.
If the current low bar setting for marketing approval is here to stay, then the requirement that some erstwhile statistically significant difference be demonstrated is also here to stay, in order to close the deal and market the drug. But current marketing approval rules may be a bit of a red herring since at that point safety issues are invisible and cannot be engaged, while worrying about safety is becoming a far greater societal concern than is efficacy.
The way ahead for safety will eventually be achieved, if it’s going to be achieved, through remote reporting of side effects. This can only happen over an extended post-marketing and open-ended period perhaps taking many months or years, using petabyte mainframes with minimal human involvement – ongoing and for the foreseeable future. Any newly marketed drug would be assigned a ‘contingent’ or ‘conditional’ code – while fully marketed – until sufficient post marketing evidence is gathered for it to be labelled as safe, – or alternatively for it to be dealt with in some other way. The FDA currently collects gross safety data which RxISK uses as well.
In the meantime some of us will wait to see whether having bet the safety farm on a witticism and its benign cartwheel moment is going to make any real difference. More likely that particular moment it is headed for the MOTS bin along with the other false CSR hopes and broken data dreams. I would be happy to be proven wrong. MOTS by the way means more of the same.
A box of tricks with a piece of paper inside is no Guarantee.
A new telly with a piece of paper inside has a Guarantee.
Every aspect is covered with a Warranty. All safety aspects, probably dissected by a lawyer, whereby the manufacturer is covered in case of ‘fault’.
In the case of a cardboard packet, containing capsules, with a leaflet inside, the advice is transferred to another party to deal with.
With my new telly, I got a Guarantee.
With my new pill, there were no guarantees, except for the possibility that ‘Curry’s’ might send me an engineer……or they might not…….but meantime, I have self-imploded……by default…….not my fault.
Michael Heseltine said about Tridant, our nuclear submarine and the debate about whether the UK needed this ‘expensive’ weapon.
“There are no guarantees that we will not be attacked and the people of Britain have a right to speak up about their defence.”
This is our right to want a ‘guarantee’ about little pills in little boxes…….or if a ‘guarantee’ is out of the question at least a chance of a chat about why the guarantee is not available………..
GSK will not talk to patients.
They avoid all dialogue.
They come out with Witti cisms.
They are not credible…..I want to talk…….about it…….they won’t talk about it……
Platitudes wear thin……
A Guarantee on your Telly does not mean that it will work fine. Usually the guarantee is that it will be fixed or replaced within the time of the Guarantee. Beyond that, manufacturers feel you already got your money’s worth. There practically is no more quality control in most products. The consumers are the ones conducting the “quality control”. It saves the manufacturer not to have an elaborate Quality Control department. Their only Quality Control test is at your home. And they call this risk assessment: which will be costlier replacing failed products or conducting in-factory rigid quality control.
Same could go for medications: Is the number of deaths due to the medication acceptable statistically? How much would a lawsuit of wrongful death cost compared to not marketing the product till all tests are in? No amount is acceptable for those whom the medications failed.
Dear Dr. Healy,
This particular blog, the email comments following it, and the willingness of the professionals and the public posting here to say out loud and discuss what most will not in such a public forum, is among the most riveting and useful of exchanges that I have ever seen on the Internet. This kind of dialogue between ALL parties involved in patient welfare is exactly the kind of involvement that is needed in order to keep the eye on the prize to “first do no harm,” a largely forgotten concept in the medical profession in the United States as it is today. Please keep talking and please keep it public. Those of us who realize we must be our own health care advocates are starved for reliable information. The differences of opinion and back-and-forth requests for more elucidation are what my gut tells me is missing in profit-driven science and medicine. Although there are many of us out here who are not the scientists and experts that you and your colleagues are, we can provide a missing piece of the puzzle – the part that requires a doctor to listen to our stories and respect them as an important and necessary piece of treating the entire patient rather than just one part. Among the many things that can be found here, from a patient’s viewpoint, is the possibility for restoration of a patient’s sense of dignity that is very often denied during an illness. A placebo effect by asking what we think and what our experiences have been with prescribed drugs, including reporting adverse events and side effects? I do not think so. This site is a sanctuary for many who have no other place to go and have found a sense that there are actual people here who will respond with care and who have the burning desire to find real medicines based on real and unbiased science and who also recognize our self-reporting as an important part of the ongoing process of treatment. This struggle will continue. Thank you so very much.
David,
Because I find the matter so important for me as a child psychiatrist, and because I found your review so articulated and deeply thought of, I went over the book a second time around. I have the following to offer as a contribution to the discussion:
First of all, I fully agree with you that this book strikes the right note of outrage and is accessible to anyone interested in health and politics. It has added many facts and figures to my previous knowledge of this scene, some of which I found most interesting and useful.
But I have to agree with you that no satisfactory plan of action comes out of the book though one finds throughout the book many a “What should be done capsules”. Some of the suggestions are very well taken from a rational point of view but will remain barely efficient on the field. For example, in a situation where the access to a family doctor is difficult (our present situation in Québec) you cannot imagine the patients easily challenging their doctors about their links to the industry. The important point you raise is that these invitations to action are all fueled by the idea that Pharma is Bad and that shooting on bad Pharma will straighten things out. This leads us to realize that the problem of the influence of industry on modern medicine is a multifaceted pervasive problem. How do we solve a multifaceted pervasive problem ? It reminds me of the kind of problems I have to deal with on a daily basis with my patients. Like it of not, human behavior, be it from a personal or societal view point, is a super byzantine entity. We all agree there is not “fit them all solution” . Where should we put our energies in this complex system where there are so many groups, and powerful ones for that matter, taking advantage of the present situation ? I thought, and still think, that we should talk and wake up people around us about what is going on. So I started talking (few colleagues come to my presentations) ; I wrote a book freely accessible on the internet (apart from a number of my students, few colleagues read it) and so on.. You once told me something like we were beyond writing books and you went into action with RxISK.org. I thought talking to the doctors would be the most efficient way to go. Probably wrong. I have been pushing for four years for our students in medicine to hear, at least once a year, from the very first to the last year, about the influence of the industry on modern medicine. I finally succeeded to give a lecture on the topic to the third year students, then, only this present year, to the second and third yr. I wanted to hold a half-day forum on the topic to an educational encounter among us at the Fédération des Médecins spécialistes; it was turned down. I have known today that my offer to hold a forum on this topic at the 47th Congress of the Association des Psychiatres du Québec was also turned down. It obviously raises the question as to what is really being turned down, the message or the messenger. My (?? biased) hypothesis: essentially the message; then the messenger will keep on going.
One of the ideas that kept my attention in Ben’s book was to carry out ongoing trials in our clinical settings (end of chapter 3) but you inevitably have to convince a majority of doctors. Once again probably better go directly to the patients.
How did the Inquisition came to an end ? A Spanish monk, Alonzo Salazar y Friar, put forward what is obvious for us today namely that the accusations had been false, the confessions induced by torture, which meant there was no credible evidence of witchcraft at all; this was the beginning of the end, implying a different rule of evidence, namely that the accusations had to be supported by independent evidence. “ This was the recognition that unstructured “facts” can assume perverse and essentially meaningless forms (W.C.Clark 1981)”. Think Tests of significance, so pervasive in our clinical trials. It looks like “ it makes sense to us doctors when Fisher tells us it is only when there is bullets in nineteen chambers of a twenty-chambered gun that the gun can be said to be loaded” (Pharmageddon chapter 7). I think this image could have the power to slowly induce a transformation in our actual rules of evidence in modern medicine, because it wakes you up and kicks your neurons around in front of a familiar situation where they usually keep on sleeping. And I tend to agree with you (if I understand you correctly) that as long as our clinical trials are driven by this kind of rule of evidence, as presently is pervasively the case –doctors, researchers, regulators, drug sellers, lawyers etc…- the system is at a standstill. This important point is missing in Ben’s plan. I wonder what he would have to say about this.
Jacques Thivierge
Laval University
Québec,Canada
dear David
this debate leaves me a bit confused. what to actually believe and what not too. guess that is better than blindly going with EBM. a wise proff recomended us to wait for more studies and experience before changing practise.
regards
thambu
Thambu,
there is absolutely no way that you can follow EBM blindly: you have to learn it to feel it healthy core: it’s method is open and explicitly based on SOUND (and not on any) research.
Pharma is indeed for profit (like any industry), so is medicine in many countries without publicly funded healthcare systems. It is needless to say that both they try their best – either to make better drugs or to treat patients: but for profit.
I am fortunate to get re-trained and work in healthcare system in Canada, where EBM was conceived. It is unfortunate to see that the governments are willing to spend millions on controversial programs like H1N1 vaccination having only evidence own by industry, which is resistant to disclose the details of their own research of vaccine efficacy. It is indeed sad and very much concerning story that the money almost always govern medical decisions, not the best compelling evidence. This is why I belong to EBM and hope YOU will .