The Valley of Death

Editorial: Part 3 of this Cochrane sequence follows on from last week’s Data Based Medicine.

While the histories of randomized controlled trials (RCTs) celebrate a first use of randomization in 1947, and there was some use of RCTs, primarily in mental health, in the mid-1950s, RCTs were a minority interest until 1962 and might have remained so, never giving rise to Evidence Based Medicine (EBM), but for an accident.

In 1956, no-one knew much about the limitations of RCTs or the harm they could do, when Louis Lasagna suggested that in addition to checking if drugs were safe, FDA might want – as a contribution to safety – to check and see if drugs worked. He had just come across RCTs, was enthusiastic about them and figured they might be a way to check for effectiveness. He repeated the suggestion at the Kefauver Hearings on the pharmaceutical industry in 1960. No one paid any heed to him.

In the wake of a public relations crisis triggered by “irresponsible” media reports in the US in July 1962, about a drug that had been shown in an RCT to be effective and safe, the US political class felt obliged to be seen to do something.  The doing involved adapting the wording of the 1938 FDA Act from an Act to ensure the Safety of Drugs to an Act to ensure the Effectiveness and Safety of drugs.

After some head scratching and political horse-trading it was decided with input from Lasagna that effectiveness would be determined by having two positive placebo-controlled trials.  The assumption was that a positive RCT inevitably meant any other RCT would be positive; this shows how little people knew about “RCTs” then.

The system that had failed to detect the problem with thalidomide – RCTs – was put in place and sold as a way to prevent future problems. Nobody involved in putting this system in place knew what they were doing – there was no conspiracy.  But once RCTs became the gateway to the market, those affected – industry – were incentivized to understand the new system better than anyone else.

Three Squeals

Three sequelae of this development can be noted briefly.  One is that it has led to large scale consumer “fraud” – it’s hard to find the right word for this particularly to cover the role of regulators in what happened. Since 1963 a large number, perhaps a majority, of drugs have been licensed on the back of demonstrations of efficacy where the statute calls for Effectiveness. They have been licensed on the basis of a change in a rating scale or bone density or lipid levels brought about by drugs which in some instances kill people early or disable them more than non-treatment would have. This is not Effectiveness.

A second is that as RCTs became the gateway to the marketplace, they began to be produced on an industrial basis – both in terms of quantity and quality.  The number of industry “trials” or as they have been sometimes called “assay systems” dwarfs all others.  Industry assays are conducted by assembly lines, involving CROs and ghostwriting, a process that Cochrane has helped facilitate with acceptance of industry assays as equivalent to medical RCTs.  If an assay/trial ticks the myriad quality control boxes Cochrane has helped put in place, then content and motivation are apparently irrelevant.

The third is that almost instantly – by 1964 – industry salesmen began pushing docs to prescribe according to the evidence. “The latest RCT shows this doc!” This has continued ever since with industry to the forefront of pushing docs to adhere to evidence-based medicine.

A Fourth Squeal

The fourth sequelum was linked to safety.

RCTs are the gold standard way to hide the adverse effects of drugs. All RCTs are dangerous as a result.  Some can also make a contribution to safety. This they do when they show treatments don’t work. A negative trial may get the “wrong” answer but it puts the onus back on those who want to profit from the misery of people at their most vulnerable to work out a way to identify who specifically might be helped by what is usually a MeToo drug.

Whether industry assays or medical trials, RCTs compromise safety by focusing on a primary outcome – does this drug “work” – rather than on the 99 other things that drugs can do.  This act of hypnosis leads industry RCTs to suggest that, for instance, 5% of people on an SSRI have sexual dysfunction when effectively 100% do.

But industry assays are additionally biased in that the primary outcome is something we can make money out of – a change in mood – while something that might be much more common – an effect on sex is ignored. Or the far more common effects of statins on muscles or cognitive function are ignored while the rare effects of preventing a cardiovascular event are focused on.  Or the possibly more common effects of HPV vaccines in causing POTS or other auto-immune phenomena are ignored in favour of the likely rarer numbers of women saved from cervical cancer.

Ignoring this ignoring, as claims about RCTs – they give us the most reliable information there is about drugs – do, turbo-charges this business interest.

Throw in some pixie dust – any side effects there might have been cannot by definition be statistically significant (statistical significance only applies to primary outcomes) – and the chances of an RCT making any contribution to safety completely vanish. Just to be safe, we can add some more pixie dust – any case reports of an adverse effect on treatment are just anecdotes, the lowest possible form of evidence.

There has, as a result, effectively been no adverse effect of a drug that has come to light because of an RCT. Doubtless some readers will be able to prove the rule by naming a few exceptions but its hard work to find them.

Until a recent belated involvement of Peter Gotzsche in the SSRI and suicide issues, and now Jefferson, Gotzsche and colleagues in the questions of HPV vaccine safety, Cochrane has made no contribution to detecting adverse events.  Forty-thousand people and 25 years later, I can’t name a single adverse event emanating from Cochrane, and no-one from Cochrane has ever taken a stand on safety issues until now.

Even Doshi and Jefferson’s justly famous analysis of Tamiflu trials centered on the lack of efficacy of this drug rather than its adverse effects.

Meta Analysis (space, no hyphen)

1/ As the detection of or engagement with an adverse effect runs against industry’s and medicine’s business interests, controversy is inevitable. Gotzsche and Jefferson’s engagement with the adverse effects of HPV vaccines and the resulting debacle brings this home.

This controversy has disrupted a vision Cochrane has sold – that the value neutrality of RCTs meant that the lion of science can lie down with the lamb of industry. This is not an option. Under the current rules of the game, it is still the case as Thorsten Veblen said over a century ago in the Theory of Business Enterprise,  that the pharmaceutical business needs to keep its scientists in check.

Under the current rules of the game, however, as high cost prescribers, doctors risk going out of business if drugs are wonderfully effective and free of side effects.  Nurses and pharmacists are a much cheaper, and a more readily forceable to be guideline adhering, option for governments, and managers.

2/ Beyond the structural features of RCTs noted above that pose a threat to safety, an even greater risk comes from the selling of efficacy. In a risky world, who wants to balance risks? Better to nuke ‘em.

The debate about guns in the US in the wake of school shootings brings the point home. We want to make schools safe. Guns are effective, why not have an armed guard in each school?  A lot of people figure this can’t be right but get stumped by the follow-up question – well if you don’t agree, do you think we should remove armed guards from the White House also?

Philippe Pinel offered the definitive answer to this question two centuries ago when he said (paraphrasing) that it’s great to have effective weapons but it’s even greater art to know when not to deploy these weapons.

It didn’t occur to Pinel to offer a view about a weapon that had efficacy (bullets come out the barrel but cannot be stopped from going anywhere even backwards?) but not effectiveness.  Readers are invited to offer an update on Pinel to take this into account.

As a result of Cochrane Based Medicine, we have lost the subtlety and nuance that Pinel put at the heart of clinical medicine. Cochrane have us put an armed guard at every school door.  It has institutionalized a split between effectiveness and safety that leaves us with no other way to grapple with the problem of school shootings.

The Charge of the Light Brigade

The headlines on BBC today are about falling British Life Expectancy.  All sorts of people are piling in blaming austerity and inequality.

Nobody so far has mentioned the fact that 50% of us over the age of 45 are on 3 or more drugs and 50% of us over the age of 65 are on 5 or more drugs – none of which are safe and very few of which offer an effectiveness to balance the risks we take in consuming them.

It’s a modern day repeat of the Charge of the Light Brigade.

theirs is not to make reply,
theirs is not to reason why,
theirs is but to take and die
into the valley of death….