The Best Bias That Money Can Buy

Randomized controlled trials (RCTs) were adopted by FDA in 1962 following the thalidomide disaster. This was a way to manage the risks posed by potential poisons. If the toxicity from a drug could be shown to overcome to some extent the toxicity stemming from the illness, a risk-benefit ratio would be set up that would warrant taking the risk of giving the poison.

But what happens when both the poison and the disease produce superficially similar problems – when both an antidepressant and depression produce suicidality for instance? In this case there may be almost nothing that can be done to persuade people that it’s the poison and not the disease producing the effect. Even people of goodwill may look and look at the data and still not make the connection, as with the Shepard illusion (demonstrated in my previous post The Spin that no Data can overcome).

In the case of the antidepressants, given the controlled trial data they were faced with, the FDA had little option but to concede that the drugs caused suicide. Despite this and the Black Box warning put on the drugs, many (maybe most) doctors continue to believe that the FDA only did this because of political pressure from groups like the Church of Scientology.

The only reason the problem showed up in the RCTs

Given a slightly different turn of events, it would have been possible to hide the problem forever – using controlled trials to do so. The only reason the problem ultimately showed up in SSRI RCTs was because the SSRIs were so weak they could only be given to people who were at almost no risk of committing suicide. In this group, the increased risk from the drugs became obvious.

Had the SSRIs been as effective as the older tricyclic antidepressants (TCAs), they could have been put into trials of more severely ill patients in which case the risk from the disease would have been greater and might have been greater than the risk from the drugs.

When in clinical trials the risk from the drugs is greater than the risk from the illness, the relative risk is greater than 1.0. For trials of the TCAs or effective antidepressants given to severely depressed patients, it would likely have been less than 1.0 – even for a medication that caused suicide. The fact that the risk from trials was less than 1.0 – that is, less people committed suicide or went on to a suicidal act on antidepressants than on placebo – would have been taken by the field as evidence the drugs didn’t and couldn’t cause suicide (see Healy 2011: Science Rhetoric and Causality).

The field would have been wrong.

We know from meetings as early as the first scientific meetings held to discuss the effects of the first TCA (imipramine) in 1958 that clinicians using it had recognized it could cause suicide. These doctors could still believe the evidence of their own eyes; they were not inhibited by clinical trial data the way doctors are now.

The benefit of hindsight in the case of antidepressants and suicide

The broader message though is this: RCTs in principle cannot solve the problem of whether a drug causes an injury or not. They may in fact contribute to answering the question, but if they do, it will be as much by accident as design. In the case of the antidepressants, it was by accident.

With the benefit of hindsight we can see what has happened in the case of antidepressants and suicide, and could design an RCT to confirm the risks from the treatment. But what happens in the case of other problems that might also stem from an illness, where we do not have the benefit of hindsight? How many of these problems have been hidden from view by RCTs?

There are two ways out of this bind. One is to get back to referring to drugs as poisons. We have no difficulties in recognizing the poisonous effects of illegal or over-the-counter drugs, and great difficulties in believing they could be beneficial. This is just the opposite bias. This is exactly the perspective we need to bring to today’s prescription drugs, about which we often know much less than we know about yesterday’s prescription drugs (now over-the-counter or illegal). Prescription drugs are prescription-only because they are riskier than other drugs, not because they are safer.

The other option is to run RCTs of the drugs. Isn’t this what we do at the moment?

No.

The numbers will conceal the problem

At present our RCTs are of drugs used for an illness – or rather for a marketing indication. The illness backdrop hides all problems. Almost anything happening on the drug can be filed under the same codes as something happening in the illness. In depression, the drug may induce agitation; the illness does too. The drug-induced agitation may be much more severe than illness-related agitation, but companies will just present the frequencies of both, stripped of their severity codes, and the numbers will conceal the problem.

Worse again, the drug may prove marginally superior to placebo, perhaps only on a rating scale which shows a benefit because of a side effect – a sedative or anxiolytic effect – but this marginal benefit is taken as demonstrating the drug works and doctors are left with no evidence base for practicing the art of medicine – that is deciding when it would be best not to treat with an active agent.

If we want to design an RCT to reveal the effects of the drugs, the first step would be to run them in healthy volunteers. This would make it much easier to spot poisoning.

The immediate response of many to this will be that that would be unethical. Giving potential poisons to people who do not stand to benefit would not be right. Curiously, this mirrors the ethical dilemmas posed by the first RCT in tuberculosis patients where the concern was about whether it was ethical to give placebos to people who needed treatment.

Phase 1 studies are the most corrupt in medicine

In fact companies do healthy volunteer trials. These are called phase 1 trials. Far from being well-designed or informative, as one might expect in such an ethically fraught situation, these studies are the most corrupt in medicine. An industry has been created that takes poor people or students in need of money and pays them to be guinea pigs in such trials.

The trials are designed solely to suit company purposes, such as looking at the half-life of the drug, rather than designed to shed light on the range of things a drug might actually be doing. The data from these studies is never released, and the non-publication rate in these trials is much greater than the non-publication rate in clinical trials – which approaches 50% in some cases. Finally, where clinical trials in patients are registered these days, healthy volunteer trials are not – so no one knows what’s going on. Without access to these studies it is close to impossible to practice medicine – doctors just become conduits for distribution of drugs.

See Healy 2010 – Stress Syndromes.