Marilyn’s Curse

May, 14, 2013 | 6 Comments


  1. Dear Dr. Healy,
    Interesting post. It appears to me, if I am not misunderstanding your point, that you are arguing against RCTs based on our inability to differentiate between medication adverse effects vs. under-treatment of underlying illness when the observed outcomes are similar.

    However, our drug development system starts with Phase 1 experiments with normal patients to establish safety threshold of dosages for investigational new drugs. As you briefly alluded to in your argument, one can only see adverse events in this group of individuals because there is no “efficacy” to speak of in normal people. Phase 2 trials establish dose response and appropriate dosing that provides the greatest efficacy, which is then carried out Phase 3 trials. Perhaps we are too fixated on Phase 3 studies while ignoring Phases 1 and 2, but if these earlier phases were carried out properly, there is no reason to believe why Phase 3 results will be unreliable. We need to respect the Bradford Hill criterion of dose response. Differentiating between adverse event vs lack of efficacy (due to inadequate dosage) when they produce the same outcome can be solved by increasing the dose (assuming we begin at the minimum effective dose) and evaluating the dose-response curve. If there is a true adverse event, with increase in dosage one sees a persistence or worsening of outcome (aka adverse effect). If there is lack of efficacy due to inadequate dosage, with increased dosage one will see improvement in outcome when you increase the dose.

    At any rate, the Christmas light doesn’t solve the issue – for example, if one stops an SSRI in a patient and they commit a self-harming act, there is no way to differentiate whether the act was due to residual adverse profile of the drug, the patient convincing themselves that they feel worse because of the placebo effect, or a relapse of depressive episode.

    I am not sure what the tables on paroxetine and suicide are trying to illustrate – we all know that absolute risk differences provide more objectivity than RRs, and it is rather disappointing to see this not reported in the tables. Once this has been incorporated, it will become clear that the suicidal effect crossed to paroxetine’s favour because the placebo group had a higher suicide rate than paroxetine in the RBDD trial in Table 2, in contrary to the MDD trial. Further, the absolute risk difference of RBDD trial (in favor of paroxetine) was larger (1.4%) than the MDD trial (in favour of placebo, 0.37%), so the results are not surprising. Yes, it is possible to “dilute” the absolute difference between two groups by adding to both numerators and denominators in both groups, but you also lose statistical precision – I am not sure why it is provided only for the first column and not the rest – they are likely non-significant. If that is the case (where an intervention arm is not proven superior to a placebo), I don’t see how a new investigational drug will be approved. At any rate, if one attempted to combine different patient populations into a single trial, one would be able to see this in Table 1 and cringe.

    Your article does not allow for a genuine dialogue on EBM by unnecessarily debasing RCTs, pointing out their roots in agriculture as if nothing from that industry can be trusted; well, I consider it fortunate that their origins did not involve Christmas lights.

    • Yan

      Many thanks for taking the time to read and comment. First, almost no independent investigators run healthy volunteer studies – which are perhaps the only trials that should be called drug trials. The phase 1 studies industry do remain unregistered with total lack of access to the data from them even though there are no issues of confidentiality. The events that happen in condition trials are not collected systematically in part because the primary outcome of a condition RCT is the benefit and its not possible to capture all events in detail – but also companies to some extent know what other events may be happening and should be downplayed. This leads to a poor recording of the vast majority of events that happen on treatment. Companies also get away with claiming that these events are not statistically significant and hence didn’t happen when they should not be able to apply statistical significance testing to such data and any increase in the number of events should be regarded as a true increase until proven otherwise.

      The findings here don’t depend on a use of relative versus absolute risk. They hinge on a poor understanding of the clinical conditions being treated so that the patients entered into trial are highly likely to be heterogeneous. Where randomization can in some cases help us manage clinical complexity, in a number of instances it can do the opposite and the point behing Tables 1-3 is to show how easily this could happen in practice making it almost inevitable that it does happen to some extent.

      Re Christmas tree light bulbs (CTLBs), this is not set up as infallible. The argument is that RCTs are not infallible either and in many instances are less helpful than CTLBs. In fact the antidepressant and suicide story is all about how companies were able to convince academia the media, the proponents of EBM and everyone else that shoddy RCT data on antidepressants and suicide trumped convincing CTLB reports.

      In terms of the issue you mention though a CTLB approach does better than you seem to think – in that one of the options for suicidal acts on stopping treatment is dependence on the drug being stopped and withdrawal suicidal acts. If the agitation clears quickly on reinstating the drug this is most likely to be dependence and withdrawal related. The CTLB works well here.

      We are not talking about rare events. The agitation/anxiety that results in suicidal acts on antidepressants occurs in up to 20% of patients put on them. Over half the people put on antidepressants will have significant withdrawal problems when stopping.

      Its easy to blame pharmaceutical companies for all that is wrong in clinical practice. We really should also scrutinize our own instruments and practices. We need to ask whether just like screening RCTs have the potential to cause more harm than good. I think the post shows how this could happen. This doesn’t mean RCTs don’t have a place.

      A second point is this. Where exactly have the advocates of EBM and RCTs been when it comes to the issues of adverse events? None have been involved in the antidepressants and suicide story, or the adverse effects of statins story.

      But maybe there is one point of agreement. I think RCTs are useful (not infallible) when they show a drug hasn’t got a claimed benefit. In this case the onus is where it should be – on companies to find some patient group who may benefit. In practice because we fail to recognize the limitations of RCTs we have a world where patients are injured because of RCTs and the onus is on them to prove the drug caused it, and in this context the voluntary efforts of others to do something good for their fellow citizens becomes the means by which patients injuries go unrecognized and unaddressed.


  2. Sorry – two last points after watching your Nov 2012 talk at Cardiff:
    1. I feel the need to reiterate that RCTs as they are currently designed do not have the power to detect rare adverse events – this has been conclusively shown: Again, detecting the rare adverse events is best done in the post-marketing pharmaco-surveillance, unless we are willing to accept a system whereby we randomize at least 100,000 people for 5 years for each drug before it is considered for approval.

    2. When the condition that the drug prevents and its alleged adverse events result in the same outcome, one either needs to look in normal controls (as per my comment above), or one clearly needs to look at the composite of the two. Going to your imipramine example, it is illogical to conclude that imipramine causes suicides when it prevented more suicides than the ones it “caused”. One can just as forcefully argue that depression’s adverse effect is suicide.

    By your logic, I can say that warfarin causes ischemic strokes because patients with atrial fibrillation who are on warfarin have ischemic strokes – no, warfarin PREVENTS strokes, but not every one. Not treating patients with warfarin also causes strokes. The only way do prove warfarin’s efficacy, indeed, is through a placebo-controlled study. Similarly, when imipramine prevents more suicide than the ones it supposed “causes”, then it is an anti-suicidal agent by definition.

    • Yan

      We are not talking about rare adverse events, we are talking about events that happen in up to one in every five people taking these drugs. These events happen in healthy volunteers as well as depressed patients.

      Maybe I failed to make the imipramine point fully clear. Put imipramine into the populations SSRIs were trialled in and it too will show a doubling of the rate of suicidal acts compared to placebo. I can construct trials of imipramine to give relative risks varying from 0.5 to 6.0.


  3. I’m imagining a worried parent taking her six-year-old son home from yet another trip to the emergency room. She believes she knows what the problem is: strawberries. Every time he eats them, he breaks out in a rash and his face and hands swell. Never happens otherwise. This time, after the class trip to the U-Pick Berry Farm, his throat swelled up too. My god, he could have died! She’s taking him back to that wretched “Evidence-Based HMO” and telling them what’s what.

    But the doctor still won’t provide a “no strawberries” note for the child’s school. Worse yet, he tells the mother, “As long as there is even a six percent chance that you’re wrong, I want you to keep feeding him strawberries. They are a very good food, full of vitamins and absolutely fat-free. I have many children in this practice who might be living on chocolate milk and Cheez Doodles if it weren’t for strawberries. They won’t eat apples, you know. Listen, these problems are not as rare in children as you might think. I had another boy in here last week in the same condition. And you know what the dad thought the problem was? Peanut butter. I kid you not. Parents have all these interesting theories, but they’re just anecdotes.”

    “Most likely your son has a chemical imbalance – his histamines just run unnaturally high. Luckily, there’s some really powerful medications we can put him on. If he takes them for the rest of his life, he will be able to live almost like a normal child. And we can always add a stimulant if he gets too sleepy on the meds.”

    Maybe the only reason this hasn’t come to pass is that GSK doesn’t have a patent (yet) on strawberries or eanut butter? Give them time. They have a patented prescription-only fish oil called Lovaza, and it’ll only set you back about $200 a month …

    Thanks for helping us see the madness.

  4. Heally, do you have any information on whether or not the general public is catching on to how drug companies operate in such a shady way? I know many people know this, but I’m wondering if this knowledge is being learned by more and more as time goes on?

    What I’ve seen for the most part on the forums I go on is that many of them think severe, permanent effects of psychiatric drugs are only slightly less rare than seeing a unicorn. I know this because I will tell how these meds have affected me, and they tell me I must be mistaken. They say this is next to impossible. I ask why they feel so strongly about their idea of it being next to impossible and I get the same reaction every time. They say “well it hasn’t affected me like that”. I guess their logic is something like, “well, its not raining where I’m at so it must not be raining anywhere else.” I ask for them to share facts behind their ideas and they either point to a sensational article about antidepressants or they just say “its common sense.”

    They’re so mislead.

Leave a Reply