The problem with drug trials

The problem with drug trials

Should randomised trials be the only type of evidence accepted for rolling out drug treatments?

If so, then two researchers wrote in the Lancet this week that that we face a problem:

The evidence we have might not be the evidence we need, and the evidence that we need may never become available.

They are writing in response to the publication of a trial of a new combination drug treatment for stomach cancer that seems to extend people’s lives by nearly 3 months.

According to these guys, there’s probably lots of effective combinations of current cheap drug treatments to treat cancer, but we’re never going to get the evidence to prove it.

The new trial must have been crazy expensive: It consisted of patients from 122 different institutions in 24 counties on 4 continents and, as the editorialists point out, it would never have been paid for had it not promised huge returns for the pharmaceutical company that funded the trial, Roche.

This raises a sticky issue. Drug companies like Roche produce treatments that save lives. But they don’t do so because they save lives. Rather, they produce the drugs because they can make the company money.

So what happens when these two motivations come apart — when drugs that could save lives don’t make drug companies money? That’s the problem that the Lancet editorialists are referring to.

If we presuppose that only randomised trials produce evidence of sufficient quality to support decisions about the allocation of scarce resources, there is a problem. There is a lot of evidence on the effects of adding expensive new drugs to conventional therapies, but little evidence for when older, less expensive interventions are combined.

And boy are these drugs expensive. In the same comment piece, the authors calculated that the cost of each year of life gained by this new treatment, is about $100,000 AUD ($85,000 USD).

The authors of the study argue that this should be rolled out as a “new standard option” in the treatment of gastric cancer — and that’s not surprising given that the manufacturer of the drug not only funded the trial, but was involved in the data analysis and editing of the report.

So how are we supposed to get evidence about cheap drugs that will help millions of people but not make money for drug companies?

I don’t know. The authors mention the model in physics where large projects that don’t have obvious practical benefits are funded by governments and research institutions. And they seem to imply that there might be a way of not relying solely on randomised trials — but they don’t say what that is.

Whatever the way forward, it seems crazy that in the mean time, we can only get evidence about drugs which, for the majority of people around the world, are prohibitively expensive when there’s almost certainly some great cheap alternatives right under our noses.

ResearchBlogging.org Munro, A., & Niblock, P. (2010). Cancer research in the global village The Lancet DOI: 10.1016/S0140-6736(10)61022-7

ResearchBlogging.org Bang, Y., Van Cutsem, E., Feyereislova, A., Chung, H., Shen, L., Sawaki, A., Lordick, F., Ohtsu, A., Omuro, Y., & Satoh, T. (2010). Trastuzumab in combination with chemotherapy versus chemotherapy alone for treatment of HER2-positive advanced gastric or gastro-oesophageal junction cancer (ToGA): a phase 3, open-label, randomised controlled trial The Lancet DOI: 10.1016/S0140-6736(10)61121-X

Jesus did (not) cure someone of the flu

Jesus did (not) cure someone of the flu

This week we got a quick and entertaining lesson in the way science sometimes works… or doesn’t work.

Just last week, someone asked me how to tell good science from bogus science. My first, not entirely adequate response, was that you should check whether it’s been published in a peer-reviewed journal.

There’s lots of reasons why that response is not completely adequate, and one of those reasons was highlighted recently by an article being published in Virology Journal about how Jesus cured a woman of the flu.

Within three weeks the article was retracted but the fact that it got published in the first place shows that peer-review does not qualify something as good science.

I won’t say much more about this, but instead direct you to some good discussions of the debacle.

- Ivan Oransky, editor of Reuters Health, has a good overview of the debacle on his new and noteworthy blog Retraction Watch.

- Bob O’Hara has a funny discussion of the original article in This Scientific Life, written before the retraction. The editor of the journal posts his apology and retraction in the comments to the post.

Can smoking pot make you smarter?

Can smoking pot make you smarter?

Is it possible that if you have schizophrenia, smoking marijuana will actually improve your cognitive performance?

Since this blog is often concerned with the relationship between science and its communication, something which has come up once or twice here is the way drug and alcohol research is reported in the media.

Very often, it is reported that marijuana use causes schizophrenia and psychosis. This is despite the fact that there is quite strong evidence that it does not do so.

We also see government campaigns presenting very misleading information about marijuana and its effects.  Consider this one stating that “Cannabis can leave you permanently out of it”.

So it was not surprising this week, when we did not see headlines such as “Marijuana makes schizophrenics smarter,” even though there was some interesting research possibly suggesting exactly that.

But what’s really interesting about this research is that it seems to suggest that not only does smoking marijuana make people with schizophrenia smarter, it seems to show that the more often you smoke, and the younger you start, the smarter you get. (Yes, there are also other interpretations of the data. Continue reading.)

The researchers did two things: a meta-analysis and an experimental study. They found 10 studies that looked at the cognitive performance of people with schizophrenia who smoked marijuana and found a remarkable homogeneity between the studies.

Together, these 11 studies (the 10 in the analysis and the 1 they performed) found that people who had schizophrenia, and had a history of smoking marijuana, had a better visual memory as well as better planning and reasoning than their non-using peers. For several domains, there were no differences between the groups but for no domain, were the non-using groups better than the using groups.

Additionally, they found that a higher frequency of smoking was associated with higher cognitive performance, as was earlier age of smoking onset. (In their own study, the association was only present for people who started smoking before they were 17.)

How could these findings possibly be explained?

Well, of course one option is that smoking marijuana causes the better performance. This is plausible since it is known that people with schizophrenia and psychosis tend to have poorer cognitive performance and so something like marijuana might perform what the authors call “a neuroprotective role”. That is, the cognitive deficits that become apparent in people with schizophrenia at around puberty might be avoided if they smoke marijuana before, or at, that age.

However, there is another explanation of the data suggested by the authors that turns on the question raised earlier: does marijuana cause psychosis? If it does cause psychosis, the researchers suggested that there might be a group of people who would not have developed schizophrenia if they had not smoked marijuana. And that group of people, as a group less prone to schizophrenia, might also be less cognitively impaired. So people who have a history of marijuana would be more likely be a group that is less prone to schizophrenia and therefore perhaps have better cognitive performance.

So should people who think they might be schizophrenic go out and smoke lots of dope? Clearly not. While it seems to me that the evidence stands against the hypothesis that pot causes schizophrenia, the matter is far from closed and a lot of researchers do think there is such a link. Moreover, this latest research is not based on an awful lot of data and really needs to be replicated in larger studies.

What’s the bet, however, that the government website “PermanentlyOutOfIt.com.au” won’t mention this new and interesting research? Much better, they think, to stick with skewed, misleading messages rather than provide believable, balanced evidence from which people can make informed decisions.

[Via me at Psychiatry Update]

ResearchBlogging.org Yücel M, Bora E, Lubman DI, Solowij N, Brewer WJ, Cotton SM, Conus P, Takagi MJ, Fornito A, Wood SJ, McGorry PD, & Pantelis C (2010). The Impact of Cannabis Use on Cognitive Functioning in Patients With Schizophrenia: A Meta-analysis of Existing Findings and New Data in a First-Episode Sample. Schizophrenia bulletin PMID: 20660494

ResearchBlogging.org Frisher, M., Crome, I., Martino, O., & Croft, P. (2009). Assessing the impact of cannabis use on trends in diagnosed schizophrenia in the United Kingdom from 1996 to 2005 Schizophrenia Research, 113 (2-3), 123-128 DOI: 10.1016/j.schres.2009.05.031