UPDATE: When news outlets get it spectacularly wrong
This morning I was alerted to two very different headlines – in as much as they say the opposite – for the very same story.
This concerns a news story about a Phase III drug trial for Alzheimer’s Disease (AD), the results of which were announced at a conference. (I have more to say about this in a moment).
So the first headline screams, “TauRx Alzheimer’s Drug LMTX Fails in Large Study Although Some Benefit Seen.” The second headline comes from opposite land and declares, “Unprecedented Alzheimer’s drug slows disease by 80 per cent.”
So which is it New Scientist and NBC News?
This trial can be fairly classified as a FAIL since the original hypothesis was not met. Doing a post-hoc analysis after the fact and finding a group of people for whom it did benefit is interesting and may stimulate further research, but is not enough to say the trial was a success.
Now clear. Doesn't work, except in post-hoc identified 15% not on current AD meds. Another failure, I'm afraid. https://t.co/RLS8wphg7g
— Robert Howard (@ProfRobHoward) July 27, 2016
Post-hoc analyses can be revealing (have published from own trials), but best to come clean about overall failure. https://t.co/cXDzLhl9zK
— Robert Howard (@ProfRobHoward) July 27, 2016
It seems that of the 891 subjects in the Phase III trial, fifteen percent of them were not taking any other drugs to treat Alzheimer’s. In this group, LMTX alone reduced the rate of decline in cognitive and functional skills by 85 percent. But, in the patients who were already taking some kind of medication, LMTX showed no benefit at all.
The researchers state they are puzzled by this and are still trying to figure out why current medications seemed to block the effect of LMTX. I’m unable to get anymore information at this point as these results were presented at a conference. For this reason, we also need to be cautious, since the study has yet to pass through the rigours of peer review.
But, this is interesting for several reasons.
One being that despite billions of dollars bring spent by Big Pharma on drugs for AD, as yet nothing has shown efficacy.
One study showed that between 2000 and 2012, 244 compounds were tested in 413 clinical trials. Only one was approved for use, a failure rate of 99.6 percent. This has led to researchers to look to prevention as a way to manage AD (COI declaration, my lab works on a compound that may show efficacy in this area).
Also, cases of AD will only increase with an ageing population and this is happening worldwide.
In the US for example, AD is currently the No. 6 cause of death. More than 5 million Americans have been diagnosed with the condition, with that number projected to nearly triple to 14 million by 2050. In December 2015, Congress pledged a $350 million increase for research on AD, an amount that could boost efforts to find an effective treatment by 2025.
So, whilst we desperately need AD drugs, it appears it’s too early to say if LMTX might be one. And these news headlines just add to the confusion.
Update: July 27, 1631. Ben Goldacre has lost his shit over this and rightly so. A Simple Google search show many news outlets have got this right but many more have got it oh so wrong. See below for the Time front page.
Thursday's Times: "Scientists create the first drug to halt Alzheimer's" (via @suttonnick) #tomorrowspaperstoday pic.twitter.com/8aMRYDrL1e
— BBC News (UK) (@BBCNews) July 27, 2016
When Goldacre calls them out, he’s met with this… baffling
you have misrepresented an RCT with a negative result as having a positive result, on the front page of the Times. https://t.co/O2mHAVblLR
— ben goldacre (@bengoldacre) July 27, 2016
@bengoldacre yes. But aim is not to summarise RCT. It is to point out there are signs of progress. You are confusing two different things.
— Chris Smyth (@Smyth_Chris) July 27, 2016
UPDATE: July 28th, 2016 0907
New Scientist have now corrected the headline and issued an amendment to their previously incorrect piece.
Sticking up for science. Whose job is it?
There’s been plenty of talk about science communication of late, specifically who should do it and should they be rewarded for their efforts.
The discussion started when two opposing view points were posed regarding metrics – one suggesting science communication (or #scicomm to the cool kids) should be considered in official metrics for academics and one strongly disagreeing.
(There were also some other discussions surrounding those who are already in a position of privilege dismissing the importance of #scicomm, and these played out on Twitter).
The perpetual argument for scientists not doing #scicomm could be distilled to “forcing scientists to do #scicomm distracts them from research and risks attracting sub-standard researchers to the field”. I disagree with this assessment, as you’ll see a bit later.
Nevertheless, this is an important discussion because in the world of academic science, careers are won and lost on “publish or perish”. Grants and promotions are rewarded based on “metrics” – which essentially constitutes how many papers you’ve published, how important the journals in which you publish are, and where in the author list your name falls.
This was once the only way to gain traction up the academic ladder, but times, they-are-a-changin’, and this includes coming out from behind the paywall of scientific publishing to talk about your work.
Discoveries Needs Dollars was a hugely successful grass roots campaign, instigated
by scientists at the WEHI. Photo credit Dr Francis Geronimo
One of the reasons why I think we should be doing #scicomm is because, not unlike other vocations such as law or medicine, science operates in another language, one that is foreign to a reasonable person. Its publications are complex, convoluted and verging on impenetrable, not only because they are frequently paywalled, but they’re written in jargon and use obscure acronyms.
Evidence suggests that even other researches don’t read their peer’s work, and even when papers are cited – meaning they are referenced in other scientists’ publications – it has been shown that they have not necessarily been completely read. Likely the abstract has been, but not the rest of the paper, and abstracts often misrepresent the findings in the body of the paper.
So here we have a situation where a) no one, sometimes not even other scientists, understand what scientists are jabbering on about b) even if you dared attempt to decipher a scientific publication you have to pay a large publishing company to read it and c) even other scientists CBF reading their colleagues work.
In my opinion, this is a problem for several reasons.
1) If scientists are receiving public funds to do research, then we should be required to tell people what we have done with their cash. Beavering away in a basement lab and pleading “mad scientist” is not acceptable and only serves to perpetuate myths about scientists.
2 – and this is particularly important – we cannot expect support from the public if we don’t engage with them.
Consider this with respect to funding. As far as the government is concerned, scientists as a group are low hanging fruit. We are not strongly union represented, we are not vocal, and the public does not universally love us. You don’t see people marching on the streets in our defence when the government cuts millions from the CSIRO budget. Consider that the current government did not even appoint a science minister when they came into power, only acquiescing after pressure from the scientific community and a cabinet reshuffle over 12 months later.
Dr Mel Thomson reacted to being called a “precious petal” as any good scientist would – with liquid nitrogen and a hammer!
Indeed, when we do complain about job losses and budget cuts we’re called “precious petals“.
By contrast, recall the outcry when the government discontinued funding to the automotive industry in Australia? Eventually they offered assistance packages in the form of retraining for retrenched workers, – not so for scientists who lose their jobs as every grant season comes around or when the CSIRO has $110 million cut from its budget.
Who could forget the story of the scientist tipped to win a Nobel prizer who lost his job from CSIRO so opted to work for free. I’d like to see some politicians do this…
But there have been times when public support for science has been high and the results have been very positive. Remember Discoveries Needs Dollars, a grassroots campaign designed to protest slated cuts the science budget leading up to the 2011 election?
This was instigated by researchers at the WEHI and mobilised thousands of scientists from across the country to take to the streets in lab coasts and placards in an effort to convince the government not to cut research funding.
What made this campaign so compelling was stream of people who stepped up to talk about how they’d been helped by science based medicine. It was personal – people could relate to it, it resonated with the public who saw no sense in cutting research funding when the return on investment is so high. And it was successful, the budget cuts were reversed.
This was a great case study for the power of #scicomm but that was 2011 and has the momentum continued? No.
As is their right and responsibility, (most) scientists returned to their benches and forgot all about it. But they shouldn’t and here’s why.
If we don’t talk about science, participate in discussions in the media, write blogs and articles, translate the complex into the simple, then other people will. People who do not understand science, or whose only intent is to deliberately distort science for their own gain. People like Pete Evans, Bell Gibson, Jessica Ainscough, Mike Adams, Dr Mercola, Dr Oz, the list goes on. And compared to the number of people doing a poor job of representing science, there are only a handful sticking up for it.
This is our fault and our responsibility. And as academic scientists, we should be rewarded for it. I don’t have the answer for how to best design metrics for measuring #scicomm output but I absolutely think they should be, and I’m glad this discussion is taking place. Because if we don’t stick up for science, then who will?
Post script: Follow up reading material on measuring #scicomm output can be found on Dr Cameron Webb’s blog “Putting a Value on Science Communication“. Follow Dr Cameron and Dr Mel Thomson on Twitter
Naming and shaming for ALS awareness
Anyone who knows me well (or reads this blog on occasion) will know I have little tolerance for quacks who claim to have cures or magic potions for incurable diseases.
What’s the harm I hear you say? Well, in the least, ripping people off when they’re most vulnerable costs them money and is just plain asshatty.
At the worst, people can die, either as a result of the “treatment” or more commonly because they cease their evidence based therapies for the quackery.
A particular bug bear of mine is people who claim to have “cures” for Motor Neurone Disease (MND, also known as ALS or Lou Gehrig’s Disease).
I work in ALS research, so I know some stuff about the disease and the (lack of effective) treatments. And I also know it’s a bastard of a disease. People lose the ability to walk, to speak, to swallow, to breathe, but all the while their brain functions normally. So effectively, you’re watching yourself die. Devastating.
So, with that in mind, I’m going to name and shame a chiropractor who FBorked† this post about the current viral ALS Awareness Campaign, #ALSicebucketchallenge.
I’m not going to be polite about this. This was an arsehole thing to write, arsehole for several reasons.
1) No, to the best of my knowledge this is not a marketing scam. The challenge has raised $11.4 million since July 29 for the ALS Association of America.
2) No, in over 90% of cases, we do not know the causes of ALS*
3) No, getting heavy metals and “chemicals” (how the latter differ to minerals, which he says are good, I’ll never know and neither does Mr Dean apparently) will not get their “nervous system and organs function(ing) again.”
4) No, chiropractic adjustments will not give them “healing potential”, whatever that means, or “get the toxicity” out of their bodies.
5) No, ALS is not completely “prevetable” (sic).
#ALSicebucketchallenge has so far been spread around by President Barack Obama, Bill Gates, Oprah, Ricky Gervais, and others and has resulted in increased awareness as well as a 50 times increase in donations to the ALS Association of America, which is a very good thing.
“According to The ALS Association spokeswoman Carrie Munk, the organization has collected $1.35 million from July 29 to Aug. 11. That’s not counting donations to chapter offices around the country, Munk said. During the same time period last year, donations totalled $22,000.
Tip ‘o the hat to the @roguechiro
——
If you would like to donate to ALS Research in Australia, please consider donating to our lab here. 100% of donations go to our research, no admin fees are taken out.
†I can no longer find this post on his FB page. I hope he had the decency to remove it. Asshat
*We have observed some correlations, for example, my team is working on the link between a toxin found in blue green algae, called BMAA, and ALS. We reported a plausible mechanism for this last year, and evidence continues to build that consuming foods contaminated with BMAA can trigger ALS. But this has not been unequivocally demonstrated.
“Did I say that? I don’t believe I did..” Meryl Dorey, bald faced lies.
This morning Meryl Dorey, ex-president of the AV-SN, sent an email to her Yahoo Group (message # 48461) asking her flying monkeys to hit up the editor of the Courier Mail newspaper. They are currently running a pro-vaccine campaign.
I’ve included the email in it’s entirety so you know that I haven’t edited it, but I want to draw your attention to one sentence in particular (see the highlighted text)
“So please resend your letters to the address above. And if you have a healthy, unvaccinated child, include pictures of them. The same if you have a child who was injured or killed by vaccines.”
When she disemminated the post via FB however, she removed any reference to “killed by vaccines”.
As a result of her Y! email, MD reports that she received a call from a journalist asking her why she was asking people to send in pictures of dead children. See below
Link to post here
MD then denied she had, claiming it was a fabrication on behalf of Dan Buzzard, since he had blogged about it earlier today.
(Text redacted out of respect to the family).
Of course her gullible acolytes are swallowing it whole, in spite of the fact that the group consists of 141 members and one person even replied, suggesting that they probably even read it! (This is purely speculation however, as evidence suggest most anti-vaxers don’t read further than the headline).
Bare faced lies, caught out red-handed.
Reminds me of this video from 2010, when Steve Cannane also caught her out not remembering what she’d said
This farcical tale is not over yet however. More to come
Update on Acupuncture in ED – Letter to the editor
Quick update: I promised if our letter to the ed was not published in last week’s Sunday paper, then I’d put it here.
This was written in response to a suggestion from the journalist who wrote the acupuncture/ED piece which can be seen here.
After some discussion with another science journalist and others, it seems very likely that it was The Age who instigated this article, probably via the common scenario of chatting with scientists about a particular subject, then asking the standard “are you working on anything else right now?”.
This chain of events is supported by a lack of press release on the university website or sent through the normal channels. Which makes sense as there was really nothing to release.
Link here
Whilst I’d previously put the blame squarely at the feet of the researchers, my science journo mate puts it at the feet of the newspaper. Still, the researchers have since been on radio to spruik their research, so they are still very much to blame for perpetuating this nonsense.
Link to tweet here
In the tweet above @simbelsin refers to “independent comment” which essentially mean asking for comment from another scientist or expert who was not directly involved in the study. This is commonly done in stories on science and medicine, but was not done here.
Anyway, FWIW, here is what we submitted the The Age.
(Note: the word limit for LTE is 200 for this section of the paper but 300 for LTE of the Drive section. Go figure).
The article “Acupuncture as effective as drugs in treating pain, trial shows” raises some very important questions about current research practices into complementary and alternative medicine. The cart was put well before the horse when the researchers discussed their results “while data from the study is still being analysed”.
Why did the researchers discuss their results with the media when they have yet to complete their own data analysis, let alone have the final results published in a peer reviewed journal? The Australian Code for the Responsible Conduct of Research (Section 4.12.1) advises against such practice.
Why weren’t placebo sham controls for acupuncture included in the study design, which are usually mandatory for this type of subjective outcome study? Pain is extremely sensitive to the placebo effect, and it is impossible to distinguish any real effects of acupuncture from placebo in the absence of effective controls.
Lastly, why has this trial, that was funded to the tune of over $400,000 in 2009, still not been published? Systematic investigation of complementary medicine should be encouraged, but this study, at least according to the available information and in the absence of a published study, appears to be bad practice of bad science.
Signed me and my mate John.
Aust gov: homeopathy not effective. Happy World Homeopathy Awareness Week!
Happy World Homeopathy Awareness Week
It’s official. The Aust Government says homeopathy is bollocks.
In a draft report, prepared in consultation with the public and homeopathy interest groups, Australia’s peak body for supporting health and medical research, the National Health and Medical Research Council (NHMRC) has concluded that homeopathy is not effective.
Overall finding
NHMRC concludes that the assessment of the evidence from research in humans does not
show that homeopathy is effective for treating the range of health conditions considered
These conditions include (but are not exclusive to)
• allergic rhinitis
• attention deficit/hyperactivity disorder (ADHD) in children
• bruising
• chronic fatigue syndrome
• diarrhoea in children – individualised homeopathy
• fibromyalgia
• hot flushes in women who have had breast cancer
• human immunodeficiency virus (HIV) infection
• influenza-like illness
• rheumatoid arthritis
• sinusitis
• sleep disturbances or circadian rhythm disturbances
• stomatitis (inflammation of the mouth) due to chemotherapy
• ulcers.
etc
(For a full list, please see the pdf here)
Duh, I hear you say.
Well, yes. And I agree.
This report considered both the findings of the UK House of Commons Evidence Check for homeopathy (summarised here) which was unequivocally negative and the Swiss report on homeopathy (summarised here) that claimed it was effective. However, numerous issues have since been raised with the Swiss report, the least of which was undeclared conflicts of interest.
According to this report,
“…it contains no new evidence and misinterprets studies previously exposed as weak; creates a new standard of evidence designed to make homeopathy appear effective; and attempts to discredit randomised controlled trials as the gold standard of evidence. Most importantly, almost all the authors have conflicts of interest, despite their claim that none exist.
In the NHMRC review, the panel declared all their conflicts of interests, which is a very good thing. And this report supports the UK House of Commons report, published in 2010.
As a scientist who understands the laws of, well, science I have to wonder why this was even done to be honest. Especially since, according to some reports, this has cost the Aust tax payers over $140,000. And also since a previously leaked draft report (almost 2 years ago) said pretty much the same thing).
Hi NHMRC! Wanna know how homeopathy works? Well, check this website.
Here’s my opinion: homeopathy is magic witch craft that was invented before we even fully understood the laws of physics and chemistry. It defies all of these laws which are now accepted science. If it were true, we’d need to change the basic tenets of science. And I’m not the only one. Even Wikipedia knows.
Extraordinary claims require extraordinary evidence, and in this case homeopathy has failed. Dismally.
The full draft report, as a pdf, can be found here.
NB: it’s a draft, meaning it’s open for public submission. Hop to it people.
—
Summary of findings: (my emphasis)
NHMRC’s interpretation of the assessment of the evidence on the effectiveness of homeopathy
In line with NHMRC’s function to “advising the community” ..and based on the
assessment of the evidence of effectiveness of homeopathy NHMRC believes:
• There is no reliable evidence that homeopathy is effective for treating health
conditions.
• People who choose homeopathy instead of proven conventional treatment may put
their health at risk if safe and evidence based treatments are rejected or delayed in
favour of homeopathic treatment.
• Homeopathy should not be used to treat health conditions that are serious, or could become serious.
• People who are considering whether to use homeopathy should first get advice from a
health professional (e.g. GP, specialist, nurse practitioner or pharmacist). Those who
use homeopathy should tell their health professionals, and should keep taking any
conventional medicines that they have been prescribed.
—
According to the report, the NHMRC has provided more than $86 million in funding for scientific research into complementary medicine and alternative therapies since 2000.
This is not necessarily a bad thing. Indeed, I myself work on a supplement. But when there’s no plausible mechanism and after over 200 clinical trials still showing mostly negative results, isn’t it time to stop flogging a dead horse?
I think so.
Happy World Homeopathy Awareness Week. Be aware: it doesn’t work.
Acupuncture in emergency departments: bad science and bad practice
On Monday, I blogged about a “study” that purported to show that acupuncture was as effective as analgesics for treatment of pain caused by migraine, sprained ankles and lower back pain.
The problem with this “research” was it has yet to be published – indeed the news report announcing it said the results were not even completely analysed. Except for the fact that this was mentioned, the process of going directly to the press could be in breach of the Australian Code for the Responsible Conduct of Research.
Section 4.12.1 says, “Discussing research findings in the public arena should not occur until the findings have been tested through peer review. In discussing the outcomes of a research project, special care should be taken to explain the status of the project – for example, whether it is still in progress or has been finalised”.
Regardless, this represents very poor practice in my opinion, and I can only presume the researchers did this owing to the lack of critical controls (specifically, sham acupuncture or some other acupuncture control) in their 3 years $400,000 study, meaning they probably won’t get this published. At least not anywhere decent anyway.
Well, now I have an update. One of the researchers went on radio to talk about this, and his comments were like a bingo card of logical fallacies. I couldn’t bring myself to listen to the broadcast for fear of #stabby, so a very kind clever person transcribed it for me. So here dear reader, I present it to you, with my comments in parentheses.
Audio is here
BEGINS
Chris (Announcer): Now chronic pain is something many people have to deal with on a daily basis and it can be just as debilitating as anything they’re suffering.
Well a new study out of Melbourne has shown some very promising developments when it comes to pain relief, and it doesn’t involve any new drug.
The Chinese therapy of acupuncture is currently being put tot the test across four of Melbourne’s top hospitals as well as RMIT University; and before you roll your eyes at the idea, let me just tell you about the results, because so far the results have been pretty amazing!
The study is currently indicating that acupuncture could be just as good as those serious drugs we take to relieve pain. [NB study is NOT published, results still being analysed, thus these claims can not be made]
For instance, Panadeine Forte, Votaren even Valium which is quite a breakthrough in the area of acupuncture; could acupuncture be the future in pain relief?
On the line with me now is Mark Cohen from the Health Innovations Research Institute at RMIT University. Mark, welcome to the program.
M Great to be with you Chris.
C The results from this study, pretty promising so far?
M Well they are very promising [not analysed, not published], and these come on the back of a lot of experience [anecdotal, NB there is nothing wrong with instigating a study based on observations, this is how much of science happens, but especially where pain is concerned, you must have very strict controls in place to ensure you can differentiate between placebo and any effect. This was not controlled in this study] that we’ve had in Melbourne of Emergency Medicine physicians who have been trained in acupuncture an who have been using it in the Emergency Department with great effect, and then we decided, well let’s put this to the test, and we’ve compared acupuncture for migraine, back pain and ankle sprain to acupuncture plus drugs, or just drugs alone.
C Interesting. So tell me about just acupuncture alone? You think that it can give you pain relief equivalent to something like a Panadeine Forte?
M Well, in this study we showed that they are basically equivalent [no, you didn’t. There’s an old saying in science, if it’s not published you may as well have not done it] , so that there was no difference between Panadeine Forte, and not just Panadeine Forte, even opiates, things like Pethidine, and you know, harder painkillers we often get …
C Uh huh
M …in Emergency Departments
C What about when you put these two together? Acupuncture, and some of these harder anti-pain drugs?
M Well again, together we found that… and our results are preliminary [read: unpublished], but we together found that more effective than either one alone. But, er, with the way we do statistics, and we’re still finalising the statistical analysis, is at the moment what we can say is they’re deemed equivalent. So acupuncture was deemed equivalent to pharmacotherapy [not], which is a whole range of um drugs that are used, according to a protocol in each Emergency Department
C That’s very interesting, but can you please just explain to people; I know, I’ve had acupuncture on a couple of occasions in my life for a chronic back area that I have that, er, gives me a great deal of trouble…
M Uh huh
C ..but when I’ve had acupuncture, I tell you what, I’ve gone without a pain in that area for sometimes years. So I, I , I, would opt for acupuncture, um, if I got that pain back again. But just give people an idea of what acupuncture is and what it does.
M Well acupuncture, I mean, has been used for thousands of years [logical fallacy, argument from antiquity and also disputed], and it hasn’t just been used in China, a lot of different traditional medicines have what is called counter irritation, where you irritate one part of the body with a needle, or heat, or pressure, to relieve pain in another part. Acupuncture, modern acupuncture, involves usually disposable needles inserted in precise spots, and often the best spot to put a needle is the most intensely painful spot, so you put, you know, a few needles that vary on the condition and where the pain is that you’re trying to treat, into the body, often then you twiddle the needles, or you turn, and lift them up and down, and you get this sensation that the sensation that the acupuncturists call “Dead Chi”…
C That’s right!
M …or “A Chi” and it’s a feeling of numbness, and fullness, and distension, and a bit of soreness; it’s not quite like electricity, but…
C But it’s not always pleasant, I’ve gotta say Mark
M …it’s not entirely pleasant, but it’s not directly painful…
C No
M …more sore than pain, and it feels a bit forward, sometimes that feeling travels, you know, along the body…
C Yeah
M …but that feeling generally means you’re going to get a good result, and there’s, there have been a lot of animal studies that have shown that when you insert needles, acupuncture needles, you get release of endorphins and a lot of other neurochemicals which are your body’s own natural painkillers
C So you allow the body to take the pain away from the body?
M That’s right, so that you’re basically alerting to the body, alerting to the body, saying hey, something’s going on here, deal with it.
C How, how popular is acupuncture? [note: just because something is popular doesn’t mean it works. Plenty of people believe in an invisible sky fairy, doesn’t make it real. Your logical fallacy is argumentum ad populum]
M Well acupuncture’s very, I mean, China, and around the world it’s used very very commonly; even in Australia I mean most suburban areas would have a Chinese medicine practitioner practising acupuncture.
C Does your average GP think that acupuncture is a bit “witch doctorish”
M Well, no, I did a study about ten years ago asking GPs their opinion about a whole range of complementary therapies, and acupuncture was probably the most accepted, and about 20% of Australian GPs have some training themselves in acupuncture [again, argumentum ad populum], and acupuncture has had a Medicare rebate item since the early eighties, so, I mean, you get, you can get acupuncture on Medicare [doesn’t mean it works, you can also claim homeopathy on Medicare] when it’s done by a doctor, or, you know, you can get it done by a traditional Chinese therapy…
C So it’s interesting, if you’ve got chronic pain, and you’re taking drugs consistently, over and over again, it can’t do much for your kidneys or your liver, could it?
M Well, that’s right, and one of the issues is not just chronic pain, but what we tested acute pain in the Emergency Department, and the drugs that are used to treat pain, all have fairly serious side effects, and even Panadeine Forte, I mean they make you very constipated and nauseous, and often, when you’re giving an opiate drug, for example, you can’t just give an opiate drug, you have to give an anti-nausea drug at the same time…
C Yes, yes
M …and other drugs to stop you getting constipated, because then you get abdominal pain and bloating, so, the drugs themselves, and also in an Emergency Department you can’t just give someone a dose of opiates and then send them home, you’ve got to watch them for four hours because they’re going to be drowsy, and so there are serious side effects to the drugs, so having an alternative to drug therapy, that’s not a drug, is actually a great boon for many many patients.
C Yeah. Interesting. And so you must be excited by this, but there’s more to do?
M Well, there’s certainly more to do [like, er publish the results?], and, and um this is, we did this in four hospitals, there are two private hospitals, Cabrini and the Epworth, and two public hospitals, the Northern and the Alfred, and um, you know the results are the same across the hospitals [no way to tell with no acupuncture controls], now the question is, if, if acupuncture’s going to be delivered commonly in the Emergency Department, who’s going to deliver it, can we train emergency physicians to do acupuncture…[er, easy tiger, maybe you should publish the results first]
C Yeah, right
M …you know, do we bring in other acupuncturists, what level of training do they need, but, it’s it’s quite telling that in Australia I know at least three emergency physicians who have done their own training in acupuncture [is it? your logical fallacy is argument from authority], and Michael Ben-Meir, who’s now one of the, he’s now head of the Cabrini Emergency Department, now he’s a trained acupuncturist, and he uses acupuncture pretty much as a routine therapy for patients who come to Cabrini Hospital in Melbourne
C A great devotee, all right, good luck, and thank you very much, and well done with what you’ve discovered.
M Yeah, well thanks, Chris, um yeah it’s encouraging, and we hope, we hope you know hopefully this will help a lot of people who are in pain and um give them another option… [when it’s published, maybe]
C Yeah, absolutely. Thank you, Mark!
M It’s a pleasure.
C Mark Cohen from the health Innovations Institute at RMIT University. So maybe it’s not about taking drugs to get rid of that pain that you may have, maybe you’ve got to think outside the square and think about acupuncture, it’s not that witch-doctorish as you may think. We’ll take some calls on that.
[Calls are between DJ and public]
ENDS
I find this practice extremely unethical and irresponsible and the researchers involved should be ashamed. The media also deserves a slap upside the head, as described eloquently here by my colleague Mick Vagg. I’ve also submitted a letter to the editor of the paper who reported the study, penned with my colleague John Cunningham, and I will publish it here if it doesn’t make the Sunday paper.
I don’t expect we’ll ever see this research published, but it doesn’t matter anymore, the horse has has bolted. People think acupuncture works as well as drugs, based on some very questionable evidence.
In this broadcast, Prof Mark Cohen did not make it sufficiently clear that these results have not been published and that the correct controls were not used = bad practice and bad science.
Thanks to Steven Novella who also blogged this here
Announcing scientific results in the press before peer review = bad science
I wasn’t going to blog about this because meh, but I really think it’s important to point out the obvious flaws in this “study”.
A news item announced the results of an acupuncture study today, proclaiming in the headline “Acupuncture as effective as drugs in treating pain, trial shows.”
An acupuncture trial in four Melbourne emergency departments has found it is just as good as drugs in relieving lower-back pain and that from sprained ankles and migraines.
Emergency physicians at The Alfred, Northern, Cabrini and Epworth hospitals partnered with RMIT’s school of health sciences to see if acupuncture could relieve acute pain in hundreds of patients presenting to hospital with either lower-back pain, sprained ankles or migraines.
Ok, fine. But then this
“While data from the study is still being analysed and finalised for publication in a medical journal, one of the researchers, Dr Michael Ben-Meir, said it showed acupuncture offered the same level of pain relief as analgesic drugs when patients rated their pain one hour after treatment.
Wait a second….
“While data from the study is still being analysed and finalised for publication…”?
So it’s not been published yet? And the data is still being analysed? WTAF?!
Then why are these researchers apparently alerting the media to their findings when they haven’t even passed peer review? I have a few ideas on why this might be, which I’ll get to in a minute.
Do not pass step one, do not collect $200.
But first, it’s obviously difficult, nee impossible, to critique the findings effectively in lieu of a paper, but there are some obvious faults that stand out.
So we have three treatment groups:
1) standard pharmaceutical care alone (eg Endone, Panadeine Forte, Voltaren, Valium)
2) acupuncture alone
3) combination of acupuncture with standard pharmaceutical care
Findings: acupuncture alone delivered equivalent pain relief when compared to a combination of acupuncture with standard pharmaceutical care or standard care alone, reported 1 hr after treatment.
Conclusion: acupuncture is as effective as drugs for lower-back pain, sprained ankles and migraines
Well, the trouble with acupuncture is it’s hard to blind people to the treatment (and this study was not blinded) so researchers sometimes use what’s known as sham acupuncture. This means needles are still used but they’re either inserted in the wrong place (i.e., not along the magical meridian lines) or they are not inserted far enough to have an effect. This way the researcher can control whether the acupuncture is having a real effect or it’s the placebo effect. (sham acupuncture has it’s own issues too, however).
Since it’s not indicated if this control is used here, there’s no way to determine whether the effect from the acupuncture was real.
Patients reported they felt as much pain relief from acupuncture after one hour, as they did on pain killers.
Well sure, but we can’t draw any conclusions from this without an acupuncture control group. Pain is extremely subjective and is highly susceptible to placebo. Plus, people often feel better one hour after treatment anyway – it’s because they’ve been in hospital and been attended to. They’ve been looked after. Of course they feel better. Without effective blinding of treatments there is simply no way to tell.
In addition the placebo effect is a curious beast – the more invasive a procedure, the stonger the placebo can be, so you can imagine having needles stuck in you is likely to make you think your pain has reduced as much or even more than if you took a tiny white pill.
This claim also seems questionable, “..could open the doors to Australian hospitals offering the low-cost Chinese therapy, which is used by more than 1 billion people worldwide for pain relief.”
It’s not as simple as this surely. Traditional Chinese medicine practitioners now need to be registered with APHRA, and what’s so expensive about a packet of Panadeine Forte? (And on that note, why were apparently a random assortment or analgesics used? And if that is the case, does each type of analgesic constitute a separate treatment group?).
Look, all of this is speculation because in the absence of 1) the paper or 2) the correct controls, this study is meaningless.
Which leads me back to why it’s been announced in the press and not in the pages of peer review. I suspect it might not pass peer review without the sham controls. Also odd is that the research data Australia website says the trial began in 2008, and was due for completion in 2011. So why have the results not been published yet? Is it because they likely won’t be?
This would be a very big worry for the researchers, especially given that the NHMRC gave these guys $416,301. Are they announcing it in the press in an attempt to distract the NHMRC?
Even more of a worry though is the researchers apparent disregard for the Australian Code for the Responsible Conduct of Research. Section 4.12.1 says, “Discussing research findings in the public arena should not occur until the findings have been tested through peer review.”
By all means, study the efficacy of TCM, but at least do it with correctly designed experiments and try not to waste $400K of tax payers’ money (as if research doesn’t have enough money to go around). Until such time as that is done, approach me in A&E with an acupuncture needle, you’ll get a slap upside the head.
—
EDIT: A friend just suggested this on Facebook and I want to add it because it’s an important point that I should have included.
Wendy Wilkinson: “Id love to see the methods used for this research. If it ever gets published properly, that is. How did they recruit the participants? How were they randomised into the three groups? In busy EDs where this sort of thing is not an emergency. And most importantly – how long did they wait for the treatment? Because if they had been sitting in the waiting room for longer than that 1 hr how have they accounted for regression to the mean as far as pain goes.
Yep, regression to the mean may have also played a role in this study. VIP point.
—
UPDATE Monday March 31: Another issue with this study not being blinded is it may have selected for patients who are sympathetic to the efficacy of acupuncture. So if you rock up to A&E and someone says, “do you want to be in our acupuncture trial?”, if you’re me you’ll say “%^&^%&^ get me some drugs with a demonstrated mechanism of action please!!”. But those who think there’s something to acupuncture will more likely say yes. This then influences their pain perception – if you believe it, then it works (ie placebo).
Also, we must consider the role of “wanting to please the nice Dr”. Patients may report a reduction in pain (later found to be equivalent to the analgesics, well, ok, they haven’t finished analysing the data yet so who actually knows) because they have been fussed over and maybe taken to a private room to lie down and relax etc.
All of these things can introduce confounding variables into studies, which is why controls are critical. The appropriate controls were not used here. This study is worthless.
—
Monday March 31st. 11:51: Looks like there might be a few updates on this story.
Firstly, I want to clarify something re: 4.12 Responsibly communicating research findings in the public arena. I did not include this section of 4.12.1 (an oversight, not deliberate), “In discussing the outcomes of a research project, special care should be taken to explain the status of the project – for example, whether it is still in progress or has been finalised”.
So the researchers did make this clear – that being the results are still being analysed. Thus it might be fairer to say them releasing this is bad practice rather than a breach of the Australian Code for the Responsible Conduct of Research.
Further, I asked the Society and Science editor at The Age (where this was published) whether they had received this as a press release. He’s not answered this question yet.
How to avoid false balance and why you should
This week saw new vaccination coverage statistics released by the National Health Performance Authority and they were generally encouraging showing an average slight increase overall.
Somewhat unsurprisingly, the lowest rates were in Byron Bay where only 66% of 5 year olds are fully covered. One a positive note, the number of children not fully immunised lowered by nearly 2,000 compared to the previous year (75,002 in 2012–13 compared to 76,769 in 2011–12). Of these, 15,000 are registered as vaccine refusers
The media jumped on the story and as a result, those of us who occasionally comment on issues surrounding vaccination where asked to contribute. Disappointingly, we spent a fair bit of time fighting off media who were seeking to do false balance. If you’re not familiar with the concept of false balance, I’ve written about it here and here. Also please read this excellent blog also covering false balance from Thursday.
In the wash-up though, most of the media was sensible, science based and factual (some were just amazing – see this Studio10 clip here)
But the usual suspects featured in stories with false balance. For example, one well known anti-vaccine activist and friend to the AV-SN was featured as “just a Mum” (sure, no conflict of interest there).
And even worse and more worrying, the National Centre for Immunisation Research and Surveillance (NCIRS) were once again in the mix.
*NB, Kenny Lydall begins this story bragging that he’s been to rural Africa and has not had even one vaccination. This raises the question, was he allowed to enter Australia with no yellow fever vax papers?
These guys should know better, especially since they seem to be one of the few professionals left that enable the AV-SN via “debates” (see below for examples). (And yes, Steve Hambleton, I didn’t miss you there either).
I dunno why they keep having to be told, I mean George Carlin says it well.
“Never argue with an idiot. They will only bring you down to their level and beat you with experience.”
or here you just substitute creationists for anti-vaxers
“Debating creationists on the topic of evolution is rather like trying to play chess with a pigeon; it knocks the pieces over, craps on the board, and flies back to its flock to claim victory.” — Scott D. Weitzenhoffer
Or how about this one
“Never mud wrestle with a pig, because you’ll both get dirty and the pig likes it.” — Anonymous
These debates achieve nothing for the scientist. On the other hand, they lend the quack undue legitimacy and look good on their CV (but not yours). You might say meh, who cares, but you only need to look to the families that the AV-SN have vilified over the years to see the damage this kind of stunt inflicts. It twists the knife. It’s hurts.
NCIRS, please stop it.
At SAVN, our approach has always been to ask before we say yes. It’s simple really, a reporter calls you, asks you to do a segment, you say “who else will be on, what’s your angle”?. If the reporter says they plan to do false balance, we explain why it is bad, and if they still want to do it, we politely decline.
THE END
This approach works BTW. It has resulted in one prime time news show (The Project) publicly stating (on air) that they no longer speak to anti-vaxers. Another (ABC – gov broadcaster) sent a memo to all staff warning them not to speak to the AVN unless they make it clear they are nothing but an anti-vaccine lobby and their spokesperson has no qualifications.
From my own experience, a few months ago, I terminated an interview half way through because the interviewer said they had spoken to Meryl Dorey. Even though they had already interviewed her, they didn’t go ahead with the story in the end. Why? Well maybe partly because they had no “controversy” anymore, and also partly because I took the time to explain to them why false balance is bad (again, see my previous article here for a thorough explanation) but I also filled them in on some details about Ms Dorey herself.
Below is a conversation I had with Dr David Hawkes and a journo from ABC 730. (He didn’t respond after I took the time to explain why we say no to false balance).
Other ways we are attempting to quash false balance regarding vaccination is by getting in touch with journos/ news outlets when they do a good job, just to say thanks. For example some mates of mine did a Follow Friday on Twitter this week for media outlets who do not promote false balance. Apart from just being nice, this also creates a repoir with journos, and establishes contact so that next time they have a story, they might remember to call you.
#FF for the media who don’t do #falsebalance on vaccination. Almost forgot @MamamiaNews @MiaFreedman — Alison Gaylard (@purplealibongo) March 28, 2014
#FF for the media who don't do #falsebalance on vaccination. 1/2 @Joe_Hildebrand @spicertracey @Studio10au #carolduncan @BreakfastNews
— Dr Dave Hawkes (@mrhawkes) March 28, 2014
#FF for the media who don’t do #falsebalance on vaccination. 2/2 @2GB873@StevePriceMedia@LaTrioli@mjrowland68@carolduncan@ausjourno
— Dr Dave Hawkes (@mrhawkes) March 28, 2014
@LaTrioli@theprojecttv Parents, doctors & scientists across Australia thank you. Should get @purplealibongo on, she does great work
— Dr Dave Hawkes (@mrhawkes) March 28, 2014
One of the issues with Thursday was we were essentially playing whack-a-mole, and we are aware that we need to develop a pro-active strategy rather than a reactive one.
More work to do, but at least for the time being we have stopped the AVN (in name). And thanks to the media who have helped us along the way.
A(nother) bad week for the AVN. Or should I say AVSN
It’s been a bad week or the AVN. Or should I say AVSN.
You can hear the audio version of this report on The Skeptic Zone episode 282, here.
The AVN have a new name – The Australian Vaccine Skeptics-Network. This was officially introduced on March 7th, 2014 when it was approved by the Dept of Fair Trading.
Initially, I was pretty annoyed at this, since I classify myself as a sceptic of the scientific-scepticism kind (not the loony 9/11 or climate change kind).
However, it was clear to many of us following the findings from the name change challenge – where the magistrate defined the AVN as “sceptical of vaccines” – that the AVN would go after the word. This is in spite of Merpyl* having used it in a derogatory fashion for years, often shortening it to “septics” when referring to myself and my colleagues. At the same time however she had also decided that she was a “real sceptic” (she even set-up a blog called The Real Australian Skeptics” nyuk nyuk Marpyl*). Not “pseudo sceptics” as we were apparently deemed to be.
So, after more than 12 months (it was Nov 2012 when the minister first ordered them to change their name) the AVN is now the AVSN.
In reality, the public perception of the term “sceptic” is of climate change and moon landing sceptics – the term has definitely evolved to mean something other than scientific scepticism – so in actual fact it’s probably not a bad thing. However, in my (and others) opinions, it doesn’t go far enough. In an ideal world, I’d have liked to see The Anti-Vaccine Network, as this absolutely describes what they do, but we can’t have nice things all the time.
In a twist, Merl* is claiming she can maintain the acronym AVN and the URL avn.org because Vaccine-Skeptics is one word. Er, I dunno about this.
Also of interest is that another person had registered the name Australian Vaccination Sceptics Network (note the “C” and lack of hyphen) several months before the AVN did. According to business law here in OZ, they have a right to challenge such a similar name.
I imagine the AVN are pretty pleased with themselves for pulling this little stunt, but unfortunately for them, the media coverage of the name change was peppered with “AVSN” and anti-vaccine network. This prompted Mapel* to huff that she had “told these people” they are still known as the AVN. I wonder what the Minister for Fair Trading will have to say about that…
The AVSN smugness was short lived however, as the following day, the first draft of the shiny and new 2013 HCCC investigation into them was released.
Readers may recall that the HCCC first investigated the AVN back in 2009 and this resulted in them issuing a public warning about the information on their website. The AVN successfully challenged the investigation (on a technicality) and thus the public warning was quashed and their charity licence re-instated. There must have been some very red faces in various government acronym that day.
However, the government was determined not to give up, so the technicality that was the HCCC’s undoing was changed by legislation to include an amendment that enables the Commissioner to instigate investigations, even in the absence of public complaints (in this case, there were 2 complaints from the public and one from the Commissioner).
Thus, almost as soon as the law was changed, the HCCC began a second investigation into the AVSN. The draft report released on Tuesday was published by the AVSN and they are now given an opportunity to respond to the Commission’s requests. Essentially, the HCCC trawled the AVSN’s website and selected 8 points that they identified as misleading, they then asked experts to provide evidence for why they were misleading. Their findings make for some good reading (and you can access the full pdf here) and are laden with “remove and keep permanently removed”.
For example “Remove and keep permanently removed the misleading and incorrect statement that “vaccines have never been tested, either individually or in combination.”
“Remove and keep permanently removed the misleading information about the link between mercury and autism.”
One wonders just how much stuff will remain on their site if they agree to co-operate with the HCCC on these points.
The AVSN have also published several of their responses to the HCCC and as you can guess they’re chock-a-block filled with hubris, (as we’ve come to expect). If you can stand it, go have a look, they are now up to Part 6 of calling the HCCC incompetent and hopeless. This is not gonna end well.
It seems to me very likely the AVN will not comply with the requests of the HCCC and it seems the HCCC also think this as they’ve already written the public warning (pdf) that will be released when the AVSN refuse to comply.
I’m so very pleased to see this back after 3 years! (Just quietly, I think it’s better than the last one).
But the icing on the cake came earlier today when it was noticed that the AVSN had surrendered their charity license. SAVN have been trying to get the mainstream media interested in the AVSN’s finances for some time since we have been pouring over their financial statements for years. For a long time, we have noticed discrepancies regarding expenses including such things as,
• nearly $5000 for travel in 2012 for Meryl to attend high school reunion/”conference”,
• $39,000 on computers in 2003/2004 in addition to web costs,
• $5000/edition for the LW magazine in editors fees
• $170,000 of “other” expenses in 2010
Even more of a worry for a charity however, is that we can find no evidence of any monies going to charitable purposes. (In fact, in the entire time I’ve been observing the AVN I’ve never seen them do anything charitable).
To cite a recent example, most recently, the AVSN were promoting a crowd sourcing campaign for a child who suffered a very rare vaccine reaction which rendered her blind. Her parents were raising money to build her a sensory garden, yet despite a monetary challenge from an SAVNer to match his $1000 donation, they never did donate to that cause (at least not as far as we know – and the campaign lists all the donors). This is despite her claiming publicly that “the AVN helps vaccine injured children and families everyday”. Despicable.
You can read the full story which appeared in the Fairfax press here. I like it best for this bit
“A chartered accountant who examined the documents for Fairfax Media, but declined to be named for fear he would be harassed, said the documents were ”the worst set of financial statements I have ever seen”.
Heh, ”the worst set of financial statements I have ever seen”.
See below for the evidence that the AV-SN have surrendered their license.
Finally, the was much fun to be had on SAVN the last few days and one of our eagle eyed “likers” noticed Marly* had liked a post on a FB page with her secondary account, but signed it off as a particularly nasty vicious (previously anonymous) AVN admin, B52. (Full post here).
This admin has been around for some time, and could be classified as the attack dog of the page, whereas Maryl* portrays the image of the “kindly Aunt”. See the image below.
This was not the funniest part though, it was her backpedalling and lies to try to cover up her error that were the most hilarious. First she came onto SAVN claiming the account was not her and we had made it to impersonate her. She then went back to her aciolytes and declared we were “ebil septics” and could people please report the “fake account”.
Only problem was, the account that posted as B52 has the url –meryl.dorey. The account she said we had made to impersonate her and had posted as B52 had the url –meryll.dorey. See the difference? So essentially, she got caught, decided to create a new account to pretend “ebil septics” but it backfired when even her followers spotted the different urls and questioned her.
Oh a tangled web we weave.
So it has been a busy week. But it’s also been packed full of lulz. And when that concerns the AVSN, it can’t be a bad thing.
—
*Note: I have used various incarnations of the name of the ex-president of the AVN in this blog post – not out of disrespect – but because over the new year, she has a slew of people banned (some for weeks) on FB for “using her name without her permission”. Since I did not seek her permission, I have not used her name here.
