... and other stories.
Papers, listed by lead author: C
C
Camerlink, I., Ellinger, L., Bakker, E.J., Lantinga, E.A., (2010) Homeopathy as replacement to antibiotics in the case of Escherichia coli diarrhoea in neonatal piglets Homeopathy Vol. 99 no. 1, pp. 57-62
This must be one of the few papers ever published to study a condition in animals which doesn’t take the trouble to discover whether the animals concerned actually have the condition in the first place! That’s right, not one of the pigs under study had a confirmed enteropathogenic E. coli infection. The limited number of cultures which were carried out revealed no sign whatsoever of the enteropathic E. coli under study. Instead the authors preferred to make this judgement by virtue of the colour of the diarrhoea, a laughable and ridiculous way of making a diagnosis, particularly one they were going to base such farfetched claims (homeopathy works) upon.
This fact plus the journal of publication is a trade journal for homeopaths positively bursting at the seams with vested interests and given that the study is only single blinded so we can’t be sure that the humans involved were aware of what groups received what medication means this study has got “sloppy protocol” written over it - just more homeopathic puff I’m afraid.
That of course hasn’t stopped homeopaths the world over making the bogus claim that this is proof positive of the effectiveness of homeopathy - the swines!
Caspi, O., Millen, C., (2000) Sechrest. Integrity and research: introducing the concept of dual blindness; how blind are double-blind clinical trials in alternative medicine? J Altern Complement Med. Vol. 6 no. 6 pp. 493-8
The authors claim that double blinded trials aren't appropriate for some types of CAM so they propose "... a new term in research methodology, dual-
Links: [abstract - pub med ]:[abstract - liebertonline ]
Caulfield, T., and DeBow, S., 2005 A systematic review of how homeopathy is represented in conventional and CAM peer reviewed journals BMC Complementary and Alternative Medicine Vol. 5 no. 12
“While a small study with clear limitations, there was a stark difference between the numbers of studies that were negative in the conventional journals (69%) as compared to the CAM journals (30%)”
Oh, and we’ll make out that it’s proper journals that are prejudiced, not homeopathic journals which are wildly optimistic and uncritical when it comes to selecting papers for publication!
Links: [full text BMC]
Responses: [Gimpy’s blog: Publication bias in CAM? Nope, it’s orthodox research that is flawed. Err maybe not]
Responses: [Gimpy’s blog: Publication bias in CAM? Nope, it’s orthodox research that is flawed. Err maybe not]
Cavalcanti, A.M., Rocha, L.M., Carillo, R., Lima, L.U., Lugon, J.R., (2003) Effects of homeopathic treatment on pruritus of haemodialysis patients: a randomized placebo-controlled double-blind trial. Homeopathy Vol. 92 pp. 177-181
Links: [abstract pubmed]:[abstract, science direct]
Chapman, E.H., Weintraub, R.J., Milburn, M.A., et al. (1999) Homeopathic treatment of mild traumatic brain injury: a randomized, double-blind, placebo-controlled clinical trial Journal of Head Trauma Rehabilitation Vol. 14 pp. 521-542
Lets play a little game -
Criticalist, a poster at the James Randi Educational forum (link below), reports a problem with the statistics in this one and also comments on the limitations of not using a validated questionnaire:
‘Their outcome measure is the results of a questionnaire, which they administered to both groups, before and after the treatment. The authors admit that this tool had not been validated in previous trials, which in itself is a major problem - they simply have no idea if the primary outcome measure of the trial measures what they think it does.
‘The questionnaire comprises 3 sections with a total of 65 questions about various activities, to which the the subjects reply by scoring a 1 -5; (1 = never, 2 = rarely, 3 = sometimes etc...). These are rank ordered ordinal variables which means that while they progress in order, they don't provide quantitative information. So for example, someone who has a weight of 60kg is always exactly twice as heavy as someone whose weight is 30kg. However someone who replies "most of the time' (a score of 4) to the question "How often do you feel frustrated" is not always frustrated exactly twice as much as someone who had responded "rarely" (a score of 2). Ordinal variables like these have to be analysed statistically in a different manner to ratio scales like weight, but the authors do not do this.
‘Instead they add up all the numbers they get in each part of their questionnaire before and after the treatment and present it an arithmetic mean. They then perform a t test for each of seven sections of the questionnaire. Doing this they found no real differences in the majority of the data ("Our data revealed the limitations of our standardized tests to detect changes from treatment") Out of the seven analyses they performed one is significant (p=0.009) the rest are not. They then perform a whole raft of multivariate analyses, with again one or two results they claim are significant.
‘Overall, I am reminded of a quote from one of my statistics lecturers; "If you torture the data long enough, eventually it will confess"’
Links: [abstract, pubmed]:[abstract, journal head trau rehab]:[full text homeopathy.org]
Responses: [JREF forum]
Responses: [JREF forum]
Clover, A., (2000) Patient benefit survey: Tunbridge Wells Homoeopathic Hospital British Homeopathic Journal Vol. 89, pp 68-72
Yet another patient questionnaire -
But even with all this pressure for an answer in the affirmative in this self selected population of patients who presumably had favourable inclinations towards homeopathy (otherwise they wouldn’t have been there) an incredible 26% actually said they had either got worse or felt no benefit whatsoever following homeopathic treatment, with a further 19% saying they only felt “slightly better” following treatment. How rubbish is that?! When you take into account the number of questionnaires which were handed out but not completed the situation looks even worse with only 40% of surveys returning a positive report. What with all this and despite the author’s admission that this study is “not a definitieve research paper” and runs contrary to homeopathic principles it’s no wonder the only journal they could find which would publish this paper was the British Homeopathic Journal, one of the major trade magazines for homeopathic profiteers. Even still, you will find this report touted in countless web sites as evidence favourable for homeopathy. Lies, damned lies and statistics.
Links: [abstract science direct]
Colquhoun, D., (2007) Treating Critically Ill Patients With Sugar Pills Chest Vol. 131 no. 2 pp. 635-This is a response to Frass (2005) and is followed by a response from the authors of the original paper.
Links: [full text, Chest]
‘Clark’s claim in 1927 that: “some travesty of physical science appears to be the most popular form of incantation” is even truer today. Homoeopaths regularly talk nonsense about quantum theory, and “nutritional therapists” claim to cure AIDS with vitamin pills. Some of their writing is plain delusional, but much is a parody of scientific writing, in a style that Ben Goldacre calls “sciencey.” It reads quite plausibly until you check the references.’
This is Professor Colquhoun at his finest giving an elegant, straight to the point summary of the state of quackery in the UK. He is especially annoyed at Prince Charles -
Links: [abstract, pubmed]:[fulltext html BMJ]:[original, unedited text DCScience]
Responses: [rapid responses BMJ]
Responses: [rapid responses BMJ]
Cracknell, N.R., Mills, D.S., (2008) A double-blind placebo-controlled study into the efficacy of a homeopathic remedy for fear of firework noises in the dog (Canis familiaris). The Veterinary Journal Vol. 177 pp. 80-88
“Seventy-five dogs that showed a fear response to fireworks participated in a double-blinded, placebo-controlled clinical trial to assess the efficacy of a homeopathic remedy for the alleviation of their behavioural signs... There were significant improvements in the owners’ rating of 14/15 behavioural signs of fear in the placebo treatment group and all 15 behavioural signs in the homeopathic treatment group. Both treatment groups also showed significant improvement in the owners’ rating of the global severity of their dog’s responses. However, there was no significant difference in the response seen between the two treatment groups.
A post hoc analysis was carried out... to investigate the likelihood that an effect could be detected with the sample size used... we believe the clear lack of a value approaching significance, means we can be confident that there was a real lack of it in this study and the lack of significance is not a type II statistical error.”
A study which shows, once again, that homeopathic remedies are indistinguishable from sugar tablets - how many times does this need repeating!
A study which shows, once again, that homeopathic remedies are indistinguishable from sugar tablets -
The investigation of canine behaviour using this type of technique is fraught with difficulties and pit-falls. Such matters are dealt with most exquisitely by Overall and Dunham (2008) (see link below).
Cucherat, M., Haugh, M. C., Gooch, M. and Boissel, J.P., (2000) Evidence of clinical efficacy of homeopathy. A meta-analysis of clinical trials. HMRAG (Homeopathic Medicines Research Advisory Group). European Journal of Clinical Pharmacology Vol. 56 no. 1 pp. 27-33
“OBJECTIVE: To establish, using a systematic review and meta-analysis, whether there is any evidence from randomised controlled clinical trials of the efficacy of homeopathic treatment in patients with any disease... The combined P value for the 17 comparisons was highly significant P = 0.000036. However, sensitivity analysis showed that the P value tended towards a non-significant value (P = 0.08) as trials were excluded in a stepwise manner based on their level of quality...
CONCLUSIONS: There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies...”
CONCLUSIONS: There is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials. Studies of high methodological quality were more likely to be negative than the lower quality studies...”
This paper derives from a report commissioned by the EU by Boissel et al (1996) and is one of the "Usual Suspects". Both versions of the report are dealt with together here as one of the pieces of evidence homeopaths believe best supports their views.
What the homeopaths say: The European Network of Homeopathy Researchers (ENHR) mention a report which seems similar to Boissel et al (1996) although it is listed as “short version” about which they state - “ HMRG report with overview of clinical research in homeopathy, identified 184 controlled clinical trials. They selected the highest quality randomized control trials, which included a total of 2617 patients for a meta-analysis. This meta-analysis resulted in a p-value of 0.000036 (which means that results are highly significant) indicating that homeopathy is more effective than placebo. The researchers concluded that the "hypothesis that homeopathy has no effect can be rejected with certainty" .
This rather optimistic conclusion does not concur with the conclusion of the resulting paper (Cucherat et al, 2000) cited above despite the fact that the authors are mostly the same. It is significant that the origin of this quote by ENHR is not available on line - we have no primary source for it as we do for the much more cautious European Journal of Clinical Pharmacology version (see link to abstract below). It is also of interest that the "short version" of the report (or at least the quote taken from it) doesn't contain the highly significant qualifier which appeared in the later paper following the ' P=0.000036 likelyhood that homeopathy works' claim, namely "However, sensitivity analysis showed that the P value tended towards a non-significant value (P = 0.08) as trials were excluded in a stepwise manner based on their level of quality ". In other words, the better the quality of the trial the worse the result for homeopathy.
Contrast the ENHR comment with this one, from Stephen Barret of homeowatch:
“In 1996, a lengthy report was published by the Homoeopathic Medicine Research Group (HMRG), an expert panel convened by the Commission of the European Communities... After examining 184 reports, the panelists concluded: (a) only 17 were designed and reported well enough to be worth considering; (b) in some of these trials, homeopathic approaches may have exerted a greater effect than a placebo or no treatment; and (c) the number of participants in these 17 trials was too small to draw any conclusions about the effectiveness of homeopathic treatment for any specific conditions.”
Another comment from Wayne Spencer in the Skeptical Intelligencer, August 1997, Volume 2 Number 2 states:
“... 184 placebo-controlled trials were examined. Most studies were of poor quality and only 20 trials were found that (a) were randomised; (b) had a clearly described primary outcome; and (c) had a curative intention. After further eliminations because data was unavailable or insufficient for the purpose at hand, results from 15 trials (17 comparisons) were pooled and employed .
“The diversity of diseases and outcomes studied in the trials precluded a classical meta-analysis. Instead, Boissel et al (1996) applied a statistical test to the 17 eligible comparisons with a view to testing the hypothesis that "the treatment effect is not present in any of [the] trials pooled"... The result indicated that the probability that the results could be attributed to mere chance was less than 1 in a 1000 (p<0.001). The authors explained:
"’This means that, in at least one trial, the null hypothesis of the absence of effect can be rejected, namely that, in at least one trial, the experimental patients (i.e. those who were treated with homeopathic medicine) had some beneficial effects compared with the trial patients (i.e. those who received nothing, or a matching placebo), assuming that none of the pooled trials were biased in any way’ ...”
Spencer points out that the authors stated they were unable to rule out such bias and that evidence for homeopathy became weaker as the quality of trials increased, a sure indication that bias was, in fact present. His final comment on this paper is "as factors identified within the report itself qualify its support for the claims of homoeopathy virtually out of existence, little additional comment would seem to be necessary."
Of the original report, Edzard Ernst states in the House of Commons Science and Technology Committee on homeopathy (see link) "Boissel et al merely combined p-values of the included studies. This article is now... outdated. Furthermore it is not unambiguously positive. " and of the published paper, "Cucherat et al is the publication of the Boissel document which was a EU sponsored report. [The authors themselves noted that “there is some evidence that homeopathic treatments are more effective than placebo; however, the strength of this evidence is low because of the low methodological quality of the trials.”]"
So, the final verdict (and remember, this is some of the absolute best evidence that homeopaths can produce): only 16 out of 184 trials were of adequate quality; analysis using an unorthodox statistical method suggested that one or a few seemed possibly to show homeopathy might have some benficial effect compared with plain sugar tablets but even this result could be due to bias which, for the purposes of the analysis, the authors had to assume wasn’t present. Furthermore, better quality trials gave less favourable results for homeopathy.
Not too convincing really, for a system of medicine which claims to be as powerful as homeopathy, to barely scrape past a sugar tablet in the "beneficial effects" stakes, and then only if everyone involved could be trusted to have had no vested interest in the results.
Next please...
| History of RVM |
| The Best They Can Do |
| The Usual Suspects |
| How to look at evidence |
| CAM journals and evidence |
| Scepticism about homeopathy |
| Arguments |
| Acupuncture references |