A couple of weeks ago I talked about some rules of thumb for assessing fact vs fiction, and want to pass on something similar in the realm of medicine, based largely on points made by Steve Novella during the Science Based Medicine forum at TamOz.
For assessing claims regarding some treatment or other, consider:
- How extraordinary is the claim (is it plausibly supported by what we know currently, or does it require a huge leap in knowledge or understanding)?
- What is the mechanism of the treatment? An absence of a clear mechanism should be cause for caution.
- How many conditions are treated by this one modality? The higher the number, the greater the cause for caution.
- Are specific claims made for the treatment, or, are the effects non-specific. For example, something like ‘improves general balance and well-being’ is a non-specific claim which is difficult to prove or disprove.
- Is the treatment generally accepted, or is it being promoted by a single individual or group? The latter is cause for caution.
- Do proponents of the treatment claim that broad acceptance of the treatment is limited due to a conspiracy to suppress the information?
Also some useful guidelines on the quality of evidence, which has the following characteristics:
- Anecdote: worthless, except as a starting point to investigate something interesting
- Preliminary: Some initial investigation has revealed a real effect which must now be tested properly
- Rigorous: As it suggests, rigorous testing is applied – generally double-blind, placebo controlled. This evidence can be either observational or experimental. Obviously the latter is preferred, but it is not always possible to do – for example, if testing life-saving drugs, giving one group a placebo is unethical
- Systematic Review: Finally a broad review of all the rigorous testing to assess whether all the findings are pointing to the same conclusions.
Keep these guidelines in mind next time you have a health decision to make based on claims being made.