Internal validity of clinical trials is the extent to which you trust the methods that the results are based on. There are some simple methodological issues, such as the presence or absence of randomization, which allows you to understand whether the effect size is dramatically inflated or not due to differences in the baseline characteristics that can occur.
There are other important methods, also, that affect the results, such as blinding, allocation concealment and intention to treat. Now bear with me while I explain one of these that is often a confusing concept, the intention to treat analysis
In an intention to treat analysis the results are based on those initially assigned to the treatment and not just those who took the treatment (often referred to as per protocol analysis). The intention to treat analysis mainly leads to smaller effect sizes, often because the denominator is greater in the ITT group than the per protocol group (as people stop the treatment, sometimes they don’t take it as prescribed, they are lost to follow up and for a host of other reasons).
The key is that the folk who stopped taking the intervention may have gone onto have the adverse effect of interest. If you don’t analyze them, as in a per protocol analysis, you may be missing important adverse effects. This is exactly what happened in the Vioxx studies where per protocol analysis data presented to the FDA underestimated the true rates of heart attacks.
It is with surprize, and with some confusion that I have therefore seen the dramatic rise in the use of the ‘modified’ intention to treat analysis.
A 2010 systematic review in the BMJ analsysed modified intention to treat reporting in randomised controlled trials: at the time 475 RCTs used a modified intention to treat analysis. When the description was examined 40% reported one type of deviation from the intention to treat approach, 55% reported two or more types.
Is this blindingly obvious, or am I missing something: modified intention to treat is just a play on words, and in any other language it is per protocol analysis. Hence-with it should be removed from the published literature on clinical trials, as it sole purpose is to confuse readers in to missing an inherent bias in the trial methods.
In 2004 CONSORT warned using the term modified intention to treat may lead to confusion and inaccurate results. This what they say about its use:
‘The term “modified intention-to-treat” is quite widely used to describe an analysis that excludes participants who did not adequately adhere to the protocol, in particular those who did not receive a defined minimum amount of the intervention.(232) An alternative term is “per protocol.” Though a per protocol analysis may be appropriate in some settings, it should be properly labelled as a non-randomised, observational comparison. Any exclusion of patients from the analysis compromises the randomisation and may lead to bias in the results.
Does it surprize anyone that a 2011 analysis published in Trials, of 367 RCTs reported a strong association between trials classified as modified ITT and for-profit agency sponsorship. The adjusted odds ratio was 7.41 – Oh, and as well, author conflicts of interest were also strongly associated.
Peer reviewers, editors need to sort this out or at the very least modify something soon. Readers look out for the modified intention to treat analysis, it should come with a health warning.
No Comments