Sign up for our Newsletter 
February 2014 – an interview with Kevin Thorpe
While both explanatory and pragmatic trials have a useful role to play, pragmatic trials are becoming exceedingly more attractive to some decision makers, notably, policy makers. In 2009, Kevin Thorpe led a team in the development of the PRECIS tool which helps researchers determine whether their trial design aligns with their end goals.
Kevin Thorpe is an assistant professor in the Dalla Lana School of Public Health at the University of Toronto and Head of Biostatistics at the HUB Health Research Solutions, Li Ka Shing Knowledge Institute of St. Michael’s. He teaches biostatistics in the Dalla Lana School of Public Health and has over 20 years of experience in clinical trials and more than 60 publications in peer-reviewed journals.
The following interview has been edited for clarity and brevity by Sarah Grant Alvarado.
What are the differences between pragmatic and explanatory trials?
At the most basic level, the explanatory trial seeks to answer the question ‘can this intervention work under the ideal circumstances?’ whereas the pragmatic trial seeks to answer the question ‘will this intervention work in usual practice?’. However, it’s important to realize that whether or not a trial is pragmatic or explanatory is not a dichotomous decision but rather a continuum of possibilities between “idealized” extremes.
For example, in an explanatory trial the patient criteria are typically going to be tightly controlled and the eligibility criteria fairly detailed. This results in a lot of exclusions since you are trying to maximize the chance of selecting a patient population in which the experimental treatment has the best chance to work. There are also likely to be standardized treatment regimens with careful attention paid to adherence.
The pragmatic approach is typically going to be less specific with regards to the patients enrolled and delivery of interventions in order to have a sense of how things are going to play out in general practice.
There is also an important distinction in how the results of these two types of trials are interpreted. For example, just because a highly explanatory trial declares that a treatment works is no guarantee that the treatment will work in general. However, a treatment that fails to “work” in the highly explanatory trial is also unlikely to “work” in practice. On the other hand, if a highly pragmatic trial declares a treatment “works” there is a very good chance that it will also work in the explanatory setting. However, if a treatment fails to “work” in a highly pragmatic trial that does not mean that there are no subpopulations in which it could work.
For a list of ‘Ten ways in which pragmatic and explanatory trials can differ’, read Kevin’s publication titled: A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers.
How do you know whether to use an explanatory or a pragmatic design for a clinical trial?
The answer to this ultimately comes back to the nature of the question you are trying to answer. For example, if you are studying an intervention about which very little is known, the primary interest is likely on understanding if the intervention has the potential to “work.” In such cases a more explanatory trial is indicated. On the other hand, if you are looking at treatments that are recommended based on other results and other trials, the question then becomes “does a particular intervention actually work when implemented in the general population?” In that situation, a more pragmatic trial design would be recommended.
How do you ensure your trial is designed to fit its purpose?
As stated earlier, whether an investigator chooses a pragmatic design over explanatory design depends on their ultimate goal. There are a number of trial design features that, depending on how they are designed, will either be more consistent with an explanatory approach or a pragmatic approach.
The PRECIS tool was designed to help trial designers think about the trial elements that separate explanatory approaches from pragmatic approaches so they can make design decisions on those elements that are more consistent with their goals.
I believe the tool is most useful when used as follows:
The principle investigator produces an initial protocol, guided by PRECIS to fit their intent, be it explanatory or pragmatic. Next, the co-investigators individually evaluate the protocol using the PRECIS tool. The results are compared, both overall, to see if the design goals (pragmatic/explanatory) have been met on the whole, and for consistency within the individual domains. Those domains that exhibit a lack of consistency are precisely the ones that need to be more clearly described in the protocol. The result of such a process should be a trial that matches the intent and clearly identifies those elements that are less in line with intent as well as a protocol that will be more consistently interpreted with regard to its place in the explanatory-pragmatic continuum.
To view PRECIS assessment example of 4 trials, read Kevin’s publication titled: A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers.
In order to meet the needs of the decision makers, what is the most important thing clinical researchers need to think about when designing their pragmatic trial?
First, it is important to realize there are different decision makers with rather different needs. For example, policy makers are often more interested in real-world effects of treatments and so pragmatic trials often carry greater relevance. On the other hand, a specialist may be more likely to see the kinds of patients frequently enrolled in more explanatory trials and they may also have the skill to deliver the treatments per-protocol and so explanatory trials may be more fitting for these folks. Regardless of the decision maker and the kind of trial one is designing, there are some common considerations.
It’s important to have the population defined as well as the delivery of the intervention, but I think the primary outcome of the study is particularly important. The primary outcome has to matter – it really needs be something that’s going to resonate with the decision makers. It’s not unreasonable to consider the economic analysis as well so they can look at things from a cost-benefit perspective, particularly for policy makers. I think it comes down to really making sure the trial reflects, as closely as possible, real world practice with an outcome that resonates clearly with the decision makers.
Ultimately, when designing the pragmatic trial the researcher should try and replicate, as closely as possible, the real world setting in which the treatment will be implemented. Furthermore, rigorous, highly pragmatic trials are more challenging than explanatory trials, but that is the price to pay for greater generalizability. Nevertheless, even the most pragmatic design is still likely to deviate from “real world.” This does not negate their value but serves as a reminder that all research has limitations and understanding those limitations is essential for the appropriate interpretation of research findings.
Featured publications by Kevin Thorpe:
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K. A pragmatic—explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Journal Clin Epidemiol 2009; 62: 464–475.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, Chalkidou K. A pragmatic—explanatory continuum indicator summary (PRECIS): a tool to help trial designers. Can Med Assoc J 2009; 180(10).
Maguire JL, Birken CS, Jacobson S, Peer M, Taylor C, Khambalia A, Mekky M, Thorpe KE, Parkin P. Office-Based Intervention to Reduce Bottle Use Among Toddlers: TARGet Kids! Pragmatic, Randomized Trial. Pediatrics 2010; 126:2.
Laupacis A, Lillie E, Dueck A, Straus S, Perrier L, Burton JM, Aviv R, Thorpe K, Feasby T, Spears J. Association between chronic cerebrospinal venous insufficiency and multiple sclerosis: a meta-analysis. Can Med Assoc J 2011; 183:16; doi:10.1503/cmaj.111074
Jaglal SB, Donescu OS, Bansod V, Laprade J, Thorpe K, Hawker G, Majumdar SR, Meadows L, Cadarett SM, Papaioannou A, Kloseck M, Beaton D, Bogoch E, Zwarenstein M. Impact of a centralized osteoporosis coordinator on post-fracture osteoporosis management: a cluster randomized trial. Osteoporos Int 2012 23:1; 87–95
Saposnik G, Lanthier S, Mamdani M, Thorpe KE, Melo M, Pope K, Selchan D, Moore DF. Fabry’s disease: A prospective multicenter cohort study in young adults with cryptogenic stroke. Int J of Stroke 2012; 7:3; 265–273.
Lau ANC, Tang T, Halapy H, Thorpe K, Yu CHY. Initiating insulin in patients with type 2 diabetes. Can Med Assoc J 2012; 184:7; doi:10.1503/cmaj.110779
Maguire JL, Birken CS, Khovratovich M, DeGroot J, Carsley S, Thorpe KE, Mamdani M, Parkin PC. Modifiable Determinants of Serum 25-Hydroxyvitamin D Status in Early Childhood. JAMA Pediatr 2013; doi:10.1001/2013.jamapediatrics.226