Importance of Mixed Methods in Pragmatic Trials and Dissemination and Implementation Research


      With increased attention to the importance of translating research to clinical practice and policy, recent years have seen a proliferation of particular types of research, including pragmatic trials and dissemination and implementation research. Such research seeks to understand how and why interventions function in real-world settings, as opposed to highly controlled settings involving conditions not likely to be repeated outside the research study. Because understanding the context in which interventions are implemented is imperative for effective pragmatic trials and dissemination and implementation research, the use of mixed methods is critical to understanding trial results and the success or failure of implementation efforts. This article discusses a number of dimensions of mixed methods research, utilizing at least one qualitative method and at least one quantitative method, that may be helpful when designing projects or preparing grant proposals. Although the strengths and emphases of qualitative and quantitative approaches differ substantially, methods may be combined in a variety of ways to achieve a deeper level of understanding than can be achieved by one method alone. However, researchers must understand when and how to integrate the data as well as the appropriate order, priority, and purpose of each method. The ability to demonstrate an understanding of the rationale for and benefits of mixed methods research is increasingly important in today's competitive funding environment, and many funding agencies now expect applicants to include mixed methods in proposals. The increasing demand for mixed methods research necessitates broader methodological training and deepened collaboration between medical, clinical, and social scientists. Although a number of challenges to conducting and disseminating mixed methods research remain, the potential for insight generated by such work is substantial.


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Academic Pediatrics
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Glasgow R.E.
        • Vinson C.
        • Chambers D.
        • et al.
        National Institutes of Health approaches to dissemination and implementation science: current and future directions.
        Am J Public Health. 2012; 102: 1274-1281
        • Szilagyi P.G.
        Translational research and pediatrics.
        Acad Pediatr. 2009; 9: 71-80
        • Thorpe K.E.
        • Zwarenstein M.
        • Oxman A.D.
        • et al.
        A Pragmatic-Explanatory Continuum Indicator Summary (PRECIS): a tool to help trial designers.
        CMAJ. 2009; 180: E47-E57
        • Rabin B.A.
        • Brownson R.C.
        Developing the terminology for dissemination and implementation research.
        in: Brownson R.C. Colditz G.A. Proctor E.K. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford University Press, New York, NY2012: 23-54
        • Rubenstein L.V.
        • Pugh J.
        Strategies for promoting organizational and practice change by advancing implementation research.
        J Gen Intern Med. 2006; 21: S58-S64
        • Curry S.J.
        Organizational interventions to encourage guideline implementation.
        Chest. 2000; 118: 40S-46S
        • Bergman D.A.
        • Beck A.
        Moving from research to large-scale change in child health care.
        Acad Pediatr. 2011; 11: 360-368
        • Saville A.W.
        • Albright K.
        • Nowels C.
        • et al.
        Getting under the hood: exploring issues that affect provider-based recall using an immunization information system.
        Acad Pediatr. 2011; 11: 44-49
        • Kempe A.
        • Saville A.W.
        • Eisert S.
        • et al.
        Population-based versus practice-based recall for childhood immunizations: a randomized controlled comparative effectiveness trial.
        Am J Public Health. 2013; 103: 1116-1123
        • Lingard L.
        • Albert M.
        • Levinson W.
        Grounded theory, mixed methods, and action research.
        BMJ. 2008; 337: a567
        • Teddlie C.
        • Tashakkori A.
        Major issues and controversies in the use of mixed methods in the social and behavioral sciences.
        in: Teddlie C. Tashakkori A. Handbook of Mixed Methods in the Social and Behavioral Sciences. Sage Publications, Thousand Oaks, Calif2003: 3-50
        • Proctor E.K.
        • Landsverk J.
        • Aarons G.
        • et al.
        Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges.
        Adm Policy Ment Health. 2009; 36: 24-34
        • Landsverk J.
        • Brown C.H.
        • Rolls R.J.
        • et al.
        Design elements in implementation research: a structured review of child welfare and child mental health studies.
        Adm Policy Ment Health. 2011; 38: 54-63
        • Glenton C.
        • Lewin S.
        • Scheel I.B.
        Still too little qualitative research to shed light on results from reviews of effectiveness trials: a case study of a Cochrane review on the use of lay health workers.
        Implement Sci. 2011; 6: 53
        • Wisdom J.P.
        • Cavaleri M.A.
        • Onwuegbuzie A.J.
        • et al.
        Methodological reporting in qualitative, quantitative, and mixed methods health services research articles.
        Health Serv Res. 2012; 47: 721-745
        • Miller W.L.
        • Phillips W.R.
        • Acheson L.S.
        • et al.
        New knowledge for and about primary care: a view through the looking glass of the Annals of Family Medicine.
        Ann Fam Med. 2005; 3: 197
        • Drahota A.
        • Aarons G.A.
        • Stahmer A.C.
        Developing the Autism Model of Implementation for autism spectrum disorder community providers: study protocol.
        Implement Sci. 2012; 7: 85
        • Raymond K.P.
        • Fiese B.H.
        • Winter M.A.
        • et al.
        Helpful hints: caregiver-generated asthma management strategies and their relation to pediatric asthma symptoms and quality of life.
        J Pediatr Psychol. 2012; 37: 414-423
        • Tsimicalis A.
        • Stevens B.
        • Ungar W.J.
        • et al.
        Determining the costs of families’ support networks following a child’s cancer diagnosis.
        Cancer Nurs. 2013; 36: E8-E19
        • Webb C.M.
        • Collin S.M.
        • Deave T.
        • et al.
        What stops children with a chronic illness accessing health care: a mixed methods study in children with chronic fatigue syndrome/myalgic encephalomyelitis (CFS/ME).
        BMC Health Serv Res. 2011; 11: 308
        • Iloabachie C.
        • Wells C.
        • Goodwin B.
        • et al.
        Adolescent and parent experiences with a primary care/Internet-based depression prevention intervention (CATCH-IT).
        Gen Hosp Psychiatry. 2011; 33: 543-555
        • Swallow V.M.
        • Allen D.
        • Williams J.
        • et al.
        Pan-Britain, mixed-methods study of multidisciplinary teams teaching parents to manage children’s long-term kidney conditions at home: study protocol.
        BMC Health Serv Res. 2012; 12: 33
        • Gorzkowski J.
        • Kelly E.H.
        • Klaas S.J.
        • et al.
        Obstacles to community participation among youth with spinal cord injury.
        J Spinal Cord Med. 2011; 34: 576-585
        • Tucker J.S.
        • Sussell J.
        • Golinelli D.
        • et al.
        Understanding pregnancy-related attitudes and behaviors: a mixed-methods study of homeless youth.
        Perspect Sex Reprod Health. 2012; 44: 252-261
        • Patton M.Q.
        Qualitative Evaluation and Research Methods.
        3rd ed. Sage Publications, Thousand Oaks, Calif2002
        • Palinkas L.A.
        • Aarons G.A.
        • Horwitz S.
        • et al.
        Mixed method designs in implementation research.
        Adm Policy Ment Health. 2011; 38: 44-53
        • Landsverk J.
        • Brown H.
        • Chamberlain P.
        • et al.
        Design and analysis in dissemination and implementation research.
        in: Brownson R.C. Colditz G.A. Proctor E.K. Dissemination and Implementation Research in Health: Translating Science to Practice. Oxford University Press, New York, NY2012: 23-54
        • Stange K.C.
        • Crabtree B.F.
        • Miller W.L.
        Publishing multimethod research.
        Ann Fam Med. 2006; 4: 292-294
        • Creswell J.W.
        • Fetters M.D.
        • Ivankova N.V.
        Designing a mixed methods study in primary care.
        Ann Fam Med. 2004; 2: 7-12
        • Creswell J.W.
        • Plano Clark V.
        Designing and Conducting Mixed Methods Research.
        Sage Publications, Thousand Oaks, Calif2007
        • Marshall T.
        • Rapp C.A.
        • Becker D.R.
        • et al.
        Key factors for implementing supported employment.
        Psychiatr Serv. 2008; 59: 886-892
        • Marty D.
        • Rapp C.
        • McHugo G.
        • et al.
        Factors influencing consumer outcome monitoring in implementation of evidence-based practices: results from the National EBP Implementation Project.
        Adm Policy Ment Health. 2008; 35: 204-211
        • Rapp C.A.
        • Etzel-Wise D.
        • Marty D.
        • et al.
        Barriers to evidence-based practice implementation: results of a qualitative study.
        Community Ment Health J. 2010; 46: 112-118
        • Creswell J.W.
        Educational Research: Planning, Conducting, and Evaluating Quantitative and Qualitative Research.
        4th ed. Pearson Education, Boston, Mass2012
        • Povee K.
        • Roberts L.
        • Bourke J.
        • et al.
        Family functioning in families with a child with Down syndrome: a mixed methods approach.
        J Intellect Disabil Res. 2012; 56: 961-973
        • Quarmby T.
        • Dagkas S.
        • Bridge M.
        Associations between children’s physical activities, sedentary behaviours and family structure: a sequential mixed methods approach.
        Health Educ Res. 2011; 26: 63-76
        • Lee G.M.
        • Santoli J.M.
        • Hannan C.
        • et al.
        Gaps in vaccine financing for underinsured children in the United States.
        JAMA. 2007; 298: 638-643
        • Tan A.C.
        • Rehfuss M.C.
        • Suarez E.C.
        • et al.
        Nonsuicidal self-injury in an adolescent population in Singapore.
        Clin Child Psychol Psychiatry. 2012 Dec 3; ([Epub ahead of print])
        • Creswell J.W.
        Mapping the field of mixed methods research [editorial].
        J Mixed Methods Res. 2009; 3: 95-108
        • Aarons G.A.
        • Fettes D.L.
        • Sommerfeld D.H.
        • et al.
        Mixed methods for implementation research: application to evidence-based practice implementation and staff turnover in community-based organizations providing child welfare services.
        Child Maltreat. 2012; 17: 67-79
        • Wisdom J.
        • Creswell J.W.
        Mixed Methods: Integrating Quantitative and Qualitative Data Collection and Analysis While Studying Patient-Centered Medical Home Models.
        Agency for Healthcare Research and Quality, Rockville, Md2013 (Available at:) (Accessed June 3, 2013)
        • Glasgow R.E.
        • Vogt T.M.
        • Boles S.M.
        Evaluating the public health impact of health promotion interventions: the RE-AIM framework.
        Am J Public Health. 1999; 89: 1322-1327
        • Creswell J.
        • Klassen A.
        • Plano Clark V.
        • et al.
        Best Practices for Mixed Methods Research in the Health Sciences.
        Office of Behavioral and Social Sciences Research, National Institutes of Health, Bethesda, Md2012 (Available at:) (Accessed March 21, 2012)
        • Alexander J.A.
        • Hearld L.R.
        Methods and metrics challenges of delivery-system research.
        Implement Sci. 2012; 7: 15
        • Bouchard K.
        • Dubuisson W.
        • Simard J.
        • et al.
        Systematic mixed-methods reviews are not ready to be assessed with the available tools.
        J Clin Epidemiol. 2011; 64: 926-928