If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Address correspondence to Kathryn M. McDonald, MM, Stanford University, Center for Health Policy, Center for Primary Care and Outcomes Research, 117 Encina Commons, Stanford, CA 94305-6019.
Growing consensus within the health care field suggests that context matters and needs more concerted study for helping those who implement and conduct research on quality improvement interventions. Health care delivery system decision makers require information about whether an intervention tested in one context will work in another with some differences from the original site. We aimed to define key terms, enumerate candidate domains for the study of context, provide examples from the pediatric quality improvement literature, and identify potential measures for selected contexts. Key sources include the organizational literature, broad evaluation frameworks, and a recent project in the patient safety area on context sensitivity. The article concludes with limitations and next steps for developments in this area.
Making mental connections is our most crucial learning tool, the essence of human intelligence; to forge links; to go beyond the given; to see patterns, relationships, context.Marilyn Ferguson
HEALTH CARE DELIVERY is highly complex, from the patient–clinician encounter to the organizations in which these encounters occur, and is ultimately embedded in and affected by an even more varied environment. This situation makes delivering consistently high-quality care challenging for its participants and ripe for opportunities for improvement. Researchers and practitioners together develop and test quality improvement (QI) strategies, often inside of organizations, common test beds for QI research. Delivery organizations also implement strategies that have succeeded elsewhere. What evidence does an organization (eg, a children's hospital, a patient-centered medical home) need to know about whether a strategy researched in one place will work in another? Efforts to improve quality of health care occur in many different situations and may work better in some and worse in others.
McDonald K, Chang C, Schultz E. Through the Quality Kaleidoscope: Reflections on the Science and Practice of Improving Health Care Quality. Closing the Quality Gap: Revisiting the State of the Science. Methods Research Report. Prepared by Stanford-UCSF Evidence-based Practice Center under Contract No. 290-2007-10062-I. Rockville, Md: Agency for Healthcare Research and Quality; February 2013. AHRQ Publication No. 13-EHC041-EF. Available at: http://effectivehealthcare.ahrq.gov/ehc/products/509/1406/CQG-Reflections-on-Science-130219.pdf. Accessed June 13, 2013.
McDonald KM, Chang C, Schultz E. Closing the Quality Gap: Revisiting the State of the Science. Summary Report. (Prepared by Stanford-UCSF Evidence-based Practice Center under Contract No. 290-2007-10062-I.) AHRQ Publication No. 12(13)-E017. Rockville, Md: Agency for Healthcare Research and Quality; January 2013. Available at: http://effectivehealthcare.ahrq.gov/ehc/products/496/1375/ClosingtheQualityGap_SummaryReport_20130109.pdf. Accessed June 13, 2013.
Just as medical treatment is indicated for some patients and not others, interventions aimed at quality improvement may be indicated for some organizations and not others. Appropriate selection of quality improvement strategies requires information about context. But what exactly is context, and how should it be measured to be useful in research and for interpretation in organizational decision making about quality improvement investments?
Drawing from growing consensus that context matters and needs more concerted study for helping quality improvement decision makers, we sought to define key concepts for the study of context, offer approaches to measure and report context in pediatric quality improvement research, and assess limitations and next steps for future research in this area.
Early research on improving quality and safety focused on efficacy, without much consideration of context, sparking debates about the relative value of different approaches.
For concrete, easily packaged, and describable quality improvement interventions (QII), such a traditional evidence-based medicine approach, may be applicable: observe a system, introduce a perturbation (the intervention) to some participants but not others, and then observe again. The experiment is designed to determine if the changes in outcome are attributable to the perturbation (with statistical confidence limits). Yet most experiments are short on details about how the intervention works, including the influence of context. For more complex interventions developed to improve quality, the field is moving to incorporate complementary research paradigms that study how context enables, thwarts, and interacts with interventions.
A useful alternative evaluation model comes from Pawson and Tilley's realist evaluation framework of context, mechanism, and outcome (ie, the CMO model), where “programs work (have successful ‘outcomes’) only insofar as they introduce the appropriate ideas and opportunities (‘mechanisms’) to groups in the appropriate social and cultural conditions (‘contexts’).”
This model makes the context sensitivity of many QIIs explicit and applies whenever local adaptations are expected to influence outcome patterns. Defining context and considering when and how to study this concept effectively become paramount for assessing transferability of an intervention package from one context to another.
For the purpose of our focus on context sensitivity, a broad definition of a QII is used in order to inform research on efforts to improve quality of pediatric care comprehensively. On the basis of the Robert Woods Johnson Foundation's definition used for evaluating training programs in this domain, quality improvement is “the formal approach to the analysis of performance and systematic efforts to improve it.”
This definition of quality improvement is not restricted to any specific level of analysis (eg, performance could be assessed for a doctor's practice, an emergency department, a hospital, or a geographic area). An intervention is a newly created attempt to improve the performance of the specified unit over time, and it can be described in terms of what could potentially be packaged and replicated elsewhere.
Cochrane Collaboration. Cochrane Effective Practice and Organisation of Care Group. 2012. Available at: http://epoc.cochrane.org/. Accessed December 9, 2012.
As such, they are sometimes termed strategies, initiatives, or programs. Interventions can target any or all of the Institute of Medicine's quality domains (safety, effectiveness, efficiency, timeliness, equity, and patient-centeredness) where performance is lacking.
If the point is to improve health care quality, then all of these efforts fall under the QII research umbrella when considering ways to conceptualize, measure, and interpret potential contextual effects. For simplicity's sake, implementation elements that can be described, packaged, and replicated from one environment to another may be considered part of the QII. In this case, the mechanism in the realist CMO model is the QII and its implementation. Another tactic is to distinguish the content of a QII from its implementation, and consider the contextual influences on each (QII and implementation) separately and interactively.
What Is Context?
Simply put, context (ie, the C in the realist CMO model) is potentially everything else—anything that cannot be described as part of the QII or the outcomes of interest (ie, the M and the O in the CMO model). But only those contextual factors hypothesized to affect the success of the intervention need to be studied for a given QII, after careful consideration of potential factors and competing hypotheses. Which contextual factors are most likely to matter? Which factors may influence the mechanisms and outcomes favorably or unfavorably? How does one tell a priori? What parts of the QII may require adaptation from one local setting to the next, and one time period to another? The overarching question to ask at the outset of a QII evaluation is, what are the local conditions that could influence the outcomes of interest? The line between the QII and the local conditions may be clear initially and blur during implementation of the intervention. For better knowledge translation, those involved in QII implementation and evaluation need to anticipate potential scale-up and spread to other environments as they describe the QII versus the context in which it was tested. Any blurring between context and QII is often related to perspective. If QII developers believe that local leadership engagement is important to QII success, they may spend time up front gaining buy-in for the new program (the QII). If commitment of a certain package of resources is a prerequisite for implementing the intervention, then the presence of that commitment can be seen as a contextual variable (eg, money and personnel time committed). Alternatively, the QII may be designed with an initial leadership intervention component (eg, letter to the hospital CEO from the investigative team noting project backing from a major health plan in the area) aimed at gaining a particular level of commitment.
More concretely, the Figure depicts 3 scenarios that have differing combinations of QII components and contexts, but are all effectively the same. The first scenario (labeled “original hospital”) reflects a study by McKee and colleagues,
who describe an improvement model to reduce catheter-associated bloodstream infections at a children's hospital with 6 intervention components. Organizational commitment (ie, enthusiastic leadership, adequate resources) is implied as a contextual factor. In addition, the study authors describe a context responsive to catheter product issues occurring during a period of time when infection rates climbed. In one alternative hypothetical scenario (hospital A), a hospital might implement only 2 components (ie, the checklist and the dedicated cart) because their nurses have already received empowerment training and their physicians have already experienced the necessary type of insertion training. They may also have a real-time infection rate feedback system to the unit leadership. Their QII implementation is simplified in response to their local conditions. On the opposite end of the spectrum, another hospital (hospital B) might choose to implement the originally described QII along with extra interventions to compensate for limitations in their context.
FigureQuality improvement intervention (QII) and context distinctions vary depending on setting.
Thus, context is what is left over after specifying the QII (with implementation components) and may in fact vary from one location (context) to another.
Such a simplified definition should not mask the challenges inherent in understanding the interplay between a QII and the context, where components of a QII may be tailored to a particular context, and in turn, the context may adapt during implementation of the intervention. The active aspects of the interaction theoretically may differ in structure and impact depending on specifics of the situation, including whether a factor is viewed as endogenous or exogenous to the intervention. There is a need for clear and detailed specification of each part of the QII and its context in any evaluative project, as well as particulars about the ways that each influenced the other, as was reported qualitatively in the McKee et al study.
Three main audiences increasingly care about context sensitivity of QIIs. Practitioners need to know what QIIs are suitable for implementation in their delivery settings and how to increase the chances of the intervention working. Policy makers need to create incentives to encourage adoption and spread of effective QIIs to the appropriate contexts. Researchers aim to understand context sensitivity in order to help practitioners and policy makers choose wisely among QIIs. The accretion of evidence on a QII results from conducting different types of studies to assess proof of concept, effectiveness in single settings, replication to other settings, and ultimately widespread implementation. These 4 types of research orientations elicit different key framing questions about context (Table 1). The remainder of the article introduces a sampling of commonly used frameworks for identifying relevant contextual domains and measurement tools available for some variables related to context.
Table 1Contextual Considerations According to Research Phase
Research Phase
Typical Approach/Goal
Context Question
Process Evaluation/Proof of Concept for a QII
Develop logic model of mechanisms related to improving quality target
What are the enablers and barriers to potential interventions working in our context?
Initial QII Evaluation
Use logic model: design, implement, and test intervention in one setting to evaluate effectiveness
What context factors should we describe, measure, and/or monitor?
Further QII Evaluation for Promising Interventions
Test with more rigorous methods, expand to more than one setting to evaluate effectiveness and context sensitivity
What context factors should we measure consistently across sites?
Scale-up and Spread of Effective QII
Design scale-up plan; implement and monitor over time
Which context factors that influence effectiveness are fixed vs mutable?
Contexts to Consider for Applications to Pediatric QII
Regardless of the particular research question, we need ways to winnow down the long list of potential contexts surrounding an intervention to those that may exert influence on the success (or failure) of a QII. The organizational theory literature, several health care improvement frameworks and a recent interdisciplinary RAND-led project on patient safety practices provide some direction.
How does context affect interventions to improve patient safety? An assessment of evidence from studies of five patient safety practices and proposals for research.
Shekelle PG, Pronovost PJ, Wachter RM, et al., and the PSP Technical Expert Panel. Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria (Prepared by RAND Health under Contract No. HHSA-290-2009-10001C). AHRQ Publication No. 11-0006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2010. Available at: http://www.ahrq.gov/research/findings/final-reports/contextsensitive/context.pdf. Accessed June 13, 2013.
A rich source for generating pertinent contextual domains, organizational studies emphasize levels of action (eg, macro view of organizations and their environments, or micro view looking at behavior patterns within an organization), interrelationships (multilevel effects), and field observation for theory testing. For any given QII, therefore, we can start by asking what factors within the macro context might exert influence on the improvement effort, repeat this thought experiment for the micro context, and then hypothesize any interactions between these levels. Some macro analysis categories include: national/regional/local economy and market forces, urban/suburban/rural location, governmental and regulatory variables, and time (when the study was conducted, relevant external events during the study period).
At the micro level, possible contextual factors relate to the organization and subunits or teams. The organization level factors potentially applicable to quality improvement include type of health care entity, life cycle stage of entity (eg, new vs established, recently restructured or acquired), structural factors (eg, disciplinary departments vs care delivery processes), financial support for quality improvement in form of actual allocations and/or dedicated human resources, organizational culture (dominant, subgroups, norms/values), management interest in QII, and leadership levels involved (top level, middle management), governance leadership, and learning climate.
Contextual factors affecting job satisfaction and organizational commitment in community mental health centers undergoing system changes in the financing of care.
A front-line view focuses on what some have termed the clinical microsystem, where patient and provider encounters occur, and from which the following additional contexts may affect QII implementation success: staff factors (eg, focus on hiring decisions and integration into team, performance expectations, education and training, interdependence of care team), patient factors (focus on patient needs, focus on community needs), performance factors (data feedback systems process improvement strategies), and information factors (technology, communication).
The specific innovation processes contained in a QII typically occur at the micro level by individuals or groups of health care professionals; these micro level processes are supported, enhanced, or prevented by the macro conditions.
Kanter, RM. When a Thousand Flowers Bloom: Structural, Collective, and Social Conditions for Innovation in Organizations. In Myers, PS, ed: Knowledge Management and Organizational Design. Newton, MA: Butterworth-Heinemann; 1996:93–132.
The interactions between micro and macro levels can influence the QII's success during early stages of implementation and especially during scale-up and spread of the intervention. Behavioral and organizational context factors also influence sustainability of QII, with several intervention–context interactions suggested by researchers from the Department of Veterans Affairs Quality Enhancement Research Initiative on the basis of analysis of long-term effects of QIIs: intervention fit, intervention fidelity, intervention dose, and intervention target.
The review of 79 QI interventions for asthma in children found that many different QIIs (eg, promotion of self-monitoring/self-management, patient education to increase asthma knowledge or inhaler technique without targeting patient behavior change, organizational change) improve process and outcomes of care. The only context factors available from all studies were time related (evaluations from 1974 to 2004, study duration from 4 weeks to 5 years), region of performance (most in the United States [n = 46], Australia [n = 7], the United Kingdom [n = 6], the Netherlands [n = 4], or Canada [n = 4]), and setting (eg, outpatient clinics, home, and school). Only 2 contextual factors were associated with QII outcomes; the longer the study for the self-management subgroup, the greater the expected reduction in asthma-related school absenteeism, and the more recent the study for some QII subgroups, the better the results. In the accompanying evidence report, the review provided a narrative synthesis of findings by setting.
For example, patient education QIIs targeted at children, their families, and other adults were subdivided into those carried out in part or fully in a school, doctor's office, or home environment. Some of the designs of these QIIs followed behavior change theory and intervened in these different environmental contexts as a result. Systematic reviews offer an opportunity to balance internal validity with applicability to specific contexts. Currently, however, primary QII studies often provide limited information to allow a rich analysis of context across studies.
Existing Health Care Improvement Frameworks
Two complementary frameworks developed for health care improvement applications are described briefly: the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) framework, and the Consolidated Framework for Implementation Research (CFIR), along with several pediatric QII applications.
The RE-AIM framework was developed as a tool for planning, evaluation, and reporting of research translation into practice projects. With an orientation toward all phases of translation of complex interventions, it steps one through contextual factors at stages of program or intervention analysis applicable to QIIs. It has been applied widely to health promotion programs, with pediatric applications mostly school-based public health programs (eg, to increase physical activity).
Applications to systems-oriented improvement interventions include use of RE-AIM to develop and evaluate a Web-based behavior and self-management tool within the patient-centered medical home setting.
Some sample contextual factors elicited from the framework application are practice workflow considerations (“adoption” context), and integration with National Committee for Quality Assurance recommendations and clinical pathways (“maintenance” context within the framework).
Users of RE-AIM often write up their findings from mixed-methods research for each of the evaluation domains (reach, effectiveness, adoption, implementation, maintenance). This framework is particularly applicable to the scale-up and spread phase of QIIs (Table 1).
The CFIR comprises constructs from 19 published implementation theories.
The intent is complementary to the needs of pediatric QII researchers and practitioners—a structure for identifying candidate constructs to potentially measure or otherwise observe during different phases of QII research and adoption. The framework has 5 topic areas: interventions characteristics, outer setting, inner setting, characteristics of individuals, and process. The first and last topics relate more to the intervention and implementation, while the other 3 more closely tie to context domains. The distinctions between intervention, context (inner, outer settings, and individuals acting in each), and the process of implementation can be transient depending on circumstances, so that the entire framework may be useful for identifying relevant contexts.
it is not surprising to see alignment with the organizational literature's approach (eg, the micro level captures inner setting and individuals, and macro level relates to the outer setting). The CFIR domains are expanded and explained with detailed appendix materials and mappings to original sources to support application for future QII research. It seems that the CFIR is too new to have a published application of a pediatric QII. Table 2 supplies an application based on information available from a recently published QII on reducing pediatric identification band errors.
Table 2Application of the Consolidated Framework for Implementation Research (CFIR) to a Pediatric Quality Improvement Intervention (QII)
Pediatric Identification (ID) Band Errors Quality Collaborative Goals: Spread QII from a single children's hospital across 6 sites to reduce pediatric patient ID band error rates. Decrease ID band error rate by 50%. Demonstrate learning across institutions drives results.
CFIR Context Domains
Domain Description Example from Pediatric QII Collaborative
OUTER SETTING
Patient Needs and Resources
The extent to which patient needs, barriers and facilitators known and prioritized
•
Issues with ID band discomfort and resultant removal identified by baseline survey. One size/type of ID band not good for all children. In response, some hospitals worked with vendors to develop bands for vulnerable patients (skin integrity, difficulty keeping band in place in NICU patients).
•
Tailoring original site materials (patient and family educational handouts about the importance of the patient ID band).
Cosmopolitanism
Degree organization is networked with other external organizations
The collaborative included some variety in organizations
Peer Pressure
Mimetic or competitive pressure to implement QII
No information on this context.
External Policy and Incentives
External strategies to spread interventions
Improving patient ID first goal of the Joint Commission's National Patient Safety Goals since 2003.
INNER SETTING
Structural Characteristics
Social architecture, age, maturity, and size of an organization
Freestanding children's hospitals; children's hospitals within academic medical centers; community hospitals. Data on size included.
Networks and Communications
Nature and quality of social networks, formal and informal communication
Six collaborating organizations, monthly collaborative calls to prioritize “key drivers” (reasons for problem) using baseline data and employee survey information. Used drivers to brainstorm potential interventions.
Culture
Norms, values, and basic assumptions of organization
Noted need for further work to understand a specific culture impediment (tendency for caregivers to assume patient is already correctly banded).
Implementation Climate
Capacity for change, shared receptivity to QII, extent to which use QII rewarded, supported, and expected within organization
•
One site used audits as part of a manager incentive program
•
Sharing stories of band errors and patient impact helped staff understand importance of patient ID bands.
Readiness for Implementation
Tangible indicators of organizational commitment to QII
•
Leadership engagement noted on safety walk-rounds.
•
For all the institutions, collaboration among nurses, physicians, and quality professionals ensured collaborative was a priority. Data sharing at all levels.
INDIVIDUAL CHARACTERISTICS
Knowledge and Beliefs About the Intervention
Individuals' attitudes toward and familiarity with QII
Survey used to identify attitudes at outset (eg, belief that checking the band not needed because “I know my patient”).
Self-efficacy
Individual belief in their own capabilities to act
Engaging all levels strengthened desire to change and will to sustain gains made.
Individual Stage of Change
Phase an individual is in, in terms of progresses toward skilled, enthusiastic, and sustained use of QII
Each “failure” was an opportunity to provide just-in-time education to participants (staff, family) about importance of ID bands to safe care.
Individual Identification With Organization
Broad construct related to individual's perception of and commitment to the organization
No information on this context.
Other Personal Attributes
Broad construct including tolerance of ambiguity, intellectual ability, motivation, values, competence, capacity, and learning style
This project reviewed the relative merits of different methods for evaluating context-sensitive effectiveness of interventions based on literature reviews to identify contextual factors and structured input from an interdisciplinary panel of experts comprising international patient safety leaders, clinicians, policy makers, social scientists, and methodologists.
Shekelle PG, Pronovost PJ, Wachter RM, et al., and the PSP Technical Expert Panel. Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria (Prepared by RAND Health under Contract No. HHSA-290-2009-10001C). AHRQ Publication No. 11-0006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2010. Available at: http://www.ahrq.gov/research/findings/final-reports/contextsensitive/context.pdf. Accessed June 13, 2013.
Four key domains of context with associated factors were prioritized (Table 3), with one domain corresponding to outer settings in the CFIR structure and 3 domains connected to the inner setting. The expert panel judged the importance of assessing each factor relative to 5 representative patient safety practices separately. Table 3 provides the panel assessments of potential importance of these context factors for interventions to improve medication reconciliation (ie, the QII, with patient safety as the target for improvement). The project also recommended combined research approaches that simultaneously seek rigor in effectiveness estimation, report key contextual factors, and, if possible, assess contextual influences on the outcomes for different research goals (eg, effectiveness, implementation experience, and adoption or spread).
Y indicates yes; N, no, not expected to influence effectiveness of medication reconciliation interventions; based on technical expert panel input (see RAND report for details19).
4. Implementation and Management Tools [Inner Setting: Implementation Climate]
Staff education and training
Designated staff time to implement
Use of audit and feedback
Person responsible for implementation
Internal incentives
Local tailoring or iterative process
Help desk support
Extent of project management
Timeline of implementation
Implementation process—one unit at a time or all at once
Y
Y
Y
Y
N
Y
N
N
N
N
∗ Y indicates yes; N, no, not expected to influence effectiveness of medication reconciliation interventions; based on technical expert panel input (see RAND report for details
Shekelle PG, Pronovost PJ, Wachter RM, et al., and the PSP Technical Expert Panel. Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria (Prepared by RAND Health under Contract No. HHSA-290-2009-10001C). AHRQ Publication No. 11-0006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2010. Available at: http://www.ahrq.gov/research/findings/final-reports/contextsensitive/context.pdf. Accessed June 13, 2013.
Deciding which contextual factors to include in a study is based on theoretical and practical considerations. From a theory-based perspective, a logic model can help prioritize the contextual factors that are deemed most likely to enhance or limit the QII's success with respect to the specific research goals. It is also worthwhile (for future adopters and systematic reviewers) to describe any contextual factors that may not vary in the tested environment, but could be different and have an effect in another implementation setting. Practically, the key decision driver is feasibility with respect to measurability (measurement tool availability for the construct), its burden to those assessed when measurement is based on survey methods and costs to the evaluation team. Some contexts are relatively easy to measure or specify, such as location (rural or urban; country), organizational type (academic or community; nursing home, outpatient clinic, hospital), and applicable regulatory requirements (public reporting of immunization rates). A systematic review by Kaplan et al identified 47 QII articles and classified 66 contextual factors and the specific measures used in the primary studies.
Other contexts pose measurement challenges or choices with potential tradeoffs. The patient safety practices project identified some measurement tools for 4 context areas (teamwork, leadership, patient safety culture, and organizational complexity) based on an extensive literature search in health care databases.
Shekelle PG, Pronovost PJ, Wachter RM, et al., and the PSP Technical Expert Panel. Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria (Prepared by RAND Health under Contract No. HHSA-290-2009-10001C). AHRQ Publication No. 11-0006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2010. Available at: http://www.ahrq.gov/research/findings/final-reports/contextsensitive/context.pdf. Accessed June 13, 2013.
Because of the limited measures netted from this source for organizational complexity measures, the team also conducted a limited search of the organizational literature. Many options exist for teamwork, leadership, and culture, though some are setting specific and more or less labor intensive, while others are still under development. The patient safety practices expert group supported several specific measures for the types of applications in that project, and these choices may be applicable (or adaptable with further measure development effort) to the pediatric QII domain as well: for teamwork and leadership, the Shortell et al ICU Nurse–Physician Questionnaire, for leadership alone, either the Leadership Practice Inventory, or the Practice Environment Scale.
In addition, the Multifactor Leadership Questionnaire (MLQ), which is used to assess transformational learning style in nurse executives, has been developed and tested in children’s hospitals.
Consensus was lacking for a choice among the patient safety culture measures, and more general measures of organizational culture applicable to quality improvement were not assessed. Since that project's report, a comparison showed similar reliability and predictive validity of 2 widely used culture surveys: Safety Attitudes Questionnaire (SAQ) and Hospital Survey on Patient Safety Culture (HSOPS).
Few options were identified for organizational complexity, and the only ones applied to health care were limited to the domain of care coordination, where other measures resources exist.
An important step in strengthening the research base on the role of context is uptake of the context elements of SQUIRE (Standards for Quality Improvement Reporting Excellence) guidelines for publishing studies of QIIs.
Specifically, these guidelines are divided into 19 items (eg, abstract, analysis, funding sources), of which 6 explicitly note contextual factors. The guidelines for SQUIRE item 5, “intended improvement,” should include “who (champions, supporters) and what (events, observations) triggered the decision to make changes, and why now (timing).” Item 8, “setting,” should specify “how elements of the local care environment considered most likely to influence change/improvement in the involved site or sites were identified and characterized.” Item 9, “planning the intervention,” should include information on factors contributing to the choice of the intervention (ie, “suitability of the intervention for the specific setting”), as well as factors related to those implementing the intervention (ie, “roles, qualification and training”). Item 11, “methods of evaluation,” should describe the “contribution of … context factors to effectiveness of the intervention.” Item 13, “outcomes,” should characterize “relevant elements of setting or settings (for example, geography, physical resources, organizational culture, history of change efforts), and structures and patterns of care (eg, staffing, leadership) that provided context for the intervention” and present “evidence regarding the strength of association between observed changes/improvements and intervention components/context factors.” Finally, Item 17, “interpretation,” should draw inferences, “paying particular attention to components of the intervention and context factors that helped determine the intervention's effectiveness (or lack thereof), and types of settings in which this intervention is most likely to be effective.” These items collectively offer an initial context sensitivity checklist for researchers.
Conclusions and Next Steps
Efficient and effective means to incorporate the domain of context into research on QIIs has received relatively minimal attention in health care, even though the salience of this broad topic is well understood by practitioners and policy makers.
This article should be looked on as a sampling of materials to offer potential, but not exhaustive, starting points for further development of domain lists and measurement options for a given construct.
Evaluations of complex interventions directed at improving quality need to consider QIIs in terms of core transferable elements and adaptive aspects that respond to a local context.
Teasing apart the interrelationships requires a more dedicated focus on defining context and developing a strategy to assess its influence on and interactions with interventions. Researchers and practitioners whose attention is directed to child health already have a solid appreciation for issues of translating evidence from one domain to another, knowing that what works for adults will not necessarily apply to children. Complex QII are analogous: they may translate from one context to another, but such a conclusion is premature without considering the logic behind the likelihood of intervention mobility, the possible effects of context, and the need for triangulating the evidence from different types of research studies and practical experiences. Understanding context influence is akin to diagnosis. Differential diagnosis of context is an ongoing process to assure that no important context is missed, and this review of resources brings together concepts, domains and measures to consider the influence of specific contexts on QIIs. Further systematic and collaborative work among those who evaluate new QIIs and those who decide whether to implement a promising QII could lead to the development of context evaluation tools tailored to pediatric applications for research and quality improvement.
Funders and researchers may consider developing repositories of measurement tools for the context domains of importance to QII, noting any that are particularly relevant to pediatric applications. For the national priority area of care coordination, the Agency for Healthcare Quality and Research sponsored a project to develop a Care Coordination Measures Atlas, a resource that includes a measurement framework, identified measures with acceptable performance characteristics and maps of these measures to framework domains.
A similar resource would be useful for those involved in QII evaluations from proof of concept through spread of successful interventions with widespread applicability (ie, intervention exhibits limited context sensitivity or workarounds to the local context are well understood). Finally, reporting context and the QII in ways aligned with the SQUIRE guidelines
is crucial to advancing knowledge about what interventions work, and in what ways context sensitivity may play a role in success or failure of a particular QII.
References
Bradley E.H.
Nembhard I.M.
Yuan C.T.
et al.
What is the experience of national quality campaigns? Views from the field.
McDonald K, Chang C, Schultz E. Through the Quality Kaleidoscope: Reflections on the Science and Practice of Improving Health Care Quality. Closing the Quality Gap: Revisiting the State of the Science. Methods Research Report. Prepared by Stanford-UCSF Evidence-based Practice Center under Contract No. 290-2007-10062-I. Rockville, Md: Agency for Healthcare Research and Quality; February 2013. AHRQ Publication No. 13-EHC041-EF. Available at: http://effectivehealthcare.ahrq.gov/ehc/products/509/1406/CQG-Reflections-on-Science-130219.pdf. Accessed June 13, 2013.
McDonald KM, Chang C, Schultz E. Closing the Quality Gap: Revisiting the State of the Science. Summary Report. (Prepared by Stanford-UCSF Evidence-based Practice Center under Contract No. 290-2007-10062-I.) AHRQ Publication No. 12(13)-E017. Rockville, Md: Agency for Healthcare Research and Quality; January 2013. Available at: http://effectivehealthcare.ahrq.gov/ehc/products/496/1375/ClosingtheQualityGap_SummaryReport_20130109.pdf. Accessed June 13, 2013.
Cochrane Collaboration. Cochrane Effective Practice and Organisation of Care Group. 2012. Available at: http://epoc.cochrane.org/. Accessed December 9, 2012.
How does context affect interventions to improve patient safety? An assessment of evidence from studies of five patient safety practices and proposals for research.
Shekelle PG, Pronovost PJ, Wachter RM, et al., and the PSP Technical Expert Panel. Assessing the Evidence for Context-Sensitive Effectiveness and Safety of Patient Safety Practices: Developing Criteria (Prepared by RAND Health under Contract No. HHSA-290-2009-10001C). AHRQ Publication No. 11-0006-EF. Rockville, MD: Agency for Healthcare Research and Quality; December 2010. Available at: http://www.ahrq.gov/research/findings/final-reports/contextsensitive/context.pdf. Accessed June 13, 2013.
Contextual factors affecting job satisfaction and organizational commitment in community mental health centers undergoing system changes in the financing of care.
Kanter, RM. When a Thousand Flowers Bloom: Structural, Collective, and Social Conditions for Innovation in Organizations. In Myers, PS, ed: Knowledge Management and Organizational Design. Newton, MA: Butterworth-Heinemann; 1996:93–132.
The views expressed in this report are those of the authors and do not necessarily represent those of the US Department of Health and Human Services, the Agency for Healthcare Research and Quality or the American Board of Pediatrics Foundation.
The author declares that she has no conflict of interest.
Publication of this article was supported by the Agency for Healthcare Research and Quality and the American Board of Pediatrics Foundation.
00105-8&id=gr1.jpg){kind=link}