Navigating uncertainty with GRACE: Society for Academic Emergency Medicine’s guidelines for reasonable and appropriate care in the emergency department

Abstract

Clinical practice guidelines essentially serve three roles for emergency medicine (EM) physicians: synthesize the evidence for the most important recommendations in a way that can be accessible and actionable for clinicians, engage stakeholders to improve the quality of medical decisions, and clarify medicolegal challenges. The American College of Emergency Physicians (ACEP) was the first organization to produce EM-specific guidelines, publishing their first clinical policy in 1990—a monograph on the topic of “adults with nontraumatic chest pain.”1 Over the next 30 years, both EM and the concept of guidelines expanded and evolved. Contrarians challenge the applicability and real-world implementation of guidelines with an underlying perception of unreported biases among guideline authors.2 Consequently, clinicians often remain unaware of applicable guidelines, which limits uptake into bedside care.3, 4 Although the ACEP Clinical Policy Committee has created and updated dozens of clinical policies over the past three decades, many everyday clinical conundrums remain untouched, particularly topics for which the highest-quality research evidence does not and may never exist. The Society for Academic Emergency Medicine (SAEM) first described their role in facilitating guideline development in 2010.5 Recently, the SAEM Board of Directors approved a plan to create scientifically valid and trustworthy Guidelines for Reasonable and Appropriate Care in the Emergency Department (GRACE). The first GRACE product appears in this issue of Academic Emergency Medicine.6 GRACE emerged to address the critical need for evidence-based and expert-driven, trustworthy, and transparent recommendations for the clinical care of common chief complaints and syndromes, prioritizing those with (1) demonstrable practice variability and (2) significant malpractice angst that (3) often elicit decisional conflict or treatment uncertainty. The ACEP Clinical Policies remain an essential resource and GRACE does not intend to replicate or contradict them. Instead, GRACE will address different questions identified by SAEM members who meet these three criteria even if the evidence basis is suboptimal. To accommodate meaningful recommendations for questions that reside at the intersection of evidence and clinical expertise, GRACE uses the widely adopted Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) framework to facilitate transparency in weighing the strength of evidence balanced against pragmatic constructs of health equity, potential harms, and costs. The GRADE framework emerged in response to the National Academy of Sciences recommended approach to create trustworthy guidelines.7, 8 As depicted in Figure 1, the GRADE approach is thorough, relatively complex, and time-consuming. The first step is constructing a panel of key stakeholders that includes patient perspectives, relevant transdisciplinary viewpoints, content expertise, and GRADE methodologists. Importantly, and consistent with GRADE recommendations, GRACE has included the input of patient advocates. Next, the panel proposes and prioritizes key questions based on the construct of patient/intervention/control/outcome (PICO). Since multiple outcomes are possible and not every outcome is equally important to stakeholders, the panel prioritizes outcomes that may be previously defined in relevant literature or directly informed by patient stakeholders.9 A medical librarian then conducts a literature search (electronic, other) and the panel filters the results based on a priori inclusion and exclusion criteria. Previously published guidelines may be a valuable source of synthesized evidence that can be adapted or adopted by others, if the source guidelines are of high methodologic quality. The adaption or adoption of external guidelines can incorporate the entire guideline or only specific recommendations. The GRADE “ADOLOPMENT” framework provides guidance around when and how to adopt or adapt preexisting recommendations into new guidelines appropriate for other specialty's settings, which can save valuable time and resources while preserving trustworthiness.10 For example, the Ministry of Health in the Kingdom of Saudi Arabia aided by GRADE methodologists from the McMaster University Department of Clinical Epidemiology and Biostatistics created 146 Evidence to Decision Frameworks in 6 months using the GRADE-ADOLOPMENT approach.10 Alternative guideline adaptation besides GRADE-ADOLOPMENT has also been used with each demonstrating some limitations and knowledge gaps.11 If preexisting guideline recommendations are not available or appropriate for adaptation, then de novo evidence reviews may be required. Systematic reviews are essential in the development of evidence-based clinical practice guidelines, because these (ideally) evaluate the totality of the existing literature and include quality appraisals using validated tools. If there are high-quality systematic reviews (or other evidence syntheses) available these are used; otherwise, the panel conducts a systematic review with summary tables to synthesize the evidence.12 To define quality of evidence GRADE favors randomized controlled trials over observational studies yet provides a process to upgrade evidence based on effect size, dose response, or appropriately adjusted confounders.13 GRADE also downgrades evidence based on risk of bias,14 inconsistency,15 indirectness,16 imprecision,17 or publication bias.18 Indirectness is particularly relevant to GRACE since the populations (low-risk recurrent chest pain) are rarely the focus of any emergency department (ED) or non-ED studies and almost never the beneficiaries of randomized controlled trials. Therefore, indirectness empowers GRACE panels to consider the relevance of similar populations in different settings, or different endpoints, or outcomes. The panel then qualifies the strength of recommendations using clear and actionable verbiage before contemplating the Evidence to Decision Framework to balance those recommendations against potential benefits/harms, values/preferences, feasibility, equity, general acceptability, and costs.19, 20 GRACE publishes the Evidence to Decision Framework deliberations as a supplement to optimize transparency. In addition, GRACE provides a health literacy–appropriate patient shared decision-making aid as a supplement to facilitate bedside discussions in real time when applicable. The guidelines are then submitted for peer review and for feedback from stakeholders and scientific societies. Skepticism for both GRACE and GRADE is likely and is probably constructive. Guidelines such as GRACE add to a burgeoning information overload for clinicians with over 3,000 biomedical publications appearing on PubMed each day. The “number needed to read” to identify one manuscript ready for bedside application is 26 for Annals of Emergency Medicine.21 Yet, GRACE will actually ease that information burden by synthesizing a complex body of published research into actionable recommendations formulated to reflect the panel's confidence in that recommendation based on multiple constructs highlighted by GRADE. The value of GRACE rests on the panel's content expertise and the rigor with which the GRADE framework is applied. However, the panel's view will be skewed both by the context in which they practice (the United States with a unique approach to health care delivery, lack of a universal federal health care system, and constrained by an onerous malpractice milieu) and by inherent intellectual biases. Overcoming the limited perspectives and applicability outside of the United States probably requires a separate panel of international emergency physicians reviewing the same body of evidence using the GRADE framework.22 In addition, transparent acknowledgment of intellectual biases requires thorough conflict of interest reporting that transcends traditional financial conflicts. However, perceived or genuine conflicts of interest may be inevitable in recruiting persons with the expertise required for the GRACE panels. Finally, to reduce practice variability and malpractice risk and improve patient safety, multiple opinion leaders identify the creation of pertinent, updated, and feasible guidelines as an essential resource.23, 24 Despite these recommendations, some argue against organizations like ACEP and SAEM creating clinical practice guidelines, particularly when high-quality, indisputable evidence is lacking.25, 26 However, if EM does not create guidelines for our patients and common high-risk scenarios, other specialties will and will then leave EM accountable to other's recommendations.27, 28 It is vital for EM organizations to foster EM-specific guidelines, created by EM clinical/academic experts, to serve ED patient needs. It is unwise and frankly inappropriate for non-EM specialist organizations to create EM guidance recommendations, especially without the critical inputs of EM stakeholders (physicians, nurses, other allied health professionals, patients, and caregivers), without a lived experience of EM practice and care environments. Such recommendations would likely not be focused on EM patient outcomes of importance nor reasonably implementable in ED settings. GRACE will accelerate the creation of new evidence-based recommendations for our specialty,29 creating guidelines that follow the GRADE approach, with involvement and focus on patient important outcomes and consideration of patients’ values and preferences. GRACE will provide a systematic approach for evaluation of the available evidence and an assessment of both the quality of evidence and the strength of recommendations. Above all, we aim to transparently report our decision process because we believe that the focus on transparency facilitates understanding and implementation and should empower patients and clinicians to make informed choices. Although pediatricians find GRADE-based recommendations (compared to other frameworks) most influential in clinical decision making based on their opinions and preference,30 placing the GRADE framework under the microscope is worthwhile.31 No comparative evaluations of GRADE versus other approaches to create guidelines exist in any specialty and the philosophical underpinnings of GRADE may be flawed. For example, GRADE follows the precepts of evidence-based medicine (indeed many GRADE leaders are EBM’s forefathers) in beginning from the assumption that randomized controlled trials provide higher confidence than other study designs, neglecting the reality that many everyday scenarios cannot and will not ever be informed by randomized controlled trials. To accommodate this criticism, GRADE permits panels to upgrade observational studies with large effect sizes, but some effects in medicine are small yet clinically consequential. In addition, GRADE is purposely vague on issues such as the delineation between serious and very serious limitations for downgrading evidence and implies an equal weight for study quality limitations, inconsistency, directness, imprecision, and reporting bias.32 Undoubtedly, panels require training and first-hand experience to apply GRADE in a meaningful and reproducible manner, but the duration and attributes of adequate GRADE training for either methodologists or panel members remains undefined. Despite these uncertainties, GRADE provides a framework utilized by hundreds of organizations and endorsed by Guidelines International Network. If not GRADE, then what framework should GRACE use? EM stakeholders’ evaluation of 20 clinical policies using the Appraisal of Guideline for Research and Evaluation (AGREE II) instrument noted significant problems with applicability and issues with stakeholder involvement and clarity of presentation of current guidelines.33 Furthermore, a similar analysis of the levels of evidence supporting ACEP policy recommendations found that the majority of these were supported by lower classes of evidence and primarily expert opinion based.34 The GRADE process used by GRACE strives to remedy these deficits to maximize the production of trustworthy and useful guideline products for EM practitioners. The onus remains on guideline panels to create recommendations that are transparent and trustworthy to facilitate meaningful shared decision making with patients, based on best available evidence.35 The GRACE team expects challenges ahead, including evaluating the overall value of the GRADE framework, increasing the completeness and transparency of conflicts of interest for panel members,36 incorporating increasing numbers of patient representatives and key stakeholders,37 obtaining feedback from applicable colleagues outside of EM, and including implementation strategies and patient decision aids with each GRACE guideline.38 In addition, sustaining appropriately updated guidelines as research evolves and collaborating with EM physicians in non–United States settings to develop complimentary guidelines for reasonable and appropriate care in resource-limited settings will also be an essential component of the GRACE growth curve. GRACE inevitably exists for and because of SAEM members and so will rely on those members for active engagement by providing constructive feedback and future topic ideas and as content experts. Despite the challenges and inconsistent availability of high-quality evidence, we aim to summarize, synthesize, and present all the available evidence, along with clear clinical practice recommendations, to help EM clinicians and their patients in decision making.

Topics & Concepts