Clinical practice guidelines can influence a large number of healthcare providers and patients [8] so their quality is critically important. The quality of practice guidelines refers to the confidence that the potential biases of guidelines development have been addressed adequately, recommendations are both internally and externally valid, and the guidelines are feasible for practice [11]. High-quality guidelines are valid (unbiased), reproducible, clinically applicable to the populations of interest, flexible, clearly presented, developed through a multidisciplinary process, reviewed on a regular schedule, and well documented [1].

The quality of practice guidelines depends on the quality and rigor of each of the steps performed in formulating the guidelines: crafting of the key clinical questions to be addressed in the guidelines, identification of studies for inclusion, assessment of the quality of individual studies, synthesis of the body of evidence and assessment of its quality, and the formulation of recommendations or practice guidelines. The process of generating and assessing high-quality systematic reviews is addressed in Chapter 3 and discussion of the linkage between the body of evidence and recommendations is presented in Chapter 4.

There are a number of published instruments available to assess the quality of practice guidelines. One of the most widely used instruments is the Appraisal of Guidelines Research and Evaluation (AGREE) Instrument [11] which provides a framework for assessing the quality of clinical practice guidelines, taking into account the benefits, harms, and costs of the guideline recommendations, as well as issues of the feasibility of implementation. This instrument is largely based on theoretical assumptions as empirical data are lacking [11]. It was, however, developed by a multinational group, is widely used by organizations that use the products of other guideline groups (including the World Health Organization), and has been validated [12,13]. The AGREE Instrument encompasses six domains: scope and purpose, stakeholder involvement, rigor of development, clarity and presentation, applicability, and editorial independence, and includes 23 questions in total. An overall assessment is then made as to whether the appraiser would recommend these guidelines for use in practice.

Although AGREE is a very useful instrument for assessing practice guideline quality, it may not encompass all aspects of quality that are relevant to guideline developers. Nuckols and colleagues [14] found that even guidelines that were rigorously developed, as measured with the AGREE Instrument, frequently omitted common clinical situations and contained content rated by experts to be of “uncertain validity.” Another guideline team also noted that high-scoring guidelines with the AGREE instrument may still be problematic with respect to applicability of the available studies to general populations of interest [15].

The quality of clinical practice guidelines has not been extensively studied, but the available literature suggests that guidelines are frequently suboptimal because of flaws in their development, inadequacies in the evidence underlying the guidelines or both [15–20].

Clinical practice guidelines and the recommendations contained therein are distinct from, but should usually be based on, systematic reviews of the clinical questions addressed in the guidelines. High-quality systematic revi-ews provide an exhaustive, unbiased synthesis of the available evidence (see Chapter 3). Guidelines present recommendations about which approaches should or should not be taken in specific clinical circumstances based on the evidence identified in the systematic review. There are thus two separate junctures where guideline developers must provide a transparent, reliable, and valid assessment: quality or grade of the body of evidence identified in the systematic review process which reflects the level of confidence that if a recommendation is followed, the anticipated outcome will occur; and second, assessment of the strength of recommendations, which takes into account the balance of benefits and harms and the importance of adhering to the recommendations [21].

The criteria for an optimal system for assessing the quality of a body of evidence and linking that evidence to practice guidelines or recommendations include [21]:

• the separation of grades of recommendation from the quality of the evidence
  
• simplicity and transparency
  
• a sufficient number of (but not too many) categories for quality of the body of evidence
  
• an explicit methodology
  
• levels of evidence that can vary across different outcomes.

In the past, the strength of evidence was combined with strength of recommendation in the formulation of recommendations in practice guidelines, and the strength of evidence was based only on a hierarchy of study design. The science has moved away from this overly simplistic model, as different types of study designs are optimal for different questions; hierarchies and definitions of levels of evidence vary across guideline groups; and this approach can lead to anomalous conclusions [22].

A large number of different approaches has been used to link a body of evidence to clinical practice recommendations; in fact, a 2002 review identified more than 50 systems for grading a body of evidence [23] and many more have been developed since. These include the Grading of Recommendations, Assessment, Development, and Evaluation (GRADE) approach [24], discussed in detail in Chapter 4. GRADE provides a framework for assessing the quality of a body of evidence (based on the underlying systematic reviews) and for the formulation of recommendations [24].

The process of generating evidence-based recommendations from a high-quality systematic review is a complex one, as even when there is consensus on the body of evidence, clinical experts formulating recommendations may disagree on the specific recommendations for clinical practice. Recommendations depend on an assessment of the balance of benefits and harms, which are often presented in different metrics that are difficult to compare. Value systems are important in weighing benefits and harms, and these vary across healthcare providers and patients, and physicians’ perceptions may not be an accurate reflection of patient values and priorities. In addition, decision making by expert panel members may depend on their unique training and experiences, as well as on resource constraints and existing infrastructures in their local setting.

A number of urological, obstetric and gynecological, internal medicine, radiologic, pediatric, and governmental organizations have published practice guidelines on urological topics. These include topics related to diagnosis, prevention, screening or treatment. The NGCH provides a compendium of current guidelines which can be readily searched by topic, diagnosis, organization producing the guideline, and intended user of the guideline. Of a total of 2452 guidelines, 170 are categorized as specifically related to urology, with additional guidelines relevant to urologists found in obstetrics and gynecology, pediatric, and other categories. Organizations producing urology-related guidelines include the European Association of Urologists, the American Urological Association, National Institute for Health and Clinical Excellence, the Scottish Intercollegiate Guidelines Network, American College of Obstetricians and Gynecologists, and many other organizations.

The purpose of the NGCH is to give healthcare providers, health plans, and health care delivery systems access to objective, detailed information on practice guidelines in order to further their dissemination, implementation, and use [25]. Started in 1998, the NGCH is an initiative of the Agency for Healthcare Research and Quality (AHRQ), is readily searchable, and contains structured abstracts (summaries) of each guideline, links to the full text of the guideline, when available, and the means to readily compare and contrast guidelines on related topics.

A clinical practice guideline must meet all the following criteria to be included in the NGCH [26].

• The guideline contains systematically developed statements that include recommendations, strategies or other information that assists healthcare practitioners and patients in specific clinical circumstances.
  
• The guideline was produced under the auspices of a relevant professional organization or association or by a public or private organization.
  
• Collaborating documentation can be produced and verified that a systematic literature search and review of existing scientific evidence published in peer-reviewed journals was performed during guideline development.
  
• The guideline is in English.
  
• The guideline is current (less than 5 years since development or revisions).

Several critical evaluations of practice guidelines in urology have been published recently. Aus and colleagues [27] evaluated the methods used to produce the European Association of Urology (EAU) guidelines using the AGREE instrument. These authors concluded that the EAU guidelines scored “acceptably well” but noted several weaknesses: lack of broad stakeholder involvement (including patients); no discussion of potential organizational barriers or cost implications; the guidelines were “evidence-based medicine light” with the retrieval of only recent randomized controlled trials and meta-analyses; and no external peer review.