A central tenet of evidence-based practice is that a hierarchy of evidence exists. Among individual studies, the randomized controlled trial (RCT) provides the highest level of evidence, although ideally a meta-analysis of several RCTs will provide better estimates of treatment effects than a single RCT. Below the RCT in the hierarchy of evidence come cohort studies, which follow groups of patients through time. The key difference between cohort studies and RCTs is that patients are not randomly allocated to treatments in a cohort study. Cohort studies may be prospective – that is, patients are allocated into cohorts prior to the occurrence of the primary outcomes. Alternatively, cohort studies may be retrospective, when the primary outcome has already occurred. RCTs comprise approximately 10% of the urology literature, while cohort studies comprise approximately 45% [4]. Below cohort studies are case–control studies, case series or reports, and finally expert opinion.

The hierarchy of evidence exists because individual study designs are inherently prone to bias, that is, systematic deviation from the truth. As opposed to random error, bias has a magnitude and a specific direction. Bias may serve to over- or underestimate treatment effects, and therefore lead to erroneous conclusions about the value of therapeutic interventions. RCTs sit at the top of the hierarchy of evidence because well-designed and executed RCTs contain the strongest methodological safeguards against bias. Study designs further down the hierarchy of evidence are subject to increasing potential for bias, and therefore constitute lower levels of evidence.

Randomized controlled trials are unique in the hierarchy of evidence, as participants in the trial are not selected for specific interventions but instead are allocated randomly to a specific therapy or control. With appropriate methodological safeguards, RCTs have the potential to provide the highest level of evidence for questions of therapy. For this reason, informed consumers of the urologic literature should understand how to appropriately interpret the results of a clinical trial [5]. RCTs form only a small proportion of published studies in the urologic literature, likely due to several barriers to the conducting of surgical RCTs [4], including the lack of equipoise among surgeons and patients regarding interventions and lack of expertise among urologists with respect to clinical research methodology [6]. In addition, new techniques inherently involve a learning curve; technical proficiency is a requisite for unbiased conduct of a RCT.

One potential method for overcoming technical proficiency barriers is the expertise-based RCT [7] in which the patient is randomized to an intervention conducted by an expert in the technique. For example, in a hypothetical trial of robot-assisted laparoscopic prostatectomy versus open retropubic prostatectomy, the surgery in each arm would be performed by an expert in that specific procedure, recognizing that it is difficult to separate the surgeon from the scalpel when evaluating surgical interventions. In addition, RCTs are not always feasible or ethical [8] and for these reasons, physicians must also incorporate results from observational studies (i.e. prospective cohort), while maintaining awareness of the increased potential for bias in observational designs.

Observational designs, such as cohort and case–control studies, have certain advantages over RCTs although at the significant limitation of greatly increased risk of bias. However, for certain questions, such as those of harm, observational designs may provide the only feasible means to examine rare adverse outcomes of an intervention. Cohort studies for harm may be most useful when randomization of the exposure is not possible, and may be more generalizable than RCTs [9]. Case–control studies can overcome long delays between exposure and outcome, as well as the need to accrue enormous sample sizes to identify rare events [9]. In addition, for questions of prognosis, prospective cohort designs comprise the highest level of evidence.

Several elements of clinical trial design safeguard against the introduction of bias into the results of the treatment under evaluation. Overall, the objective of these design elements is to ensure that (on average) patients begin the trial with a similar prognosis and retain a similar prognosis (outside therapeutic effect) once the trial begins. The Consolidated Standards of Reporting Trials (CONSORT) statement provides a comprehensive list of reporting guidelines for clinical trials [10,11]. Included in the CONSORT statement are several design elements which provide important safeguards against bias in trial results. These include randomization, concealment, blinding, equal treatment of groups, and complete follow-up.

Randomization refers to the method by which patients are allocated to treatment arms within the trial. As the name implies, patients should be placed into treatment arms in a random, that is, not predictable, fashion. The purpose of randomization is to balance both known and unknown prognostic factors between treatment arms [3]. For example, in a trial of active surveillance versus radical prostatectomy for prostate cancer, it would be important for Gleason grade (among other factors) to be balanced between the active surveillance and radical prostatectomy group. It would also be important to balance potentially unknown prognostic factors, such as co-morbid conditions, in such a trial, and randomization optimizes the balance of these conditions. It is important to realize that randomization is not always successful in balancing prognostic factors, particularly with smaller sample sizes. Thus, when interpreting the results of any trial, the reader should examine the balance of patient characteristics (often presented in the first table in the manuscript) between groups.

Equally important to maintaining an initial balance of prognostic factors is the concept of concealment. Concealment refers to the principle that study personnel should not be able to predict or control the assignment of the next patient to be enrolled in a trial [3]. This concept is important because it prevents selection, either conscious or unconscious, of subjects for specific treatment arms. Remote randomization, where investigators call to a centralized center to ascertain the assignment of a study subject, is a method frequently used to ensure concealment of randomization. Other methods, such as placing study arm assignments into sealed envelopes, may not always ensure concealment. For example, in a study of open versus laparoscopic appendectomy, concealment by sealed envelope was compromised when surgery occurred overnight, potentially introducing bias to the trials results [3,12]. Lack of concealment has empirically been associated with bias in study results [13–15]. Therefore, it is very important for the informed consumer of medical literature to be aware of whether randomization in a clinical trial was concealed, in order to ensure balance of prognostic factors in the study.

Once a trial is under way, the balance of prognostic factors remains important. Other design features of RCTs assist in maintaining the balance of prognostic factors through the progress and completion of the study. During the study, it is critically important, to the extent feasible, that several groups remain blinded