This presentation will discuss quality appraisal methods for assessing research studies used in systematic reviews, research syntheses, and evidence-based practice repositories such as the What Works Clearinghouse. The different ways that the methodological rigor and risk of bias of primary studies included in syntheses is assessed means that different studies with greater or lesser quality might be included in the recommendations generated from such synthesis. Using the postsecondary education literature as an example, the presenters will describe how different methods of quality appraisal can result in potentially different reports on the extent of evidence on a topic and the level of confidence readers should have about that evidence. Most evidence-based repositories, including the What Works Clearinghouse produce evidence reports for particular interventions. For example, the WWC has produced reports on interventions to teach mathematics (e.g., Pre-K Mathematics, DreamBox Learning) and interventions to prevent high school dropout (e.g., Middle College High Schools). CrimeSolutions.gov has reviewed a number of school-based interventions including career academies. These interventions are rather narrowly defined, and are often (though not always) a single branded program. While organizations might use this positive or negative evidence to evaluate a program they are using or to justify adopting or dropping a program, other important issues can weigh into decision-making such as the level of confidence placed in the evidence; that is, with internal and external validity. An organization might ask, how confident are we that a program is effective for producing the intended outcomes? How likely is it that a new organization will achieve the same results if they implement the same program? This paper will summarize the quality appraisal methods employed by several national research clearinghouses that produce evidence reports relevant to education, highlighting the elements of the quality appraisal and how the quality information is translated into the evidence recommendations that are reported to the public. Tabular data is appended.