The transformative turn in research and innovation (R&I) policy calls for new approaches to monitoring and evaluation, yet most evaluands are still rooted in previous policy paradigms. For evaluators tasked with conducting ex-post evaluations, this situation creates multiple challenges that have received little attention to date. In this…

The mission of the National Implementation Research Network (NIRN) is to contribute to the best practices and science of implementation, organizational change, and system reinvention to improve outcomes across the spectrum of human services. As such, it has been involved in advancing implementation practice through numerous initiatives over the…

In health technology assessment (HTA), population-adjusted indirect comparisons (PAICs) are increasingly considered to adjust for the difference in the target population between studies. We aim to assess the conduct and reporting of PAICs in recent HTA practice, by performing, a methodological systematic review of studies implementing PAICs from…

While enrolling in a college for a degree is still considered to be more of the expectation than the rule in households across the United States it is facing public criticism regarding its overall cost versus long-term benefits. This has prompted researchers to reevaluate practices in higher education to determine ways in which post-secondary…

Background: Bradford Hill (BH) viewpoints are widely used to assess causality in systematic reviews, but their application has often lacked reproducibility. We describe an approach for assessing causality within systematic reviews ('causal' reviews), illustrating its application to the topic of income inequality and health. Our approach draws on…

With a lack of consensus of what evaluation is within the field of evaluation, there is a difficulty in communicating to nonevaluators what evaluation is and how evaluation differs from research. To understand how evaluation is defined, both evaluators and researchers were asked how they defined evaluation and, if at all, differentiated evaluation…

How research is assessed affects what types of knowledge are valued, incentivized, and rewarded. An increasingly important element of contemporary research evaluation is the measurement of the wider impact of research (e.g. benefit to society, culture or economy). Although the measurement of impact has been highly contested, the area is…

This paper investigates how the emotional responses towards research assessment reflect both social position and strategy in the struggle for scientific authority. This is examined through interviews with humanities researchers conducted as a part of a study on the implications for research practice of the Danish Bibliometric Research Indicator…

For the last 40 years, special education and applied behavior analysis researchers have discussed the notion of social validity, particularly those researchers who conduct single-case research. A substantial part of this discussion has involved critiquing the rigor of methods used to assess social validity. We conducted a scoping literature review…

J.D. Brown (JD) served as the author of an instructional column titled "Statistics Corner" in the "Shiken": JALT "Testing & Evaluation SIG Newsletter" for an extensive period spanning over two decades, from 1997 to 2019. This publication was under the auspices of the Testing and Evaluation Special Interest Group,…

Performance-based research funding systems have been extensively used around the globe to allocate funds across higher education institutes (HEIs), which led to an increased amount of literature examining their use. The UK's Research Excellence Framework (REF) uses a peer-review process to evaluate the research environment, research outputs and…

As institutions of higher education evolve and adapt to meet the increasing needs of their communities, faculty are faced with the choice of where and how to employ their time and expertise. To advance and encourage partnerships between institutions and their greater communities, academic reward structures must be designed in ways that support…

One of the key criticisms of the ethical review process is the time taken to decision, and associated resource use. A key source of delay is that most submissions are required to respond to at least one request for further information or clarification from the Human Research Ethics Committee (HREC). This study audited the request letters of a…

Ethics review processes are frequently perceived as extending from codes and protocols rooted in biomedical disciplines. As a result, many researchers in the humanities and social sciences (HASS) find these processes to be misaligned, if not outrightly obstructive to their research. This leads some scholars to advocate against HASS participation…

Each summer over the past decade, the Chemical Engineering Department at the University of Washington has hosted the "Distinguished Young Scholars Seminar" (DYSS) series, bringing outstanding research trainees from top-tier programs around the country for day-long campus visits filled with student discussions, faculty meetings, mock…