In this article, we focused on the nature of the relationship between research evaluation systems that are based on metrics and those that are based on a critical comprehensive judgment of scientific content with researchers' burnout and work engagement. The research was conducted on a sample of 1,191 Polish researchers in the context of a highly metricized research evaluation system. Our contribution to the field of research evaluation is fourfold. First, we provide empirical support for the recent call for more nuanced views on publication metrics in research evaluation, showing that evaluations lacking in the judgment of scientific content are associated with higher demands and lower job resources. Second, our study focuses the attention of various policymakers on the fact that performance measurement should not be confused with performance management and that thoughtless use of performance publication metrics might lead to research performance deterioration instead of improvement. Third, our findings illustrate the tensions between different views on publication metrics as means of research evaluation; this tension might be observed among researchers and their supervisors but also within the research community as a whole. Fourth, our article may inspire further research of evaluation systems, as we design measures of demands and resources generated by the evaluation system based on Job Demands-Resources theory. Moreover, based on the Leiden Manifesto and Royal Society Resume for researchers, we have validated an evaluation based on a judgment scale that might be used in further research and practical settings to audit research evaluation systems.