It is widely acknowledged in research that common criteria and aligned standards do not result in consistent assessment of such a complex performance as the final undergraduate thesis. Assessment is determined by examiners' understanding of rubrics and their views on thesis quality. There is still a gap in the research literature about how analytic and holistic judgments are made and how they are integrated in decisions about the final grade. This interview study aims to identify the sources of inconsistency in analytic and holistic assessment. Ten examiners assessed three final undergraduate theses from a primary school teacher education programme at a Swedish university. The analytic assessment focused on two parts: "theoretical framework" and "academic language" style, which revealed inconsistencies in assessment. The analysis of the holistic assessment showed how examiners weighted different dimensions of the thesis and how they made judgements for the final grade awarded. The findings reveal several sources of inconsistency, such as examiners' own constructs, their interpretations and expectations about students' ability to manage academic work. The study calls for further discussion about whether it is possible to make criterion-based assessment reliable.