Clinical supervisors are known to assess trainee performance idiosyncratically, causing concern about the validity of their ratings. The literature on this issue relies heavily on retrospective collection of decisions, resulting in the risk of inaccurate information regarding what actually drives raters' perceptions. Capturing in-the-moment…

Learning to take safe and effective action in complex settings rife with uncertainty is essential for patient safety and quality care. Doing so is not easy for trainees, as they often consider certainty to be a necessary precursor for action and subsequently struggle in these settings. Understanding how skillful clinicians work comfortably when…

Whenever multiple observers provide ratings, even of the same performance, inter-rater variation is prevalent. The resulting "idiosyncratic rater variance" is considered to be unusable error of measurement in psychometric models and is a threat to the defensibility of our assessments. Prior studies of inter-rater variation in clinical…