Research designed to establish the empirical underpinnings of psychotherapy relies heavily on arbitrary metrics, and researchers often do not know if clients receiving an evidence-based treatment have improved in everyday life or changed in a way that makes a difference, apart from the changes the arbitrary metrics may have shown. In other words, it is possible that evidence-based treatments with effects demonstrated on arbitrary metrics do not actually help people, that is, reduce their symptoms and improve their functioning. Clarifying the nature of arbitrary metrics and assessing their implications are important initial steps. Needed next steps are detailing the range of strategies to better connect arbitrary measures to real-world referents and developing measures in which arbitrariness is eliminated or minimized from the start.