Purpose: We explored the reliability and validity of 2 quantitative approaches to document presence and severity of speech properties associated with apraxia of speech (AOS). Method: A motor speech evaluation was administered to 39 individuals with aphasia. Audio-recordings of the evaluation were presented to 3 experienced clinicians to determine AOS diagnosis and to rate severity of 11 speech dimensions. Additionally, research assistants coded 11 operationalized metrics of articulation, fluency, and prosody in the same speech samples and in recordings from 20 neurologically healthy participants. Results: Agreement among the 3 clinicians was limited for both AOS diagnosis and perceptual scaling, but inter-observer reliability for the operationalized metrics was strong. The relationships between most operationalized metrics and mean severity ratings for corresponding perceptual dimensions were moderately strong and statistically significant. Both perceptual scaling and operationalized quantification approaches were sensitive to the presence or absence of AOS. Conclusions: Perceptual scaling and operationalized metrics are promising quantification techniques that can help establish diagnostic transparency for AOS. However, because satisfactory reliability cannot be assumed for scaling techniques, effective training and calibration procedures should be implemented. Operationalized metrics show strong potential for enhancing diagnostic objectivity and sensitivity.