Several validated adult autism symptom screening tools exist; however, there are concerns about the validity of instruments in adults who self-identify and those who have an autism diagnosis. This study evaluated the psychometric properties of the RAADS-R and RAADS-14 across gender, autism diagnosis and autistic identity, and age among a sample of 839 adults. Participants included individuals who reported a prior diagnosis of autism, self-identified as autistic without a diagnosis, were exploring a diagnosis of autism, or neurotypical. Psychometric evaluation included confirmation of scale factor structure, and examination of item performance, including bias and discrimination, across the target sociodemographic characteristics. The RAADS-R and RAADS-14 were found to be psychometrically sound, unidimensional instruments in the present sample without systematic bias by age, diagnosis/identity, or gender. The use of dichotomous response options would improve measure parsimony. Close examination of extant item-level bias is also recommended in the development of new measures. Notably, few psychometric differences between diagnosed and self-identifying individuals were identified. There was a distinction, however, between diagnosed individuals and those exploring autistic identification. Four items representing the consequences of adverse sensory experiences and the hidden rulebook of social interaction were most informative for differentiating autistic individuals from neurotypical.