Background: This study investigates whether parents, professionals, and university students have different sexual attitudes towards men versus women with intellectual disability (ID) and which factors are related to such attitudes among these three groups. Method: A self-administered survey using the ASQ-ID questionnaire was implemented, and 130 parents, 173 professionals, and 645 university students completed it. Results: University students showed more positive attitudes than parents and professionals. Parents were more likely to have a positive perception of men with ID than of women with ID with regard to "parenting." Whether they were parents, professionals, or university students was not correlated with their attitudes; instead, their attitudes were associated with participants' age, education, and religion. Conclusions: Awareness of sexual rights related to people with ID, especially women with ID, needs to be discussed among parents and professionals, particularly those in old age, with a low level of education, and those who follow the Buddhist religion.