Background: We sought to assess diet quality among people with intellectual disabilities or borderline intellectual functioning, living in residential facilities or receiving day care. Methods: We measured diet quality using the Dutch Healthy Diet Food Frequency Questionnaire (DHD) and compared this between participants with (n = 151) and controls without intellectual disabilities (n = 169). Potential correlates of diet quality were explored. Results: We found lower mean diet quality among people with intellectual disabilities (M = 80.9) compared to controls (M = 111.2; mean adjusted difference -28.4; 95% CI [-32.3, -24.5]; p < 0.001). Participants with borderline intellectual functioning and mild intellectual disabilities had lower diet quality and higher body mass index than individuals with severe to profound intellectual disabilities. Being female was a predictor of better diet quality. Conclusions: Overall, we found that diet quality was low in the sample of people with intellectual disabilities or borderline intellectual functioning.