This population-based cohort study examined the relationship between level of continuity of primary care and subsequent emergency department (ED) visits for adults with (n = 66,484) and without intellectual and developmental disabilities (IDD)(n = 2,760,670). Individuals with IDD were more likely than individuals with no IDD to visit the ED (33.96% versus 20.28%, p < 0.0001). For both groups receiving greater continuity of primary care was associated with less ED use, but this relationship was more marked for adults with IDD. While continuity of primary care can reduce ED use for populations with and without IDD, it is a higher priority for individuals with IDD whose cognitive and adaptive impairments may complicate help-seeking, diagnosis, and treatment. Improving primary care can have far-reaching implications for this complex population.