The present study explored the effects of the pandemic on individuals with Down Syndrome (DS; n = 67) compared to other groups with Special Education Needs and Disabilities (SEND; n = 48) and their Typically Developing Siblings (TDS; n = 56). In total, 115 caregivers reported on their own anxiety and worries and of their children. Anxiety levels for individuals with DS appeared to be lower compared to other SEND populations and to TDS. In terms of worries, individuals with DS worried more about social-related worries but worried less about family-related aspects compared to the other groups. In sum, individuals with DS might show less anxiety but still worried more about specific aspects related to the impact of COVID-19 pandemic on their lives.