Challenging behavior (CB) is a common occurrence in early childhood and frequently occurs in young children with disabilities. CB is also culturally perceived and includes differences in how caregivers understand and define the topography of CB. Despite the cultural interpretation, CB is known to impact the child and their family but there has been little exploration of what marginalized caregivers perceive as the impact of the CB that their young children with disabilities exhibit. We used semi-structured interviews to explore the perceptions of 24 caregivers, who identified as Black, Mexican American, and White, about the impact of their child's CB. Caregivers shared how the CB impacted themselves, their families, their child, and others. Although much of the impact was similar among caregivers, we discuss nuanced differences across ethnic groups.