School accountability and improvement policy are on the precipice of a paradigm shift. While the multiple-measure dashboard accountability approach holds great promise for promoting more meaningful learning opportunities for all students, our research indicates that this can come with substantial challenges in practice. We reflect upon the lessons learned from our recent research on the CORE districts' use of multiple-measure data dashboards. Our research indicated that a shift to greater flexibility and locally determined capacity building efforts brings its own set of challenges. Building on this foundation and drawing upon the expertise of a central practitioner-leader, we explore these issues through one promising case: the use of these multiple-measure accountability systems in one CORE district, Oakland Unified School District (OUSD). OUSD's approach suggests that districts have substantial agency to help modify the mindsets of practitioners and the community, by modeling the values of inquiry, openness, and flexibility. By loosening the reigns, districts can give school stakeholders the space and authority to meaningfully manage their own accountability and improvement.