Purpose: The Every Student Succeeds Act (ESSA) represents a notable shift in K-12 accountability, requiring a more comprehensive approach to assessing school performance and a less prescriptive approach to intervening in low-performing schools. In this article, we seek to leverage the experiences of California's CORE (California Office to Reform Education) waiver districts to better understand what it means to implement an ESSA-like system. Specifically, we examine educators' attitudes about CORE's accountability system, how it was implemented, and its intermediate outcomes. Research Methods: We use a multiple, embedded case study design, examining the implementation of CORE's accountability system across all six CORE Districts. We draw on interviews with CORE staff (n = 4), district leaders (n = 6) and administrators (n = 29), and school principals (n = 15); observations of CORE meetings (42 hours); and documentation. Findings: We find strong buy-in for CORE's accountability system and considerable adaptation of its key elements. District administrators also reported challenges with achieving reciprocity in collaborative activities, and limited capacity, validity concerns, and policy misalignment constrained implementation. Reported effects on practice and learning indicate CORE efforts were a work in progress. Implications for Research and Practice: This research suggests lessons about what it means to be "data-driven" in a multiple-measures accountability era and raises questions about how to facilitate school improvement. While efforts to motivate change via test-based measures, sanctions, and prescribed interventions in prior accountability eras may not have yielded all the expected positive results, our study indicates that a shift to multiple measures, greater flexibility, and locally determined capacity-building efforts brings its own set of challenges.