Our Research Questions
The CRM Theory of Action rests on the belief that CMOs are the most effective agents of change within the CRM system. CMOs are ostensibly more nimble than districts or state agencies. CMOs are purportedly better positioned to understand and respond to school- and student-level need. Additionally, CMOs are perhaps equipped, as mission-driven organizations, to intervene in, appropriately staff, continuously monitor, and effectively implement turnaround strategies in lowest-performing schools. Yet we see variation in both CMO capacity and performance prior to selection. We observe selection of CMOs that do not (or just barely) meet eligibility criteria throughout the study (click here). Thus, the positioning of CMOs as agents of change, innovation, and quality is diminished from the earliest days of the CRM initiative.
The CRM also posits that all CMOs embark on turnaround work with the same degree of operational capacity, both at the CMO and the school level. This may have held true for the CRM’s original formulation – in which all students in a failing school would be served by a turnaround operator – but the inclusion of fresh start schools in the CRM disrupts this assumption. In this evaluation, we find that fresh start schools on average served students who performed lower in reading at entry, but by virtue of having only one grade at opening, they served far fewer students in their early years. Further, fresh start schools do not have to confront culture-building and other operations at scale when they open with only one grade’s worth of staff. As such, full school turnarounds and fresh starts do not, in fact, start in the same place operationally. Full school turnarounds start from a place looking much more like the Closing school, while fresh starts have an opportunity to replicate their Flagship from the very beginning. As such, the Theory of Action plays out differently for different actors, based on their turnaround strategy.
Given this reality, the expectation that CMOs would “become the system” – i.e., cumulatively recreate all functions held by districts, from procurement and facilities management to accountability and continuous improvement practices – seems unrealistic at best. Transferring those functions from a single entity to the 17 CMOs of the CRM – some of which were newly founded entities that had not existed prior to the i3 investment – is both the crux of the Theory of Action and the biggest gamble made by the CRM.
Further, the idea of a completely decentralized system failed to anticipate the very real possibility that externalities would materialize – some schools across the community (CRM and others) took more assertive stands on prioritizing less challenging students. To address this issue, there exist some functions that were eventually recentralized (One App, SPED, differentiated funding formulas, expulsion policy, systemic barriers to access such as transportation) click here.
Interestingly, we see no relationship between CMO history and performance of CRM schools. Having a robust national network and fully operational central office did not guarantee success for CRM schools; nor did first-time school operators necessarily fail. We do observe higher on average PMO scores for those charters with robust national networks, but the single highest PMO score was that of Tubman, the first school of a brand new CMO (Crescent City Schools). Click here for more information about the PMO.
Schools’ relationships with their CMOs are dynamic. School-CMO alignment is high in schools’ inaugural years, with CMOs serving a dual role of support and oversight. As schools age into their second year of operation, we see some disruption to CMO-school relationships as schools seek to establish an independent identity within their networks and within the broader education landscape; and as CMOs expand and must recalibrate roles and responsibilities. But, by the time schools transition to their third year, we see a re-stabilization of CMO-school relationships with a few notable exceptions. These exceptions represent schools that ultimately failed, perhaps suggesting that while growing pains for schools are to be expected, the school-CMO relationship does ultimately support school stability.
All operators receiving i3 funds from NSNO are governed by CMO-level boards. The boards of these CMOs vary in the degree to which they adhere to best practice standards of good governance. Boards demonstrate a range of approaches to stewardship of mission and sustainability. Some take a very hands-off approach, allowing CMO CEOs to drive mission critical work and strategic thinking. Others engage deeply around specific issues, such as fiscal oversight or ambassadorship in the larger community. Nearly all board members report an affinity to their CMO’s mission, but a degree of disconnect from individual schools and a willingness to take cues from their CEOs. Board members also recognize expertise gaps in key areas including academic, legal, financial management, parent/community input, and marketing/PR.
Despite the functional separation between boards and what goes on in the daily life of a school, our analysis revealed that boards matter in the student outcomes that a school produces. However, we find that very little in terms of board composition or activity connects with academic outcomes. Neither board engagement, board training, nor board expertise is significantly correlated with academic outcomes, and where we find significance regarding board activity, that factors are negatively correlated with outcomes.
While three quarters of board members express a commitment to educational equity in their communities, only one has an operational definition of “equity” within their network. However, the CMOs do see themselves as agents of equity in three overlapping ways: they lead by example with a targeted focus on making their own schools as strong as possible; they commit to serving all students, thereby increasing seats and supports for historically underserved or unserved students; and in the later years of the program, they engage in systems-level advocacy around targeted equity functions. As such, CMOs behave as agents of change, innovation, and quality for both their schools and for the system within which they are embedded.
To a lesser extent, CMOs see themselves as agents of sustainability, creating a set of cultural norms, relationships amongst themselves and with the state, and informal practices that will sustain beyond the current CEOs’ tenure. But over the course of the evaluation, CMOs had not fully formalized these functions, nor have ASD/RSD or NSNO ceded these functions to the CMOs.
Ultimately, school turnaround is a larger enterprise than simply fixing education functions. Rather, turnaround requires sustained intervention across multiple domains: administrative, leadership, mission/vision, change management, human capital. To the extent that CMOs own some but not all of these functions, and to the extent that these functions are haphazardly supported, we conclude that CMOs operate at their own systems’ level but they have not become “the system”.