Our Research Questions

NSNO set out to seed 27 turnaround schools and 3 new CMOs. These targets were developed in light of the urgent need to turn around failing schools, an incontrovertible state of crisis in 2010. However, these targets also rested upon assumptions about the quality and quantity of likely aspirants to the CRM. Despite concerted efforts to recruit national operators to both Louisiana and Tennessee, few national CMOs saw real opportunity. High market share already in place in New Orleans made it difficult for outside providers to envision a pathway to building a fully robust network. Limitations on ASD schools in terms of zoned enrollment and availability of turnaround schools disincentivized high quality operators from entering the state. Additionally, the CRM was being implemented at a time when other districts across the country were making real efforts to introduce charters into their portfolios, attributable in part to NSNO’s dissemination of the CRM ahead of proof of concept. Other cities strategically enticed national operators to run large networks of schools, rather than create a competitive RFP process as NSNO and RSD did.

Thus the CRM’s targets for schools and operators presumed that high quality, high capacity operators would be abundant and apparent. These operators did not need to be existing CMOs. In fact, the CRM was designed accommodate four different types of operators: organizations operating their first school while simultaneously building CMO infrastructure in order to scale later; organizations running a single high performing school in order to open a second school and build CMO infrastructure simultaneously; operators with networks of three to five schools looking to scale further; and operators of more than five schools looking to scale further (see table below).

Each type of operator would fall in a different place along an organizational development curve depending on their maturation from single school to robust network, but all selected operators were expected to bring to or develop in their schools a set of capacities necessary to succeed. However, we find that by the end of the study period, only half the CRM schools show positive student impacts, and that the type of operator selected is not statistically associated with student growth.

To understand these findings, consider the CRM operators. By the end of the study period, the i3 funds had supported 4 new CMOs and 25 schools (21 of which are included in this evaluation):

Over the five years of the initiative, investments were made in CMOs whose starting endowments ranged widely, as the NSNO Type classifications indicate above. The selection process was designed to test the CMOs under consideration for certain core capacities: experience serving similar populations to those attending the schools targeted for closure; administrative and staffing capacity to initiate a full school turnaround; and the internal systems and resources necessary to support and guide the turnaround school from low to high performance in a three to five year window. A Type 1 or Type 2 operator could be expected to face a greater lift in order to establish a school model or first-time replication and central CMO infrastructure simultaneously. But all four types were expected to have a demonstrably successful school model on paper or on the ground in order to qualify for selection.

However, we see departures from those foundational expectations immediately in Year 1 and throughout the course of the initiative. The definitional boundaries between new operators and single-school operators were blurred. The selection criteria in Years 1 and 2 expected successful CMO applicants to have certain structural features, a solid and proven school model, and leadership competencies in both the CEO and the school leader. Yet the criteria were repeatedly circumvented. In the first round of selections, none of three schools selected met the full array of criteria (link). In particular, only one of the three Cohort 1 schools entered the field in 2011-12 as a full school turnaround. Additionally, In the first iteration of the selection rubric, CMO leadership was weighted more heavily than any other domain in the selection rubric, and those talents might be adequate to develop a school leader on the job. As a result, applicants with weakly rated school leaders or applicants that named no leader at all received grants in both Year 1 and Year 2.

Additionally, neither NSNO nor RSD executed appropriate philanthropic oversight functions in the first two rounds of selection. NSNO determined grant amounts in full at the commencement of the first year; and held back only a portion of funds, and only in specific instances, in later years. NSNO lacked qualified in-house grants management and financial management staff until the third year of the initiative. NSNO developed a process whereby applicants’ weak or missing capacities were articulated and appended to grant agreements but NSNO built no concurrent mechanism to enforce the organizational development processes contained in these appendices.

Over the course of the study period, we do see a growing recognition by NSNO that school leadership is at least as critical – if not more so – than CMO leadership. Selection rubrics for later selection rounds more heavily weight the evaluation of school leader competency. Similarly, NSNO grew its own grants management and financial oversight capacity beginning in Y3, as its position in the landscape of the CRM shifted to accommodate regranting on a larger scale. NSNO also worked explicitly to transfer this knowledge to Tennessee by providing staff support and expertise to ASD during their selection rounds.

At its core, the CRM Theory of Action requires that turnaround operators possess the necessary capacities, executed with an adequate degree of quality, to turn around failing schools. The selection process was expected to deliver a particular caliber of applicant. But when the application process resulted in too few, and too few strong, applicants in the early years, NSNO, RSD, and ASD downgraded selection criteria rather than withhold funds until future years produced stronger applicants. While we see recognition and remediation of this in later years, the bulk of selection activity occurred in the second year of the CRM initiative, setting up a critical mass of schools engaged in highly visible activities that ultimately only produced positive student impact in half the CRM schools.

These challenges – both existential and administrative - resulted in a global downgrading of the CRM promise before the turnaround schools even opened their doors. Rather than an unwavering commitment to turning lowest-performing schools into highest-performing schools, we see selection of operators who demonstrate the capacity to intervene in failing schools, but not necessarily the capacity to achieve “high performing” status.

For more in-depth exploration of these findings, click here to download the full-length Organizational Capacity report.