Evaluation of Scaling the New Orleans Charter Restart Model

Student Impact - Louisiana

image description Impact Methods: Explore what we did

Ultimately, the CRM aimed to improve student academic outcomes. CREDO's Student Impact Study examines the academic growth of students attending CRM schools. We also examine the ways in which observed phenomena from the Implementation Study may be related to average student growth and achievement. We present results in aggregate for all CRM schools, and in aggregate by each state. While CREDO's evaluation does not report student effects for individual CRM schools, we do provide profiles of each school that provide information on their overall performance, as well as particular successes and challenges encountered during the course of CRM implementation.

Our analysis uses the Virtual Control Record (VCR) methodology that has been used in previous CREDO publications. The approach is a quasi-experimental study design with matched student records followed over time. Using this methodology, we examine whether students in newly-opened CRM schools outperform their traditional public schools (TPS) counterparts. Our analysis controls for prior academic achievement, race/ethnicity, special education status, socioeconomic status (as measured by eligibility for subsidized lunches), English proficiency, grade level, and retention in grade, in order to prevent the tainting of the estimate of CRM schooling by those effects. To learn more about the VCR method, visit http://credo.stanford.edu/virtual-control-records/.

Performance Targets
Achievement (in Percentiles)
Reading
66
Math
66
Growth (Reading and Math)
Reading
57
Math
57
The Charter Restart Model set high performance goals for the CRM schools.CRM schools in New Orleans were expected to perform at the top 33% of schools in the city. This achievement goal translates to an annual school growth effect in each subject (reading and mathematics) equivalent to 57 additional days of learning. This effect size serves as our point of reference for the evaluation of the CRM schools.
Closing School Performance
Achievement
Reading
26
Math
26

We compare the achievement of students in the Closing schools in New Orleans to the achievement of all students in Louisiana. Compared to state averages, the average achievement level in reading of students in the Closing schools is at the 26th percentile. The average achievement level in math of students in the Closing schools is also at the 26th percentile.

Growth
Reading
-97
Math
-148

Our results show that students in the last year of the Closing schools have significantly weaker academic growth in both reading and math when compared to the average student in a non-closing traditional public school (TPS). Closing school students exhibit 97 fewer days of learning in reading and 148 fewer days of learning in math.

CRM Schools Overall Performance
Achievement
Reading
32
Math
35

We compare the achievement of students in CRM schools in New Orleans to the achievement of all students in New Orleans, and to all students in Louisiana. The average achievement level in reading for CRM students is at the 32nd percentile and the average achievement level in math for CRM students is at the 35th percentile. The overall achievement of students in the CRM schools is higher than that of students in the Closing schools in both reading and math.

Growth
Reading
0
Math
-6

CRM students exhibit 6 fewer days of learning in math compared to the average student in a TPS school, although estimated learning gaps are not statistically significant. At the same time, CRM students experience comparable progress in reading compared to their TPS peers. This means that the academic growth of CRM students in reading is on par with their TPS peers. Albeit small and statistically insignificant, the overall effect size of the CRM on student performance is much higher than that of the Closing schools. In particular, our findings show that overall, each student in a CRM school experiences an average test score growth that is 17 times bigger in reading and 25 times bigger in math than that of a student in the last year of a Closing school, as compared to TPS peers.

How many schools reached the CRM's proposed performance targets?
TargetTop 33 percent of schools in Louisiana in 2015-2016Top 33 percent of schools in New Orleans in 2015-2016
Reading02
Math03
Performance Management Organization Rubric Scores
Reading
296
Math
34

The Performance Management Organization (PMO) score is a composite index of implementation quality. The PMO index is positively associated with average school growth in reading and math. An one increment on the PMO scale is associated with 296 additional days of learning in reading and 34 additional days of learning in math for students in the New Orleans CRM schools.

Fresh Start Schools vs. Full School Turnarounds: Two Approaches to Turnaround
Fresh Start Schools Growth
Reading
120
Math
80

The CRM permitted two types of school transformation: fresh starts and full school turnarounds. Students in fresh start CRM schools have stronger academic growth in math and reading when compared to the average student in a TPS school, although only reading growth is significant. These gains translate to 120 additional days of learning in reading and 80 additional days of learning in math for students in New Orleans fresh start CRM schools.

Full School Turnaround Growth
Reading
-23
Math
-11

Students in CRM full turnarounds have weaker academic growth in both math and reading when compared to the average student in a TPS school. CRM students in full turnarounds exhibit 23 fewer days of learning in reading and 11 fewer days of learning in math. These growth effects are significantly weaker than those of the fresh start CRM schools.

Go back to the Student Impact
Landing page or explore the
data by individual states:
New Orleans and Tennessee