Impact Methods: Explore what we did
Ultimately, the CRM aimed to improve student academic outcomes. CREDO's Student Impact Study examines the academic growth of students attending CRM schools. We also examine the ways in which observed phenomena from the Implementation Study may be related to average student growth and achievement. We present results in aggregate for all CRM schools, and in aggregate by each state. While CREDO's evaluation does not report student effects for individual CRM schools, we do provide profiles of each school that provide information on their overall performance, as well as particular successes and challenges encountered during the course of CRM implementation.
Our analysis uses the Virtual Control Record (VCR) methodology that has been used in previous CREDO publications. The approach is a quasi-experimental study design with matched student records followed over time. Using this methodology, we examine whether students in newly-opened CRM schools outperform their traditional public schools (TPS) counterparts.
Our analysis controls for prior academic achievement, race/ethnicity, special education status, socio-economic status (as measured by eligibility for subsidized lunches), English proficiency, grade level, and retention in grade, in order to prevent the tainting of the estimate of CRM schooling by those effects. To learn more about the VCR method, visit http://credo.stanford.edu/virtual-control-records/.
We compare the achievement of students in the Closing schools in New Orleans and Memphis/Nashville to the achievement of all students in Louisiana and Tennessee, respectively. Compared to the respective state, the average achievement level in Reading of students in the Closing schools is at the 21st percentile and the average achievement level in Math of students in the Closing schools is at the 22nd percentile.
Our results show that students in the last year of the Closing schools have significantly weaker academic growth in both reading and math when compared to the average student in a non-closing traditional public school (TPS). Closing school students exhibit 63 fewer days of learning in reading and 86 fewer days of learning in math.
We compare the achievement of students in CRM schools in New Orleans and Memphis/Nashville to the achievement of all students in Louisiana and Tennessee, respectively. Compared to the respective State, the average achievement level in Reading of students in the CRM schools is at the 29th percentile and the average achievement level in Math of students in the CRM schools is at the 33rd percentile. The overall achievement of students in the CRM schools is higher than that of student in the Closing schools in both Reading and Math.
CRM students exhibit 6 additional days of learning in reading and 11 fewer days of learning in math compared to the average student in a TPS school, but those estimated learning gaps are not statistically significant. Albeit small and statistically insignificant, the overall effect size of CRM on student performance is much higher than that of the closing schools. In particular, our findings show that overall, each student in a CRM school experiences an average test score growth in relation to students in a traditional public school ten and seven times bigger than that of a student in the last year of a closing school in terms of Reading and Math, respectively.
| Target | Top 33 percent of schools in Louisiana in 2015-2016 | Top 33 percent of schools in New Orleans in 2015-2016 | Top 25 percent of schools in Tennessee in 2014-2015 | Top 25 percent of schools in Memphis in 2014-2015 | Top 25 percent of schools in Nashville in 2014-2015 |
| Math | 0 | 2 | 0 | 0 | 0 |
| Reading | 0 | 3 | 0 | 0 | 0 |
The Performance Management Organization (PMO) score is a composite index of implementation quality. The PMO index is positively associated with average school growth in reading and math. An increase of one increment on the PMO scale is associated with 291 additional days of learning in Reading and 29 additional days of learning in Math for students in the CRM schools.
The CRM permitted two types of school transformation: fresh starts and full school turnarounds. Students in fresh start CRM schools have stronger academic growth in math and reading when compared to the average student in a TPS school, although only reading growth is significant. These gains translate to 51 additional days of learning in reading and 51 additional days of learning in math for students in fresh start CRM schools.
Students in CRM full school turnarounds have weaker academic growth in both math and reading when compared to the average student in a TPS school. CRM students in full turnarounds exhibit 29 fewer days of learning in reading and 6 fewer days of learning in math. These growth effects are significantly weaker than those of the fresh start CRM schools.
To Dig Deeper into the Aggregate Student Impact Data by State, Cohort, Year of Operation or Demographic, select from the Drop-down menu.