Impact Methods: Explore what we did
Ultimately, the CRM aimed to improve student academic outcomes. CREDO's Student Impact Study examines the academic growth of students attending CRM schools. We also examine the ways in which observed phenomena from the Implementation Study may be related to average student growth and achievement. We present results in aggregate for all CRM schools, and in aggregate by each state. While CREDO's evaluation does not report student effects for individual CRM schools, we do provide profiles of each school that provide information on their overall performance, as well as particular successes and challenges encountered during the course of CRM implementation.
Our analysis uses the Virtual Control Record (VCR) methodology that has been used in previous CREDO publications. The approach is a quasi-experimental study design with matched student records followed over time. Using this methodology, we examine whether students in newly-opened CRM schools outperform their traditional public schools (TPS) counterparts. Our analysis controls for prior academic achievement, race/ethnicity, special education status, socioeconomic status (as measured by eligibility for subsidized lunches), English proficiency, grade level, and retention in grade, in order to prevent the tainting of the estimate of CRM schooling by those effects. To learn more about the VCR method, visit http://credo.stanford.edu/virtual-control-records/.
We compare the achievement of students in the Closing schools in Tennessee to the achievement of all students in Tennessee. Compared to state averages, the average achievement level in reading of students in the Closing schools is at the 16th percentile. The average achievement level in math of students in the Closing schools is at the 18th percentile.
Our results show that students in the last year of the Closing schools have significantly weaker academic growth in both reading and math when compared to the average student in a non-closing traditional public school (TPS). Closing school students exhibit 40 fewer days of learning in reading and 46 fewer days of learning in math.
We compare the achievement of students in CRM schools in Memphis/Nashville to the achievement of all students in Tennessee. Compared to state averages, the average achievement level in reading of CRM students is at the 21st percentile and the average achievement level in math of CRM students is at the 25th percentile. The overall achievement of students in the CRM schools is higher than that of student in the Closing schools in both reading and math.
CRM students exhibit 23 fewer days of learning in math compared to the average student in a TPS school, although estimated learning gaps are not statistically significant. At the same time, CRM students experience comparable progress in reading compared to their TPS peers. This means that the academic growth of CRM students in reading is on par with their TPS peers. Albeit small and statistically insignificant, the overall effect size of CRM on student performance is much higher than that of the Closing schools. In particular, our findings show that overall, each student in a CRM school experiences an average test score growth seven times bigger in reading and two times bigger in math than that of a student in the last year of a closing school, as compared to TPS peers.
| Target | Top 25 percent of schools in Tennessee in 2014-2015 | Top 25 percent of schools in Memphis in 2014-2015 | Top 25 percent of schools in Nashville in 2014-2015 |
| Reading | 0 | 0 | 0 |
| Math | 0 | 0 | 0 |
The Performance Management Organization (PMO) score is a composite index of implementation quality. The PMO index is positively associated with average school growth in reading and math. An increase of one increment on the PMO scale is associated with 308 additional days of learning in reading and 51 additional days of learning in math for students in the Tennessee CRM schools.
The CRM permitted two types of school transformation: fresh starts and full school turnarounds. Students in fresh start CRM schools have stronger academic growth in math and reading when compared to the average student in a TPS school, although only reading growth is significant. These gains translate to 11 additional days of learning in reading and 29 additional days of learning in math for students in fresh start CRM schools.
Students in CRM full school turnarounds have weaker academic growth in both math and reading when compared to the average student in a TPS school, although only reading is statistically significant. CRM students in full turnarounds exhibit 6 fewer days of learning in reading and 68 fewer days of learning in math, although only the effect in math is statistically significant. These growth effects are significantly weaker than those of the fresh start CRM schools.
Go back to the Student Impact
Landing page or explore the
data by individual states:
New Orleans and Tennessee