MANY HIGH, LOW ACHIEVING CHARTERS: Charter School Association releases two-year research

By John Fensterwald - Educated Guess | http://bit.ly/h6bi8E

2/24/2011 - California has an abundance of charter schools at polar opposites ­ – those exceeding and those underperforming expectations, according to an extensive performance analysis by the California Charter Schools Association. The most successful charters predominately serve low-income, minority children, and this segment of charter schools is growing, while a sizable number of the lowest achieving charters appear to be smaller, independent study, non-classroom based schools.

With the report, A Portrait of the Movement, the Association fulfills a commitment to  “hold up a mirror” and “embolden schools to look unblinkingly at their record of performance.” A number of previous studies of charter schools have found that on average they don’t outperform district schools. The Association’s two-year research revealed a more complex, “U shaped” picture, with more charters outperforming and underperforming district schools with similar student demographics.

“By lumping all charter schools together, so much of the research did not show the fundamental truths in the movement,” said Association President Jed Wallace. “With far fewer traditional schools exceeding expectations, the question is why can’t we create conditions to bring high-performing charters to scale to make as big an impact as possible?”

Consistent with its position that bad charters should be shut down, the Association created an accountability framework that singled out 30 schools out of 83 in the bottom 5 percent that it says should be reviewed for closure. The Association will not support these schools when their charters come up for renewal, and it plans to introduce legislation that would replace a weak accountability law that the Association says districts often ignore. Using a different rubric, it identified 77 schools, out of 115 in the top 5 percent of high-performing schools, that it said should be rewarded with a longer charter and automatic renewal. They include eight KIPP middle and high schools and 10 Aspire Public Schools in Los Angeles and the Bay Area. Centrally organized, charter management organizations like Aspire comprise 20 percent of California’s charter schools but fully half of the highest performing schools – evidence that philanthropic dollars behind them are bearing results.

Along with the report, the Association has created a web site that includes a map that locates every district and charter school, along with their ratings and API scores, that parents should find useful. An alphabetical list can be found here.

Measuring each school’s impact

The Association evaluated schools using a tool, called a Similar Schools Measure, that predicts every school’s score on the state Academic Performance Index (API) for three consecutive years, based on student demographics. It factors in not only income and race but also the percentage of English learners and special education students; this is important since charter schools have been criticized for educating proportionally fewer of these students. The Association excluded alternative schools that primarily serve dropouts and at-risk students with high mobility, for which API scores are misleading.

In ranking every school by its Percent Predicted API – an estimate of the value that each school adds – the Association found that low-income students in particular are benefiting from charters. Among the results:

• 16 percent of charters (115 of 720 schools) were in the top 5 percent of total public schools based on their predicted API, compared with only 3.9 percent of district schools (293 of  7,454); 22 percent (157 charters) were in the top 10 percent compared with 8.9 percent  of district schools;

• At the other end, 11.5 percent  of charters (83 of 720) were in the bottom 5 percent, compared with only 4.4 percent of district schools (325 of 7,454); 19.2 percent (138 charters) were in the bottom 10 percent, compared with 9.1 percent (679 schools) of district schools;

• Because the lowest-performing schools tended to be smaller, 15.5 percent of charter school students attended the top 5 percent of charter schools, compared with only 6.5 percent of students in the bottom 5 percent of charter schools;

• Low-income students were more likely to attend highly performing charter schools. Of low-income students attending charter schools, 36.8 percent attended the top 5 percent of charters; that’s more than twice the percentage (16.3 percent) of low-income students attending the lowest performing 5 percent of charter schools.

The Association combined  the Similar Schools Measure of predicted API scores with two other factors – a school’s actual API score, on a scale of 200 to 1,000, and the growth in API over three years – to create an accountability framework for identifying the lowest and highest performers. Schools that score less than 700 on the API, show growth of fewer than 30 points over three years, and underperform on the Similar Schools Measure would be scrutinized for possible closure when their charter comes up for renewal. They would have to show achievement using other data. Thirty schools fell under these criteria – about 4 percent of charter schools. They include 11 schools that are non-classroom charters, probably independent study based charters. (The Academy for Recording Arts, a high school in Hawthorne, and the California Virtual Academy at Kern are among these.)

At the opposite end are the 77 high achievers: those with 800 or higher API, a student body that is at least 50 percent proficient on standardized math and English language arts tests, and those that outperform on the Similar Schools Measure. The Association would encourage their replication, with access to capital needed for expansion, and advocate that their charter renewals be expedited.***

Limitations and praise

The Association’s accountability measures improve on the current weak law that many districts ignore, but the Association acknowledges its limitations. API is a flawed measure for high-stakes accountability, because it does not measure individual students’ academic growth. It compares one year’s students  scores with the next year’s. Until CALPADS, the state’s troubled longitudinal data system, is up and running, the state won’t be able to track individual students. The second problem is that the average API scores are higher for elementary schools than for middle and high schools. So more high schools will fall in the watch list for having an under 700 API.

I would add another caution. Once there are arbitrary criteria, like under 700 API for flunking, charters schools may feel more pressure to focus on test scores than their mission. Though the evidence is scarce, critics of charters have alleged for years that some charters push out low-performing students to improve their test scores. At the same time, there are charter high schools that seek out low-performing middle school students who are two or three grades behind academically, then work hard to get them into college. They too deserve credit even if they don’t achieve an 800 API.

For now, charters must live with the state’s accountability system. One expert who praised the Association’s three-prong accountability framework is Doug McRae, a retired test publisher, occasional TOP-Ed contributor, and frequent critic of the criteria that the Department of Education has used to identify lowest performing schools. He wrote me, “The CCSA folks have done a great job cobbling together all three things into one accountability application that just plain makes common sense. It should be the way California uses assessment and accountability data for virtually all intervention applications.”

The Association said that it consulted 20 outside experts during the two years it has taken to create its methodology and assemble the report. Five individuals, including McRae, signed their names as peer reviewers for the Similar Schools Measure. Two of them, Lance Izumi and Vicky Murray, are with the pro-voucher, pro-charter Pacific Research Institute; the others are Eric Crane, senior research associate with the respected research organization WestEd, and Meredith Phillips, associate professor of Public Policy and Sociology at UCLA. Only Crane, Izumi, and McRae were peer reviewers of the full report.

The Association has posted all of the data and variables in the Similar Schools Measure and accountability framework in a technical appendix. Almost every research study of charter schools has provoked debate over methodology; a small change in variables in a formula like the Similar Schools Measure can often yield different results. The research community’s response to “A Portrait of the Movement” bears watching in coming months.

*** Gov. Brown’s budget would increase late payments to schools. See a column in TOP-Ed today about why deferrals especially harm charter schools.