At some elementary schools, students improve their test scores year after year. At others, they stagnate or fall behind. Data that go unused could offer insights into why.
Neena Trummell, 4, clings to her father, Mike, as he tours Wilbur Avenue Elementary in Tarzana. A Times analysis shows students there started out scoring high on state tests, but then lost ground. (Al Seib, Los Angeles Times / May 6, 2010)
By Jason Song, Jason Felch and Doug Smith, Los Angeles Times
Sunday, August 22, 2010 -- Five families from across the San Fernando Valley set up camp for three nights by the front door of Wilbur Avenue Elementary School in 2009, intent on getting a spot for their children in one of the best-regarded schools in Los Angeles. Others hired someone to hold their place in line.
This spring, the school in affluent Tarzana began using a lottery for applicants from outside the neighborhood. Within hours, more than a dozen children were on the list.
What these determined families could not have known is that Wilbur's record was among the worst in Los Angeles for boosting student performance in math and English.
On average, the children started out as high achievers but year after year lost ground on the state's standardized tests, according to a Times analysis of scores from the 2002-03 through 2008-09 school years. Nearly 90% of schools in the Los Angeles Unified School District saw more academic progress.
Several other respected elementary schools, including Topeka Drive in Northridge and Third Street in Hancock Park, also had poor results.
If you have taught third through fifth grades in the Los Angeles Unified School District, you may be in our database, and we invite you to comment on your value-added score. To get started, please type your first and last names in the box below.
At the same time, some of the biggest gains came on campuses in low-income areas, schools often considered failing by state and federal standards.
The school whose students improved most? Maywood Elementary southeast of downtown Los Angeles, where virtually every student qualifies for free or reduced-price lunches and almost half are still learning English.
Parents — and even principals — don't know this because the district doesn't measure progress in this way, although it could.
Schools such as Wilbur shine under the current measure of academic success — the all-important Academic Performance Index — based on students' achievement level on standardized tests. But, as state data show, such measures largely reflect students' advantages outside school, not what the school itself is contributing to their learning.
» Discuss
Do you think the value-added approach should be used to evaluate teachers? Why or why not?
The API obscures the fact that students at Wilbur had the potential for further growth that went unrealized. Instead, they tended to slip every year while those at other esteemed schools in well-off neighborhoods made great strides. It also obscures the gains in schools in impoverished areas.
A similar story is playing out across the country.
"We're measuring who is in schools rather than how effective the schools are," said Helen Ladd, a professor and testing expert at Duke University.
The Times used an approach — known as "value-added" — that, while also based on standardized tests, looks at how much students improve year to year.
The analysis was based on test scores in Grade 2 through 5 at 450 of Los Angeles' approximately 500 elementary schools. It substantially changes the picture of which schools are succeeding and which are not.
The approach generally doesn't penalize schools for things beyond their control — students' poverty, English-language ability, previous achievement or other factors commonly used to explain schools' success or failure. That's because each student's progress is measured against his or her own past performance, not that of other children.
Value-added has many critics who consider it unreliable and a narrow gauge of performance. It looks, in this instance, only at math and English scores, and it ignores many other factors that parents consider when choosing a school. Most of the controversy over value-added, however, has centered on whether it should be used to assess individual teachers, not schools.
Last Sunday, The Times published findings from a value-added analysis of more than 6,000 teachers in L.A. Unified, which noted that it matters much more which teacher a child gets than which school he or she attends. But parents don't usually pick a school for a single teacher; this analysis points to schools where teachers overall tend to be more successful at raising scores year after year.
Troubled by the exclusive focus on achievement under the federal No Child Left Behind law, the Obama administration has made analysis of student progress a priority for both teachers and schools. Several states, including California, are moving in that direction.
"I'm much less interested in absolute test scores and more interested in how kids are improving," U.S. Education Secretary Arne Duncan told The Times last week.
The results of such a shift are sure to be surprising.
"It's really shocking. I had no idea," said Nicole Miller, one of the Wilbur campers, upon hearing how the school fared in the Times analysis. "I would have definitely taken a really good look at other schools had I known those numbers."
The API
The Academic Performance Index holds great sway in California education.
Principals tout high API scores and scramble to explain low ones. Real estate agents know to keep the numbers handy for house-hunting parents.
In elementary and middle school, the 1,000-point index is based entirely on how high students score on the state's annual tests, given in Grades 2 through 12. According to state data, 81% of the differences among schools reflect socioeconomic factors such as poverty and parents' education.
The benefit of the API is that it reflects state standards, helping to maintain clear and common goals for all schools. California third-graders, for instance, are expected to be able to add and subtract simple fractions.
But even those who designed the API more than a decade ago say it was never meant to be used alone. They recommended measuring student progress as soon as possible.
"The superiority of looking at student growth was recognized from the very beginning," said Ed Haertel, a Stanford professor and testing expert who helped develop the API for the state. "It's much more sensitive and accurate than the current system."
But California, like most states, isn't doing it. Developing the sophisticated tracking systems necessary for value-added analysis takes time and money. Budget constraints and political infighting, among other things, have stood in the way.
L.A. Unified is in a better position to act. It has had the data and computer systems in place to measure student progress for more than a decade, but it repeatedly has ignored the advice of its own experts to do so.
In 2006, for example, district researchers and outside consultants proposed including value-added scores on a new "report card" for each school.
The idea was rejected by administrators as too complicated for parents, said Julie Slayton, the district's former head of research and planning who is now an education professor at USC.
"I would have no problem if somebody said there needs to be value-added as part of that report card," said Supt. Ramon C. Cortines, who had the idea of creating the report card, in a recent interview. "But it was not brought to my attention."
Today the report cards contain 108 data points for each school — including the percentage of staff who report feeling "commitment and collaboration" — but no value-added score.
Self-scrutiny
Esperanza Elementary's name in Spanish means "hope," though the school would seem to have little reason for it.
Its API score is 670, far short of the state goal of 800. For six years it has also failed to meet federal benchmarks, putting it at risk of takeover or closure.
But when the school, west of downtown Los Angeles, is judged by the progress students make in math and English, Esperanza ranks among the top 20% of district elementary schools, the analysis found. On average, students started third grade in the 39th percentile districtwide and ended it 11 points higher. In English, they started in the 26th percentile and gained 10 points.
This is despite some tough challenges: Students at Esperanza are mostly low-income and still learning English. For some of their parents, Spanish is a second language after their native Mayan tongues of Quiche or Kanjobal.
"We're under constant scrutiny, and we're under a lot of pressure to improve," said Principal Felicia Michell. "We're doing good work here, yet you don't get the recognition for those efforts."
Though the district has chosen not to use value-added, Michell long ago saw the benefit of even the most basic test score analysis.
Several years ago, she began holding meetings with teachers in which she projected their students' test scores on a wall and asked the instructors to discuss why some of the teachers were doing better than others.
"You had to be very careful at first," Michell said. "We're trying not to alienate any teachers."
Initially resistant, teachers began to recognize areas where they needed help.
"You start to question yourself, and that's the whole point of this," said Lynda Ayala, a veteran first-grade teacher at Esperanza, who was shown scores from periodic assessments of her students. "Eventually it gives you the freedom of knowing that even as a seasoned teacher you can say, 'This isn't working.'"
Ayala was struggling to get through to two of her students who had yet to learn the sounds of letters. An instructional coach suggested she pull them aside for five to seven minutes a day to focus on that area. By the end of the year, the two were reading at nearly grade level.
"Lynda became a believer," said Michell, a point Ayala confirmed. "Her fellow teachers saw that and decided to follow her lead."
Such self-scrutiny appears to be paying off, the Times analysis found. Yet by official measures, Esperanza's gains don't count.
It's not the only school whose success has been largely unheralded.
In recent years, Maywood Elementary has been the most effective elementary school in the district at raising student test scores. On average, students in second through fifth grades started the school year in the 39th percentile in math and ended it 30 points higher. Gains in English were smaller but still dramatic: 20 points. (The school's API has also risen significantly, to 830.)
Much of the credit again appears to go to the principal, a hands-on administrator who had the advantage of building the school from scratch. Before Maywood opened five years ago, Lupe Hernandez decided to visit the classrooms of nearly every teacher who had expressed interest in working at the school. She worked around rules that favor seniority by blatantly discouraging the teachers she wasn't impressed with.
"I told them straight out, 'You don't meet the caliber of what I'm looking for at Maywood,'" she said.
Those she wanted, she pursued.
Once classes started, Hernandez began tracking results from periodic exams, tallying them on sheets of paper marked with colored dots. Green meant progress, pink meant backsliding. Teachers with too much pink were offered help, and parents of struggling students sometimes found Hernandez at their doors.
"I tell everyone that they will have all the support they need but that I have extremely high expectations," she said.
The bottom third
If student progress was measured, other schools might find themselves under much more scrutiny.
Topeka Elementary in Northridge serves a community where one in four parents attended graduate school. Over the seven years analyzed, two-thirds of Topeka's students scored above grade level, contributing to its sterling API score of 879.
But the school is intently focused on bringing up those who score below. In part that's because the API, while not primarily concerned with students' progress, is designed to give more credit to gains by low achievers.
"It's where you get the most bang for your buck," said Principal Miko Dixon. "Everything we do is about getting those kids up."
Those low-achieving students made small but steady gains, the Times analysis found. The much larger group of high achievers was essentially flat in English and steadily falling behind in math. When ranked by student growth overall, Topeka was in the bottom 3% of district elementary schools. Its students made far less progress, on average, than their peers in Watts, Pacoima and other low-income neighborhoods across the city.
Dixon, who arrived at Topeka in 2009, expressed some surprise at the school's ranking, though she was aware that several teachers were struggling.
Yet Topeka faces none of the scrutiny or pressure to improve that Esperanza and other low-API campuses do.
"They ignore us," Dixon said.
Luck of the address
If fifth-grader Ilene Gallegos had lived a few blocks west in Watts, she might have ended up at 96th Street Elementary. Instead, she went to a school just a mile away: 92nd Street.
Thanks to that accident of geography, she may have received a remarkably better education.
On paper, the two campuses are practically identical. The vast majority of students are low-income and black or Latino. Both schools' API scores are below the state target of 800.
But the campuses are miles apart in terms of their students' growth: 92nd street ranks in the top 5% of the district, while 96th Street is in the bottom 5%, the Times analysis found.
In second grade, Ilene tested at grade level in math. By fourth grade, she answered every math question correctly on the state test, a feat that earned her a trophy. She made strong gains in English and scored at grade level by fourth grade.
The secret? "Every year here, she's had a pretty good teacher," said Ilene's mother, Elizabeth Rodriguez.
At 92nd Street, the average teacher was in the 76th percentile compared with peers in the district, according to the Times value-added analysis of teachers, reported on last week. Five teachers there were in the top 5%.
When former Principal Nanetta Arceneaux arrived in 2003, many teachers were retiring or moving to other campuses, giving her the rare opportunity to pick about half her staff — though without the benefit of data.
"You saw a big change once she came … teachers were on the kids about doing their homework and learning," said Sheila Phillips, whose daughter attends 92nd Street.
At 96th Street, by contrast, the average teacher was in the 40th percentile, and just one ranked above 83rd.
Principal Luis Heckmuller said that upon arriving in 2008, he found a staff dedicated but not thoroughly covering the standard curriculum. Teachers don't have to teach identically, he said, but it's a problem when some fifth-graders don't know the difference between a prefix and a suffix or what a metaphor is.
"We believe in academic freedoms, but we need to adhere to the California standards," said Heckmuller, whose school's test scores rose significantly according to results released last week.
Coveted campus
On a May morning at Wilbur Avenue Elementary, 25 parents toured the sprawling campus. Many said they were weighing Wilbur against private schools.
It's easy to see why they'd be interested, even apart from Wilbur's enviable API score of 896. Extracurricular activities include music and art, cooking, theater, sports, robotics and conversational Hebrew.
Most of this is paid for by the school's booster club, which raises more than $100,000 every year, said David Hirsch, principal at Wilbur before starting a new job this month at Hesby Oaks School in Encino.
Wilbur became a California Distinguished School two years ago based partly on its soaring API, he told the visiting parents during his pitch last spring in the auditorium.
But the Times analysis shows Wilbur's academic growth was less than distinguished. On average, students started third grade in the 77th percentile in math, but by the end of fifth grade were in the 67th. In English, they slid from 79th to 76th.
As a result, when judged by student progress, Wilbur ranked in the bottom 11% of district elementary schools.
In Hirsch's office is a framed certificate of excellence from the district, congratulating the school for raising its API score 140 points over three years. But that number may be more an indication of demographic changes than improved instruction. During the same years Wilbur's API was rising, the percentage of parents who had attended graduate school rose dramatically, and the portion of students in the free-lunch program fell. One reason was an influx of highly educated immigrants into the area, Hirsch said.
Asked about Wilbur's poor academic growth, Hirsch said, "Once you're this high, how much further can you grow?"
Yet research has shown no significant "ceiling effect." Indeed, many of the district's high-API campuses also show excellent growth, according to the Times analysis. For example, students at Wonderland Avenue Elementary in the Hollywood Hills start at an academic level similar to that of pupils at Wilbur, yet they continue to make some of the biggest gains in the district, particularly in math.
Don Wilson, Wonderland's principal, said challenging the high-performing students is a priority. For example, administrators identify some teachers' strongest subject and allow them to specialize in it — an unusual approach at the elementary level. The goal is to give students the strongest possible instruction from each of their teachers.
Hirsch, by contrast, is focused on recruiting and retaining high-achieving students, saying that pushes everyone's scores higher.
"Here you're running with the thoroughbreds," he said. "Sooner or later, it brings you up."
He told the visiting parents that he had resisted the district's attempts to make Wilbur more diverse through the open-enrollment program, which allows students from other neighborhoods to attend the school. "They say, 'We'll give you 50 seats, but you have to take 10 Hispanic kids for every white one,'" Hirsch told the mostly white group. "They do it by integration."
He said he'd rather pick local kids "out of a hat." In an interview later, Hirsch said he wasn't intending to discriminate — he simply prefers that approach to having minorities bused in from far away to fill a district goal.
He said he was puzzled by his school's poor showing in the Times analysis.
"There is not a lot of meat and potatoes here," he acknowledged, referring to the curriculum. "We do a lot of performance art ... maybe we do too much of that?"
Still, Hirsch insisted he was giving parents what they want. And many parents, including some who camped out to get their children in, say they're happy.
"Did I fail or succeed?" Hirsch said. "You be the judge."
No comments:
Post a Comment