U.S. Reforms Out of Sync With High-Performing Nations, Report Finds
“This paper is the answer to a question: What would the education policies and practices of the United States be if they were based on the policies and practices of the countries that now lead the world in student performance?“
- STANDING ON THE SHOULDERS OF GIANTS: An American Agenda for Education Reform by Mark S. Tucker
By Stephen Sawchuk – EdWeek Vol. 30, Issue 33 | http://bit.ly/iliY5n
May 27, 2011 - The United States’ education system is neither coherent nor likely to see great improvements based on its current attempts at reform, a report
released this week by the National Center on Education and the Economy concludes.
The NCEE report is the latest salvo in a flurry of national interest in what can be gleaned from education systems in top-performing or rapidly improving countries. It pushes further than other recent reports on the topic by laying out an ambitious agenda for the United States it says reflects the education practices in countries that are among the highest-performing on international assessments.
Among other measures, the report outlines a less-frequent system of standardized student testing; a statewide funding-equity model that prioritizes the neediest students, rather than local distribution of resources; and greater emphasis on the professionalization of teaching that would overhaul most elements of the current model of training, professional development, and compensation.
“I think we have been for a long time caught in a vicious cycle. We’ve been unwilling to do the things that have been needed to have a high-quality teaching force,” including raising the entry standard for teacher preparation and requiring prospective teachers to major in a content area, said Marc S. Tucker, the president of the NCEE.
“We’ve been unwilling to pay teachers at the level of engineers. We’ve been solving our problems of teacher shortages by waiving the very low standards that we have. We have been frustrated by low student performance, and now, we’re blaming our teachers for that, which makes it even harder to get good people,” Mr. Tucker continued.
The paper also states that progress on any one of the reform areas alone is unlikely to result in widespread boosts in student learning. All efforts, it says, are interconnected and should be linked to a coherent vision of what students should know and a system for ascertaining whether they achieve those goals.
The report also praises the United States’ progress on clearer, common academic standards in English/language arts and mathematics as a first step in defining such outcomes. But it notes that the success of that venture will depend on its ability to connect such expectations to the other pieces of the country’s education system.
Major Findings
Once a topic primarily reserved for academics, the “international comparisons” discussion has exploded over the past few years, with policymakers, pundits, and teachers’ unions arguing that better educating students is crucial to the nation’s economic success.
It has also been the subject of considerable federal interest. U.S. Secretary of Education Arne Duncan helped convene a major forum of education leaders from 16 countries in March, and he commissioned the Organization for Economic Cooperation and Development, a forum representing a group of industrialized nations, to produce a report about what lessons could be learned from the results of the Program for International Student Assessment, or PISA. ("International Leaders Urge Nations to Raise Status of Teachers," March 30, 2011.)
The NCEE report draws both on qualitative case studies of other countries’ systems and on the quantitative data and extensive background surveys produced as part of PISA. Much of the analysis incorporates information from the OECD report commissioned by Mr. Duncan, which NCEE also produced.
It builds on the former efforts, however, by contrasting the practices of those countries with undertakings in the United States.
For instance, the report notes that no other country has grade-by-grade national testing, pointing out that such countries as Singapore and Japan tend to use such exams sparingly, only at the end of primary and secondary schooling. The tests are closely linked to curricula and carry stakes for students in terms of progressing, rather than being used for school or teacher accountability.
Such countries also have much higher entry standards for teachers and require greater content knowledge, which is better integrated with training in pedagogy. In general, the report states, such efforts have helped to elevate the status of the profession, which is reflected in higher pay, more autonomy, and additional career opportunities as teachers advance.
Finally, teachers’ unions are prevalent in top-performing jurisdictions such as Finland and Ontario, Canada, but work in a “professional” rather than “industrial” mode. The report says that U.S. teachers must give up blue-collar work rules like seniority rights and recognize difference in performance in exchange for being treated as professional partners, who are given autonomy and trusted to diagnose and solve instructional problems on their own.
The report also takes aim at what it deems “myths” of international comparisons, such as the notion that other countries educate only an elite corps of students, or that their scores are higher because of less-diverse student populations.
The report concludes by calling on the federal government to fund a competition, modeled on the Race to the Top program, to help states adopt a comprehensive system of education practices used by other countries.
States, it says, should be the key level of government to help move toward a more coherent education system—as they have been in provinces, such as Ontario, that are part of federated nations.
On Track?
At an event where the report was released this week, panelists outlined different opinions about whether the agenda embodied in the report reflects or diverges from the current education reform efforts in the United States.
In his remarks, Secretary Duncan highlighted similarities between the two. He noted that, for instance, high-performing systems like Singapore use bonuses, scholarships, and salary supplements to reward great teaching and to attract teachers to hard-to-staff schools or shortage areas. The Obama administration has pursued such policies through the Race to the Top and other federal competitions.
“Clearly, our education system is not as far down the track as those of top performers, nor are we anywhere near where we need to be to win the race for the future,” Mr. Duncan said. “But we are not off-track or chugging down an abandoned spur line.”
He also praised the work on the common standards, which was underwritten by experts convened by the National Governors Association and the Council of Chief State School Officers. Forty-four states and the District of Columbia have adopted those standards, which draw heavily on curriculum guidelines used in top-performing countries.
Mr. Duncan stated, however, that the federal government would not prescribe a national curriculum as part of its support of the common-standards agenda. That comment came as an apparent rebuttal to a group of scholars and education advocates who have accused the Education Department of overstepping a federal law prohibiting it from mandating a national curriculum. (" 'Manifesto' Proposing Shared Curriculum Draws Counterattack," May 18, 2011.)
Other commentators, though, outlined perceived differences between international practices on teaching and the United States’ current efforts.
For instance, the Obama administration supports the idea of linking test scores to teacher evaluations. But many international education leaders at the March forum raised concerns about such policies.
“The perception is teacher evaluation based on narrow student test scores, and no country thinks that’s a good idea,” noted Vivien Stewart, the senior adviser for education for the Asia Society, a New York City-based nonprofit that facilitates policy dialogues between the United States and Asian nations, in an interview. “The evaluation systems in these countries tend to be fairly broad,” said Ms. Stewart, who is writing a paper about the issues discussed at the forum.
Singapore, she noted, has 16 domains in which evaluation takes place, including a focus on achievement, professional contribution to the school, community involvement, and relationship with parents.
Data on student performance and teaching are widely used to improve practice, but not disseminated in the public way they are in the United States, she added.
Challenging Views
William H. Schmidt, a professor of statistics and education at Michigan State University who has extensively studied other countries’ curricula, generally praised the NCEE report, especially for its focus on defining a specific body of knowledge students should master. Mr. Schmidt, who has also researched vast differences in the math skills of middle school teachers prepared in the United States, said teacher preparation should be the next frontier. ("U.S. Middle-Grades Teachers Found Ill-Prepared in Math," December 19, 2007.)
“We’re really at a precipice here. We’ve got these common standards, a nationally specified set of clearly focused standards. The problem is what comes next,” he said. “The U.S. has such a short attention span.”
The report’s general principles have been debated by other international scholars, however, who have raised concerns that the movement to common standards and tests could lead to more rigid schooling and lockstep expectations for students.
Many of the report’s recommendations also do not fit neatly within current U.S. debates about the use of assessments or how to upgrade the quality of teaching.
For instance, the national teachers’ unions have been among the strongest proponents of less standardized testing for accountability and more autonomy for classroom teachers. But doing away with seniority, which the report characterizes as a relic from “industrial” unionism, could be challenging.
The American Federation of Teachers has been reluctant to discard seniority as a factor in layoffs, noting that evaluation systems capable of distinguishing teachers by performance are not yet widespread.
At the release event, however, AFT President Randi Weingarten said that the union is open to discarding some work rules as long as teachers are treated fairly and maintain due process rights. She pointed as an example to the “thin” contract signed by AFT-affiliated teachers in a New York City charter school and the Green Dot charter-management organization, which among other provisions does not specify work hours for teachers.
And increasing teacher-preparation quality means tackling the perception of teacher education as an easy route to a diploma, a change that will have consequences, noted Mari Koerner, the dean of the education school at Arizona State University, a top preparer of teachers. She described losing teacher-candidates after the college increased the rigor of its preparation programs.
“These sentimental views of teachers [in the United States] drive me nuts,” Ms. Koerner said at this week’s forum. “[Preparation] is not about whether you love children; it is whether you can teach children.”
Panel Finds Few Learning Gains From Testing Movement
By Sarah D. Sparks- EdWeek Vol. 30, Issue 33 | http://bit.ly/kAmcTU
May 26, 2011 - Nearly a decade of America’s test-based accountability systems, from “adequate yearly progress” to high school exit exams, has shown little to no positive effect overall on learning and insufficient safeguards against gaming the system, a blue-ribbon committee of the National Academies of Science concludes in a new report: Incentives and Test-Based Accountability in Education
“Too often it’s taken for granted that the test being used for the incentive is itself the marker of progress, and what we’re trying to say here is you need an independent assessment of progress,” said Michael Hout, the sociology chair at the University of California, Berkeley. He is the chairman of the 17-member committee, a veritable who’s who of national experts in education law, economics and social sciences that was launched in 2002 by the National Academies, a private, nonprofit quartet of institutions chartered by Congress to provide science, technology and health-policy advice. During the last 10 years, the committee has been tracking the implementation and effectiveness of 15 test-based incentive programs, including:
• National school improvement programs under the No Child Left Behind Act and prior iterations of the Elementary and Secondary Education Act;
• Test-based teacher incentive-pay systems in Texas, Chicago, Nashville, Tenn., and elsewhere;
• High school exit exams adopted by about half of states;
• Pay-for-scores programs for students in New York City and Coshocton, Ohio and;
• Experiments in teacher incentive-pay in India and student and teacher test incentives in Israel and Kenya.
On the whole, the panel found the accountability programs often used assessments too narrow to accurately measure progress on program goals and used rewards or sanctions not directly tied to the people whose behavior the programs wanted to change. Moreover, the programs often had insufficient safeguards and monitoring to prevent students or staff from simply gaming the system to produce high test scores disconnected from the learning the tests were meant to inspire.
“I think there are some real messages for school districts on accountability systems” in the report, said Kevin Lang, an economics professor at Boston University who, during his time on the committee, also served as a district school board member in Brookline, Mass.
“School boards need to have a means for monitoring the progress of their school systems, and they tend to do it by looking at test scores,” he said. “It’s not that there’s no information in the objective performance measures, but they are imperfect, and including the subjective performance measures is also very important. Incentives can be powerful, but not necessarily in the way you would like them to be powerful.”
Gaming the System
Among the most common problems the report identifies is that most test-based accountability programs use the same test to apply sanctions and rewards as to evaluate objectively whether the system works. As a result, staff and students facing accountability sanctions tend to focus on behavior that improves the test score alone, such as teaching test-taking strategies or drilling students who are closest to meeting the proficiency cut-score, rather than improving the overall learning that the test score is expected to measure. This undercuts the validity of the test itself.
For example, New York’s requirement that all high school seniors pass the Regents exam before graduating high school led to more students passing the Regents tests, but scores on the lower-stakes National Assessment of Educational Progress, which was testing the same subjects, didn’t budge during the same time period, the report found.
“It’s human nature: Give me a number, I’ll hit it,” Mr. Hout said. “Consequently, something that was a really good indicator before there were incentives on it, be it test scores or the stock price, becomes useless because people are messing with it.”
In fact, the report found that, rather than leading to higher academic achievement, high school exit exams so far have decreased high school graduation rates nationwide by an average of about 2 percentage points.
The study found a growing body of evidence of schools and districts tinkering with how and when students took the test to boost scores on paper for students who did not know the material—or to prevent those students from taking the test at all.
Recent changes to federal requirements for reporting graduation rates, which require that schools count as dropouts students who “transfer” to a school that does not award diplomas, may help safeguard against schools pushing out students to improve test scores or graduation rates. Still, the National Academies researchers warned that state and federal officials do not provide enough outside monitoring and evaluations to ensure the programs work as intended.
AYP and Academics
For similar reasons, school-based accountability mechanisms under NCLB have generated minimal improvement in academic learning, the study found. When the systems are evaluated—not using the high-stakes tests subject to inflation, but using instead outside comparison tests, such as the NAEP—student achievement gains dwindle to about .08 of a standard deviation on average, mostly clustered in elementary-grade mathematics.
To give some perspective, an intervention considered to have a small effect size is usually about .1 standard deviations; a 2010 federal study of reading-comprehension programs found a moderately successful program had an effect size of .22 standard deviations.
Moreover, “as disappointing as a .08 standard deviation might be, that’s bigger than any effect we saw for incentives on individual students,” Mr. Hout said, noting that NCLB accountability measures school performance, not that of individual students
Committee members see some hopeful signs in the 2008 federal requirement that NAEP scores be used as an outside check on achievement results reported by districts and states, as well as the broader political push to incorporate more diverse measures of student achievement in the next iteration of ESEA.
“We need to look seriously at the costs and benefits of these programs,” said Daniel M. Koretz, a committee member and an education professor at Harvard University Graduate School of Education in Cambridge, Mass. “We have put a lot into these programs over a period of many years, and the positive effects when we can find them have been pretty disappointing.”
Jon Baron, the president of the Washington-based Coalition for Evidence-Based Policy and the chairman of the National Board for Education Sciences, which advises the Education Department’s research arm, said he was impressed by the quality of the committee’s research review but unsurprised at minimal results for the various incentive programs.
Incorporating diverse types of studies typically reduces the overall effects found for them, he noted, adding that the study also addresses a broader issue. “One of the contributions that this makes is that it shows that looking across all these different studies with different methodologies and populations, some in different countries, there are very minimal effects in many cases and in a few cases larger effects. It makes the argument that details matter,” Mr. Baron said.
“It’s an antidote to what has been the accepted wisdom in this country, the belief that performance-based accountability and incentive systems are the answer to improving education,” Mr. Baron said. “That was basically accepted without evidence or support in NCLB and other government and private sector efforts to increase performance.”
No comments:
Post a Comment