GREAT UNCERTAINTY OVER DIRECTION OF STATE STANDARDIZED TESTS: “State policymakers are ready to deemphasize the role of standardized tests in the school accountability system”

By John Fensterwald, EdSource Today |  http://bit.ly/Tujwme

November 26th, 2012  ::  With the statute authorizing state standardized tests due to expire in June 2014, the incoming Legislature is facing some hard decisions on the future of the state testing system: What subjects should be tested, for whom, how often (not every year in every subject, perhaps), at what cost, and, perhaps the biggest question, for what purpose?

The state will likely end up with a hybrid system, a combination of state-created tests and tests designed in partnerships with other states. The principal partnership is Smarter Balanced Assessment Consortium, one of two multistate consortia with contracts with the U.S. Department of Education to develop an assessment system aligned to the Common Core State Standards. Smarter Balanced is designing tests for California and two dozen other states. Its new tests are expected to be more demanding and will require new approaches to teaching. But the tests, due to roll out in spring 2015, will  cover only math and English language arts in grades three through eight and an important 11^th grade college and career readiness assessment.

That leaves other grades, starting with 2^nd grade, which California currently tests, as well as science, social studies, end-of-course high school exams and CAHSEE, the high school exit exam, along with the redesign of tests for English learners and special education students.

Legislators must decide which tests should be administered with incomplete information; Smarter Balanced officials have acknowledged that the more intricate Common Core assessments, which promise to measure critical thinking and higher-order skills, will take longer and cost more than the current multiple-choice California Standards Tests, which average 8-9 hours per grade (less in elementary, more in high school) and $13 per student. Initial estimates are at least 50 percent more in time and expense for the math and English language arts tests, which will include short and long-response questions requiring that students show and explain the reasons behind their answers.

Over the past year, an advisory committee to State Superintendent of Public Instruction Tom Torlakson has hashed through the issues during eight day-long meetings. Incorporating the committee’s thoughts  and more than 1,000 public comments that he received, Torlakson will issue a report with his perspective sometime in the next few weeks. But that report is more likely to be outline options than make definitive recommendations, said Torlakson spokesperson Paul Hefner.

The challenge will be to make sound decisions when so much is in flux.

• California is one of two dozen states taking a lead role in writing the Next Generation Science Standards, based on a framework created by the National Research Council, affiliated with the National Academy of Sciences. Like the Common Core standards, new science standards will stress conceptual knowledge and principles over rote knowledge. The final standards are due to be released by the end of this year. Once they’re complete, California must decide whether to create its own tests or develop them with other states.
• Earlier this month, the State Board adopted new English Language Development standards for English learners that are aligned to Common Core state standards in reading and writing. New assessments must now be created.
• In funding Smarter Balanced and PARCC, the other consortium with 23 member states, to create Common Core assessments, Congress required that the tests meet current federal accountability requirements. But Congress has remained deadlocked on reauthorizing the No Child Left Behind law, so it’s unclear how much the requirements might change.
• Smarter Balanced has committed to creating tests that will use online technologies. They not only will be administered on computers, but they’ll also be computer-adaptive – individually tailored, assigning questions based on students’ answers to previous questions. Computer-adaptive tests can reveal how many grades ahead or behind students are; thus, in theory, the 11^th grade Smarter Balanced exam could replace the state’s high school exit exam. But computer-adaptive tests also require a much larger library of questions than regular standardized tests, as well as sophisticated software. Skeptics question whether the consortium will fulfill its demanding commitments; even if they do, it’s an open question whether many California districts will have the broadband capacity and the needed computers  by 2015 to administer the test. For those that don’t, Smarter Balanced has promised pen-and-paper tests for three years as a fallback.

“We’re not sure what computer-adaptive can do,” said State Board of Education President Michael Kirst, a member of Torlakson’s Advisory Committee. “Can it really (replace) the exit exam? There are a lot of unknowns: what we can afford, how long Smarter Balanced will take, whether we will have to go to pencil and paper to simulate a computer.”

• At least in California, the pendulum is swinging in the opposition direction; after a decade of testing under No Child Left Behind and 15 years under California’s STAR (Standardized Testing and Reporting) system, state policymakers are ready to deemphasize the role of standardized tests in the school accountability system, the API (Academic Performance Index). Last year, the Legislature passed and Gov. Jerry Brown signed SB 1458, which will require the use of criteria other than test results for 40 percent of a high school’s API score. Torlakson and the State Board will decide what those measurements will be, with an emphasis on career and college readiness criteria, such as Advanced Placement participation, availability of career and technical education and a school’s dropout rate. Because state and federal accountability systems have been skewed so heavily toward math and English language arts tests, to the detriment of other subjects, Torlakson will recommend to the Legislature giving more weight to exams in history and the sciences.

Combine all of the uncertainties and cross-currents of opinions, and the Legislature will be left with a series of tough questions:

• What are the tradeoffs, in cost and length of tests, as the state takes the lead from Smarter Balanced and, in state-administered tests, shifts from pure multiple-choice tests toward more complex assessments using short answers and lengthy problem-solving tasks?

• Can the state afford the money, and schools afford the time, to administer more complex tests in every subject every year?

• Assuming the state won’t have all new state tests in place by the spring of 2015 – all but a certainty – what should the phase-in period be?

• Can the Smarter Balanced assessments incorporate the states’ current high school exit exam?

• Should the test for second grade be a purely diagnostic exam, to inform parents and teachers, and not be included in the state’s school accountability system?

• Should end-of-course exams in high school, ranging from Biology and Physics to Algebra II and Summative Math,  be turned over to districts to be administered locally and excluded from the state accountability system?

• Will the state, in order to save money and time on some tests, use matrix sampling, in which all students might take some questions, while other time-consuming portions of the test are given to equivalent samples of students? With matrix sampling, the focus is on school and district scores, not individual test results. (The National Assessment of Educational Progress, or NAEP, uses matrix sampling.)
• Will the Smarter Balanced assessments effectively measure career readiness, however it’s defined?

Summing up the dilemma facing the state, Kirst said in an interview, “We haven’t figured all of this out yet. It’s very complex.”

 

Rationale and purpose behind state tests may soon change In a report to be released to the Legislature within a few weeks, State Superintendent of Public Instruction Tom Torlakson and the state Department of Education will assert that the primary purpose of state tests is “to model and promote high quality teaching and student learning activities,” according to a document prepared this month for the State Board of Education. This differs from what the goal has been under the state STAR (Standardized Testing and Reporting) system since its creation in 1997: to hold schools accountable for teaching students the state content standards. In a nutshell, the new goal will be to improve instruction while the current goal is to measure the results of instruction. << Deputy State Superintendent Deb Sigman sits on the executive committee of the Smarter Balanced Assessment Consortium. The difference may appear semantic, but it is fundamental. State Deputy Superintendent Deb Sigman says it’s important to establish the rationale for tests before deciding what they will look like and when and to whom the tests should be administered. “Having the purpose of modeling good instruction is a different mindset as you develop the assessment system,” she says. The current California Standards Tests and the future Smarter Balanced assessments aligned with the Common Core standards reflect the difference in purpose. Except for writing tests in fourth and seventh grades, the CSTS are all multiple choice, designed to show if students know content standards. The tests aren’t intended to guide classroom instruction. To the contrary, in many districts, the tests are preceded by weeks of test preparation for figuring out how to best get the right answer. Smarter Balanced assessments will also have multiple-choice questions, but the heart of the tests in both English language arts and math will be short-answer questions and lengthy performance tasks. The latter questions, involving multiple steps, taking perhaps an hour or two, are designed to see if students can demonstrate a deeper understanding of the standards, can explain the reasons behind their answers and can think critically. For example, a sample math performance task released recently by Smarter Balanced asks fourth graders a series of questions about calculating how many 1 ½ inch-diameter tulip bulbs can be planted in a 5- by 2-foot planter (see example for the full sequence of activities). Smarter Balanced has promised to provide interim assessments, with similar types of questions that students would take during the year, to show if students are on track to pass the end-of-year or summative assessment. Sigman, who is one of only nine members of the Smarter Balanced executive committee, says that the performance tasks are the type of instruction that teachers should be using. Including them in the assessments is a “powerful lever” for changing how standards are taught, she says. And it is the model that the state should use in designing other standardized tests, she says. Doug McRae, who spent his career designing large-scale standardized tests, couldn’t disagree more with Sigman. McRae, who frequently comments at State Board meetings on assessment-related issues, argues that the purpose of statewide tests should remain measuring how much students have learned. And there should be a clear distinction in roles. Doug McRae, a retired testing publisher, from Monterey>>. According to McRae, whoever creates the summative tests, whether the state Department of Education or a national consortium, should not be designing interim assessments with an eye toward improving how teachers teach. That’s the role for school districts. “When instructional tests (such as diagnostic, formative, and interim tests) are included in a statewide assessment system, they inevitably become tools to ‘teach to the test,’ a cancer that will degrade both good instructional systems and good assessment systems,” he wrote this month in a draft comment to the State Board. “The argument that constructed response and performance task items will combat teaching-to-the-test by having items ‘worth teaching to’ is a false promise. Test takers, teachers, and administrators have a long history of finding teaching-to-the-test shortcuts that ultimately conflict with sound instructional practices… .  There will be Cliff Notes, Scoring High or Princeton Review type programs for (Smarter Balanced) tests, and efforts to isolate only material that is likely to appear on the test.” McRae says it is inherently implausible to have reliable tests designed to measure the results of instruction and to improve instruction, as Smarter Balanced is promising. But Sigman says that the two goals are not mutually exclusive. Aligning interim and summative tests and encouraging the use of performance tasks is useful in developing the skills that students are expected to master. “What may sound like test prep is actually learning,” says Torlakson spokesperson Paul Hefner. Going deeper Section of the Education Code that details what the Superintendent of Public Instruction must do to prepare for the renewal of state standardized tests or creation of new assessments as of July 1, 2014. Samples of the types of questions that Smarter Balanced tests are likely to include in math and English language arts. Sample questions of the California Standards Tests; Video of the Nov. 8 meeting of the State Board of Education meeting at which Deputy Supt. Deb Sigman and the board discussed issues related future state assessments (second item on the agenda); Document prepared for the State Board of Education that summarizes the current state standardized test system and possibilities for the future; State Department of Education information on the Next Generation Science Standards.