Saturday, October 18, 2014

UPDATED AGAIN: TWO THOUGHTS (and some statistical analysis) WORTH FAR MORE THAN 2¢ RE: THURSDAY’S TRANSITION

By e-mail to 4LAKids from a knowledgeable school-based educator and occasional contributor

Thur, Oct 16, 2014 11:58 pm | Updated further Oct 22 (see following)

As I read about Supt. Deasy's resignation, I have two thoughts:

The MiSiS debacle was more of a factor than the iPads.

The decision to implement MiSIS this year was negligent, and schools are a shambles due to MiSIS. Unfortunately, there's no obvious way to extricate ourselves from this mess that affects, to varying degrees, every school in LAUSD. Students will be hurt, and after multiple system failures, employees have lost all faith in LAUSD’s Information Technology Division. At a series of eight meetings (22 hours total) hosted by Associated Administrators of Los Angeles (AALA) between November, 2012 and May, 2014, Chief Information Officer Ron Chandler, Chief Strategy Officer Matt Hill and other high district and ITD officials were warned repeatedly, and in compelling detail, by school site administrators and coordinators, but they chose to ignore the school-based experts who would have to use the system.

Most stories state that Supt. Deasy raised test scores, so I reviewed the data this evening.

The Deasy superintendency began in April, 2011, shortly before the CST exams were given, so 2011 seems a sensible baseline. Since the CST was not given in 2014, claims about Supt. Deasy raising test scores rest on the 2012 and 2013 CST. Attached (following) is a spreadsheet that includes the 2011-2013 LAUSD and (for comparison) statewide CST ELA scores for grades 3-11, the CST Math scores for grades 3-6, and the CST Algebra 1 scores for grades 7-11. Cohort views of the ELA and Math are included so that one can see how the same (or substantially the same) group did through three years of testing. There are a few bright spots (6th grade and 10th grade English; 4th and 6th grade math; 8th grade Algebra 1), but there are no huge, across-the-board improvements. Besides, the achievement of an 8th grader on the 2013 CST is the consequence of at least nine years of schooling, only two of which were during Mr. Deasy's superintendency.

  • UPDATE: at the request of the anonymous contributor a misspelling was corrected in the first paragraph. At the request of another the spreadsheet has been made more legible.
            CST ELA % Proficient        
CST ELA % Proficient         by Cohort       Change Change
        Change Change   2011 2012 2013 2011‐2013 2012‐2013
Grade 3 2011 2012 2013 2011‐2013 2012‐2013   Grade 3 Grade 4 Grade 5    
LAUSD 39 43 40 1 (3) LAUSD 39 62 53 14 (9)
CA 46 48 45 (1) (3) CA 46 67 60 14 (7)
                       
Grade 4             Grade 4 Grade 5 Grade 6    
LAUSD 58 62 58 0 (4) LAUSD 58 54 49 (9) (5)
CA 64 67 65 1 (2) CA 64 63 60 (4) (3)
                       
Grade 5             Grade 5 Grade 6 Grade 7    
LAUSD 51 54 53 2 (1) LAUSD 51 47 47 (4) 0
CA 59 63 60 1 (3) CA 59 59 60 1 1
                       
Grade 6             Grade 6 Grade 7 Grade 8    
LAUSD 41 47 49 8 2 LAUSD 41 49 44 3 (5)
CA 55 59 60 5 1 CA 55 62 57 2 (5)
                       
Grade 7             Grade 7 Grade 8 Grade 9    
LAUSD 44 49 47 3 (2) LAUSD 44 48 45 (1) (3)
CA 57 62 60 3 (2) CA 57 59 62 5 3
                       
Grade 8             Grade 8 Grade 9 Grade 10    
LAUSD 42 48 44 2 (4) LAUSD 42 39 43 1 4
CA 57 59 57 0 (2) CA 57 57 52 (5) (5)
                       
Grade 9             Grade 9 Grade 10 Grade 11    
LAUSD 37 39 45 8 6 LAUSD 37 39 40 3 1
CA 55 57 62 7 5 CA 55 50 48 (7) (2)
                       
Grade 10             Grade 10 Grade 11      
LAUSD 35 39 43 8 4 LAUSD 35 41   6 NA
CA 48 50 52 4 2 CA 48 48   0 NA
                       
Grade 11                      
LAUSD 37 41 40 3 (1)            
CA 45 48 48 3 0            
                       

CST Math % Proficient         CST Math % Proficient        
        Change Change by Cohort       Change Change
Grade 3 2011 2012 2013 2011‐2013 2012‐2013   2011 2012 2013 2011‐2013 2012‐2013
LAUSD 66 69 65 (1)     Grade 3 Grade 4 Grade 5    
CA 68 69 66 (2)   LAUSD 66 67 61 (5) (6)
            CA 68 71 65 (3) (6)
Grade 4                      
LAUSD 67 67 70 3     Grade 4 Grade 5 Grade 6    
CA 71 71 72 1   LAUSD 67 61 46 (21) (15)
            CA 71 65 55 (16) (10)
Grade 5                      
LAUSD 60 61 61 1     Grade 5 Grade 6 NA    
CA 63 65 65 2   LAUSD 60 45      
            CA 63 55      
Grade 6                      
LAUSD 41 45 46 5              
CA 53 55 55 2              
                       

   

% Proficient

 

Change

Change

CST Algebra 1

2011

2012

2013

2011‐2013

2012‐2013

LAUSD Grade 7

77

81

81

4

0

CA Grade 7

83

86

86

3

0

LAUSD Grade 8

32

35

40

8

5

CA Grade 8

47

49

50

3

1

LAUSD Grade 9

13

15

16

3

1

CA Grade 9

23

25

25

2

0

LAUSD Grade 10

8

9

10

2

1

CA Grade 10

13

13

13

0

0

LAUSD Grade 11

9

9

8

(1)

(1)

CA Grade 11

9

10

10

1

0

 


Update: HERE IS MORE INPUT FROM ANOTHER ANONYMOUS 4LAKids READER WHO HAS MORE TO SAY ABOUT THE API SCORES DURING DR. DEASY’S SUPERINTENDENCY …SUPERINTENDENCE …WHATEVER, WE KNOW WE CAN’T SAY  ‘TENURE’!

by email from another anonymous source

Update: Oct 22, 2014

This graph shows each grade level's change in ELA CST from grade 3 – grade 11 between 2011 and 2013; LAUSD on the left, CA on the right.

This graph shows each school type's change (Upper Elementary, Middle, High) in ELA CST from Upper Elementary – High School between 2011 and 2013.

Combining the grades into these three groups makes it a little bit easier to see a pattern and the groupings do make some sense pedagogically.
The overall trends of all three groups are the same in LAUSD as in CA at large. Note the scales were not constructed to be the same (and they should be). The years are 2011 2012 and 2013 respectively – obviously axes aren't labeled properly. All three school groups are considerably less on average in LAUSD than in CA as a whole; the trends are all similar but the magnitude of change, but rise and fall, are smaller (more flat) in LAUSD than CA – which is interesting given how much larger all of CA is – one would possibly expect more buffering and less volatility. Note that LAUSD is the largest school district and contained in the CA numbers – these are not independent groups. It is therefore perhaps not surprising that the overall trends are the same. A comparison with all that is not LAUSD would be even more diminished in absolute value (that is the {CA, w/o LAUSD} curve would be even higher and more distant from the {LAUSD} curve).

All these differences in all directions and dimensions are statistically significant … but are they meaningful?

Recall that API scores cannot be used as a longitudinal comparison. They must and may only be used to compare pairs of years, not three in a row. And that only if the adjusted API is used as a base comparison. Because the components of API change completely from year to year, there is nothing comparable between years about them. However, the statistics are recalculated every year in the spring to use next year's new set of metrics. Every year therefore has two sets of API scores, one that compares with last year's, and the new one. The old one is what is reported so that it is comparable with what one knew from the previous year – the re-calculated one that is. But often instead what is compared is the previously-reported "old" API, which is comparable only with the previous years', not the forward-years'. This is all never explained – for example, who knows which APIs are reported here in this data?

It all matters…

And, it makes further analysis of these scores pointless. They are not comparable over a series of three years, as minimal as that snapshot would be!

No comments: