June 4, 2007

Value Subtracted

Filed under: Testing by Jackie Bennett @ 2:17 am

In their Children’s First brochure, Klein and his people describe the system of value-added assessments that will become the primary means to fail schools,  fire principals, bully teachers, and test-prep children unto death. In the value added system, schools will be evaluated based how upon  much progress students have made as they move from grade to grade.

For example, let’s say that last year 40% of  the fourth graders at a particular school did not meet learning standards in ELA.  The question then becomes, well, how did these same kids do this year, in grade five? Was the school able to decrease the number of failing students by 10%, or even 2%?  Or, did the failure rate go up?

Based upon the answer to that question, schools will rise and fall. In fact, according to the brochure, 55% of a school’s grade is derived from value-added data alone (and another 30% from overall performance on the tests).  

Whether or not that’s a good idea is not the subject of this post.  But if value-added data matters so much in judging all of us – and if it is going to be part of this clear and transparent information we hear so much about – then why doesn’t it seem to count in judging them?  More specifically, why didn’t Klein and the DoE discuss their own “value-added” data in the PR they released with the 2007 ELA test scores several weeks ago?  After all, with students now taking standardized state tests every year from grades three to eight, it is possible – in fact easy – to follow groups of children (cohorts) across time, from grade to grade.    

Well, maybe Klein didn’t want to draw attention to his own value-added failure.  Because  now that Diane Ravitch has examined the school wide cohorts  in a recent post  one thing is clear: should we hold Klein to his own standard, he has failed. As Ravitch points out,  “After five years of mayoral control and four years of the Children First “reforms,” test scores decline steadily for each cohort.” 

Ravitch illustrates the cohort declines in two ways. First, she shows how during the past year, passing rates declined for students in every tested grade as they moved from one grade to the next.  Every group declines. Even factoring  out the ELL students (as Klein would wish), does not change that trend. 

Ravitch also focuses on the eighth graders because these students are the first eighth graders for whom we have grade four results.  Ravitch writes, “In 2007 [as eighth graders], 41.8% of this group met the standards; in 2003, when these children were fourth graders, 52.5% met the state standards, a drop of 10.8 points.”

Clearly, Klein is not doing well when it comes to his own standards.  And to Ravitch’s observations, I add two more of my own. 

  1. This year’s group of sixth graders is the first that can be tracked across three years in a row.  Their results are especially important  because in 2003 these kids were in second grade, which means that they received  significant instruction under the DoE’s early-elementary programs, which is where Klein’s DoE focused much of its effort. Here are the scores:

ela_passingrates.jpg

  1. This year’s 4th graders were in Kindergarten when Klein and the current DoE  took over. No group in the system has had more thorough exposure to DoE programs. We can only track them across a single grade so far, but in that time, this group shows a drop-off after grade 3 of  5.5 percentage points (from 61.5% to 56%).

Overall the  trends are troubling.  What are we to make of them?   Some possibilities are:

  1. Value-added data is unreliable.   Perhaps this data was never meant to be read across the grades. But if that is the case than why is the DoE going to rely on it to judge the schools?
  1. There is a natural decline as kids get older.  I actually heard a top-level DoEer offer this explanation recently for the city’s failure to increase the success of older students.  But if that is the case, then how can schools be punished if they can’t reverse the trend?
  1. The chickens of balanced literacy are coming home to roost.  But if that’s the case, then why was it imposed on all the schools, and why were the voices of educators who resisted, marginalized – worse than that: u-rated, silenced, crushed?
  1. Test prep focusing on the discreet skills of reading  is no substitute for education.  But if that’s the case, why are we investing 80 million in ARIS – A Really Insidious System designed to ratchet up the test-prep learning that is already undermining New York City schools?

When it comes to value-added, there are a lot more questions than there are answers.  The question that bothers me most, however, is this: If value-added data is the single most important factor in judging the schools, and if the DoE plans to declare school failure based data primarily on this data, then why is it slipping its own value declines beneath the rug? 

Comment »

Comments are open for registered users and do not reflect the views of the UFT. Please read our general rules for commenters.

  • And there’s still the gorilla in the room: these tests may simply not be reliable instruments, in which case year to year comparisons may be meaningless.

    If the year to year numbers fluctuate, that would be a sign that something is wrong with the tests

    (the old 4th grade and 8th grade tests did exactly that. They were probably useless for comparison, if anyone wanted to learn anything. Useless for classifications finer than 1,2,3,4 (eg high-2 probably meant nothing), but useful for newspaper headlines, torturing kids, scaring teachers, etc)

    Jonathan

    Comment by jd2718 — June 5, 2007 @ 6:42 am

Leave a comment

You must be logged in to post a comment.