Left Ed: Arne Duncan Flunks Race to the Top

by: jeffbinnc

Sun Aug 29, 2010 at 13:00


Metrics, metrics everywhere. In this brave new world of education "reform," its the great god of "data," we are told, that will lead the way to a brighter world for all children.

Never mind that "data" do not necessarily equate evidence. Never mind knowledgeable skeptics, such as Alfie Kohn, warn that the more educators allow themselves to be fixated on numbers, "the more trivial their teaching becomes and the more their assessments miss." Never mind that experienced classroom teachers, such as Walt Gardner, point out that data on how students perform on tests is not the same thing as learning. And never mind that research from the UK has just revealed that students actually do better at school when teachers aren't obsessed over test data.

At no time has data mania been more on display than the past week when two enormous - some say historic - events played out across the education and (occasionally) general media. The first event was the announcement of the winners of the much-coveted money from Secretary Arne Duncan's Race to the Top competitive grant competition. RTTT winners were determined by an elaborate scoring formula that reform enthusiasts assure us rewards state school systems that are being the most "data-driven" in their approaches to policy.

Then, the day after the data-filled revelry surrounding the RTTT winners, Duncan endorsed using data derived from student testing as a means for evaluating individual teachers too. Following a series of reports by the Los Angeles Times, where reporters used a value added analysis of test score data to evaluate the "effectiveness" of individual teachers, Duncan apparently decided that such a flawed methodology should be spread around universally.

Certainly, tallying up numbers related to the inputs and outcomes of education has its place. We don't need a data-free world. But how, pray tell, do the people running USED know they are looking at the right data - data that are both fair and effective indicators of educational progress? If Duncan is so sure of his methodology, then maybe, for instance, he would feel okay with applying it to his own track record as a leader of Chicago public schools?

jeffbinnc :: Left Ed: Arne Duncan Flunks Race to the Top
As OpenLeft commenter VLazlo suggests in a comment to my Quick Hit on Duncan's teacher evaluation scheme, if Duncan applied his "data-drive" evaluations to the results he had leading Chicago public schools, he might not like the score. Specifically, taking the broad data indicators used to judge RTTT entrants, Duncan's record at Chicago public schools would most likely result in a failing grade:

Great Teachers and Leaders (138 total points)
In 2007, Duncan's plans for improving teacher effectiveness in Chicago relied primarily on a teacher merit pay program called Teacher Advancement Program, or TAP. Although the program was hugely controversial with teachers, Duncan charged forward with it under the mantra that "business approaches" to education would yield the best results. So now that we have the benefit of hindsight, what were the results?

Not so hot. As WaPo's Valerie Strauss explains:

"Under the program in Chicago, payments to teachers under the program averaged $1,100 for those in schools in their first year of implementation, and $2,600 for those teachers in schools in their second year.
The comparison with similar schools that didn't use the program revealed no real difference in student scores or in teacher-retention rates among those schools."

Not only did "pay for performance" do nothing to create "great teachers" in Chicago, it quite likely produced a number of very negative effects, according to this analysis, including:
* Damaging curriculum by emphasizing only those subjects that are tested
* Lowering teacher morale by damaging their sense of professionalism
* Increasing cheating
* Leading to test score inflation as teachers do more teaching to the test

Success Factors (125 total points)
Despite the damage Duncan inflicted on teachers, what effects did he have on student achievement. Here again, the results are mixed at best.

In a report (pdf) that Diane Ravitch links to "the gains registered in the elementary schools of Chicago during Arne Duncan's tenure were almost entirely the result of changes to the scoring of the tests, rather than evidence of any genuine improvement in student learning."

On a much broader scale, Duncan's policies were in fact quite damaging to Chicago. As this analysis reveals, the results of Duncan's signature program Renaissance 2010 was actually "traumatic, largely ineffective, and destabilizing to communities." By arbitrarily shutting down schools and shuffling students - primarily minority and poor - from one school to another, Duncan's policies led to severe spikes in student violence, severe dislocations, and job losses.

Standards and Assessment (70 total points)
Scoring in this area relies primarily on the extent to which schools have adopted the Common Cores Standards for curriculum, which did not exist when Duncan ran CPS. But it's quite likely that he would have embraced the standards had they been available.

Regarding testing and assessment, CPS recently announced that scores on standardized tests indeed have increased. But as this local commentator puts into perspective, 63% of Chicago black males don't graduate from high school, and CPS has one of the highest rates of suspensions of any big-city school district with more than 60% of expelled students last year being black and male.

So one has to wonder if the key to more effective standardized testing in Chicago has been achieved largely by getting rid of students who may be more apt to score the lowest on these tests?

General Selection Criteria (55 total points)
As this assessment area relies primarily on the implementation of charter schools, I'll give Duncan flying colors - all 55 points.

Turning Around the Lowest-Achieving Schools (50 total points)
Although Duncan's turnaround model of closing Chicago's lowest performing schools has become the model to be implemented across the nation, this approach has proven to be generally ineffective. According to this analysis (pdf), only 6 percent of schools affected by school closings ended up at schools that were higher achieving than the ones they left.

But if you interpret the "turnaround" criterion strictly as committing the act of turning around, regardless of the results, certainly Duncan deserves some credit. So split the difference maybe?

For the last two RTTT assessment areas, Data Systems (47 total points) and STEM (15 total points), I'm having difficulty coming up with anything definitive. But even if Duncan won flying colors in these areas, it's not enough to give him a passing grade overall, based on his own standards for "success."

So with Duncan himself being unable to measure up to the standards he has set for the nation's schools, does it mean that we dismiss Duncan's leadership, the standards, or both?


Tags: , , , , (All Tags)
Print Friendly View Send As Email

Data, like rules (0.00 / 0)
are for the little people.  

That's why teachers must be fired, but the people running the war, the economy, and education policy must remain in charge.

Politics is the art of the possible, but that means you have to think about changing what is possible, not that you have to accept it in perpetuity.


Yes (4.00 / 1)
Data is supposedly how business works.  My experience in the business world calls bullshit on that in the loudest possible terms.  

Managers want to see data that validates them and their rule.  Data that suggests otherwise is deep-sixed so fast it would make your head spin.

A successful business is one that understands the REAL criteria for its success.  It is not driven by made-up data.

sTiVo's rule: Just because YOU "wouldn't put it past 'em" doesn't prove that THEY did it.


[ Parent ]
Data use (4.00 / 3)
Data is good to base best practices on.

The problem is most people don't understand the difference between causality and correlation, nor do they understand the difference between reliability and validity as these relate to data.

If teaching is so easy, then by all means get your degree, pass your certification test(s), get your license, and see if you can last longer than the five years in the classroom 50% of those who enter the profession never make it to.


[ Parent ]
Data worship (4.00 / 1)
The problem comes in when they say that any number, is better than no number at all.  T'aint nececessarily so.  Some data really measure something important, others don't.  Even if data really measure something, they still may be over-relied upon.  Reading comprehension scores measure something, assuming they are not contaminated by teaching to the test or by cheating.  But without measuring all the other factors that go into these results, it is an abuse of data, reading more out of data than the data truly indicate, to base personnel decisions solely on them.

As a computer professional, I find that data-worship is impossible to avoid noticing.  There are as many ways to use data to avoid accountability as there are to use data to provide accountability.  How many times have you had to argue a medical bill, say, with someone whose only reply is "that's what the computer says" even if the charge is totally illogical.

In the education case, they are relying on data-worship to hide the ugly reality of union-busting.

sTiVo's rule: Just because YOU "wouldn't put it past 'em" doesn't prove that THEY did it.


[ Parent ]
Shorter sTiVo: The Problem Is Confusing Data With Lore (0.00 / 0)
No?

"You know what they say -- those of us who fail history... doomed to repeat it in summer school." -- Buffy The Vampire Slayer, Season 6, Episode 3

[ Parent ]
Not quite sure what you are getting at here (0.00 / 0)
but it sounds reasonable.

sTiVo's rule: Just because YOU "wouldn't put it past 'em" doesn't prove that THEY did it.

[ Parent ]
Any number (4.00 / 1)
The problem is that "any number" can be generated more often than not, especially if the data is unreliable or invalid or if some knucklehead in charge of policy wishes to have some half baked data set gain pubic approval and political traction via misplaced causation that fits conventional wisdom.

If teaching is so easy, then by all means get your degree, pass your certification test(s), get your license, and see if you can last longer than the five years in the classroom 50% of those who enter the profession never make it to.

[ Parent ]
exactly (4.00 / 2)
if some knucklehead in charge of policy wishes to have some half baked data set gain pubic approval and political traction via misplaced causation that fits conventional wisdom.

Couldn't have said it better myself.

Or as they say ...

There's data and there's information.  All information is data.  Not all data is information.

sTiVo's rule: Just because YOU "wouldn't put it past 'em" doesn't prove that THEY did it.


[ Parent ]
AND.... (4.00 / 1)
Not all information is knowledge, nor is all knowledge wisdom.

Let Arne test that, if he dares.  


[ Parent ]
No Accountability At The Top (4.00 / 1)
Great post!  

Let's apply the VAM to President Accountability and his merry little boys and girls in the administration:

9.5% unemployment.

19% underemployment.

Both heading higher.

1.6% GDP for QE2.

There will be one more revision for this number, so it could go even lower.

The number of home foreclosures in July actually fell for the first time since 2006, but the number of homeowners behind on payments has increased, meaning foreclosures are likely to increase again in the coming months.

And now the problem is people with prime loans, not people with subprime or Alt-A loans.

See, when people are unemployed or underemployed, they can't pay their mortgages or spend money to buy things.

This would be bad for the overall health of the economy.

But President Data Fetish, on his sixth vacation, seems wholly unconcerned about the situation.

He did have a conference call with his economic advisers about the weak economic data, but they decided NOT to change course on any of their policies.

And of course the financial "reforms" Obama touted as a major accomplishment of his have been a sham.

Banks are engaging in the exact same risky ventures and bets they did before the '08 financial crisis.

So don't look to Wall Street to stabilize this mess.

If Obama was a teacher in the LA school system, he'd see his name in the paper on Sunday under the heading "Bad President."

If the Obama White House were a school under Arne Duncan's rule, it would be closed down and re-opened as a charter school.

And while I am NOT a fan of value-added analysis of teachers or of publicly naming names in the paper in order to shame people to quit their jobs, in Obama's case, and the cases of most of the people who work for him, I think both are warranted.  


[ Parent ]
Have you seen this op-ed (0.00 / 0)
from the New York Times? http://www.nytimes.com/2010/08...

In it, they praise Arne Duncan's Race to the Top for rewarding innovation, which seems to amount to new teacher evaluations for the state of New York.  It's pretty depressing.  


So typical (4.00 / 1)
The MSM continues to refer to "reform" as only those concepts eminating from DC. What balderdash.      

Save Our Schools! March & National Call to Action, July 28-31, 2011 in Washington, DC: http://www.saveourschoolsmarch...

[ Parent ]
Well, Maybe The Bailout Model Will Help Arne, If His Own Model Fails (4.00 / 2)
After all, none of the Wall Street firms could have survived if forced to measure up to everyone else's standards.  So why should Duncan be treated any different?

Is he not one of their own, if not in fact, at least in aspiration?

"You know what they say -- those of us who fail history... doomed to repeat it in summer school." -- Buffy The Vampire Slayer, Season 6, Episode 3


Thank you (4.00 / 6)
I know it's been said before, but I'll say it again. Thank you so much for these phenomenal posts - truly progressive education reform is one of my lifelong commitments. Posts like yours at OpenLeft are rare in the blogosphere, which makes them all the more valuable.

Keep up the amazing work!

Join the fight to give students a real voice on campus: Forstudentpower.org.


I'm truly humbled. (0.00 / 0)
Thanks so much for your kind words.

Save Our Schools! March & National Call to Action, July 28-31, 2011 in Washington, DC: http://www.saveourschoolsmarch...

[ Parent ]
Merit Pay and VAM base on flawed data (4.00 / 4)
Short on time, so here's my thoughts on merit pay as expressed prior:

There's plenty of research showing that punitive incentives or monetary rewards don't work. Offering more money for a certain output level (cough... RT3... cough) can actually reduce performance. Once money is off the table (which for many teachers it largely is) then to elicit highest performance one has to have purpose (which teaching does) and autonomy - the ability to be self directed.

Also regarding data and value added evaluation, teacherken has a post at dKos that details a research from an Economic Policy Institute Briefing Paper on the problems with the use of test scores in teacher evaluation. Besides the fact that the research is pretty clear that "value added" evaluations are dubious at best, the kicker for me was just how unreliable the data was:

One study found that across five large urban districts, among teachers who were ranked in the top 20% of effectiveness in the first year, fewer than a third were in that top group the next year, and another third moved all the way down to the bottom 40%. Another found that teachers' effectiveness ratings in one year could only predict from 4% to 16% of the variation in such ratings in the following year.

The study can be downloaded here (PDF).

If teaching is so easy, then by all means get your degree, pass your certification test(s), get your license, and see if you can last longer than the five years in the classroom 50% of those who enter the profession never make it to.


Great input as always michaeal. (4.00 / 2)
Even the quality guru W. Edward Demings would think that merit pay proposals are total hogwash:
"Duncan and Gates propose developing measures of effectiveness to get rid of bad teachers and increase the pay of good ones. It sounds like common sense. Or does it?
This approach was called the 'inspection' method by W. Edwards Deming, known as the father of the science of quality improvement. Inspection, he wrote, is not an effective way to improve quality because it has no effect on the process that caused suboptimal results in the first place. Real and continuous improvement, Deming argued, occurs only when the workers themselves study outcome variability and the processes that produce it."



Save Our Schools! March & National Call to Action, July 28-31, 2011 in Washington, DC: http://www.saveourschoolsmarch...

[ Parent ]
BTW, Demming Was Opposed To Grades (4.00 / 4)
Demming was a New Deal intellectual. The idea of rightists like Gingrich trying to appropriate his image & brand is totally ludicrous.  Among other things, he was opposed to the entire idea of grading students--at least until they got to college.

He was a profound exponent of the superiority of intrinsic motivation and the inherent desire of people to do good and make a contribution to others.  AND he was a merciless critic of the unintended negative consequences of external motivations.

I haven't read him in close to 15 years, but I remember how he demonstrated the ways that external reward systems screwed up systems--like giving sales bonuses that had people chasing after illusory "high quality performance" that was basically just statistical noise, and ignoring a focus on long-term relationship-building, which is what he argued the sales force should be doing in the first place... in part because that relationship-building would turn their customers into part of their long-term quality-control and quality-impovement operation.

There was zero room in his world for the sort of hucksters who infest the "school reform" world.

"You know what they say -- those of us who fail history... doomed to repeat it in summer school." -- Buffy The Vampire Slayer, Season 6, Episode 3


[ Parent ]
Deming is a very good horse to trot out (0.00 / 0)
for this argument, even moreso because the right-wing has tried to appropriate him, thereby granting him a bit of a halo of legitimacy in Versailles-speak that you or I might not possess.

This bare-knuckles fight is going to need a root-and-branch philosophical component, I think, in addition to political mobilization.  This is why I focus on "data-worship" as well.  

sTiVo's rule: Just because YOU "wouldn't put it past 'em" doesn't prove that THEY did it.


[ Parent ]
Oh, for (4.00 / 3)
crying in a bucket.  The whole No Child Left Behind, and Obama's additions to it, is a bunch of horse manure.  I'm a retired special education teacher, and I have been private-tutoring for many years now.  The entire idea that every child will reach the "goal" of being on grade by 2014 is bogus.  Being "on grade" assumes that you have an average IQ, or better.  Look at the bell curve- half of everyone is "below average."  Where does that leave the developmentally disabled, among others?  I have a neighbor who teaches the profoundly handicapped- these are children who have a developmental age of six or eight months or so (yes, you read that right).  They still have to "sit" for the mandated state tests.  They have no idea what a pencil even is, much less how to fill in a test.  This is meaningful?  I'm not saying that teachers and schools should not be held accountable, but there are better ways of doing this than standardized tests and unrealistic national expectations.  

"We are all born ignorant, but one must work hard to remain stupid."
Benjamin Franklin


It's so nonsensical... (0.00 / 0)
...it's almost as if we're the the third Bush administration.

[ Parent ]
USER MENU

Open Left Campaigns

SEARCH

   

Advanced Search

QUICK HITS
STATE BLOGS
Powered by: SoapBlox