February 20, 2008

Billyball Strikes Out As Educational Model [Updated]

Filed under: Education by Leo Casey @ 11:10 pm

Ed Sector’s Kevin Carey recently authored a Daily News op-ed, What Public Schools Can Learn From Recent Baseball History, which argued that the experience of the Oakland Athletics’ General Manager Billy Beane in using statistics to gauge baseball talent provided a model for education. The use of value added metrics to evaluate the performance of individual teachers, Carey argued, would apply the rational kernel of Beane’s approach to schools.

It now appears that “Billyball,” as its advocates called Beane’s statistical approach, doesn’t have quite the track record of success Carey reported. The most famous account of Beane’s method was Michael Lewis 2004 book Moneyball, which looked closely at Beane’s 2002 draft picks, since the Athletics had accumulated an unusually large number of such picks that season. As New York Times sports columnist Murray Chass recently recounted, the Beane’s 2002 choices chronicled in the Lewis book have proven less than felicitous. Statistically speaking, the other teams which picked based on scouting reports did better than the Athletics.

You can count us among the skeptics that evaluating teachers is a process akin to judging baseball talent. But it is interesting to know that the baseball model being proffered as a basis for judging teaching performance was not even successful on its own terms.

UPDATE:

Not surprisingly, Kevin disagrees with Chass. But he is fighting with a straw argument. Chass does not argue that the 2002 draft picks of Beane and the A’s were failures, but rather that they clearly have not outperformed the 2002 draft picks of teams that used scouting judgments. This is true statistically, as Chass shows, but the difference becomes even more dramatic when one looks at a number of the picks Beane and A’s passed over who went on to become more successful major leaguers than their picks — including one, Scott Kazmir, that the Mets later gave up in their worst trade of recent years. [Sorry, that's the inner Mets fan speaking.] Others such as Prince Fielder, B. J. Upton, Cole Hamels and Jeff Francoeur, will be recognized by avid baseball fans as superior to the A’s picks.

And for what it is worth, the first page of a google search will show that the term “Billyball” is indeed used to describe Beane’s as well as Martin’s tenure at the A’s.

5 Comments »

Comments are open for registered users and do not reflect the views of the UFT. Please read our general rules for commenters.

  • As an avid fantasy baseball enthusiast, it’s an interesting concept. In the world of fantasy baseball, we do look for underlying skill sets as predictive indicators of future success. It’s easy to know that ARod or Wright are likely to have successful 2008’s based on past performance but fantasy baseball winners need to find the next ARod/Wright before they produce the fearsome stats in order to win.

    It’s probably quite true that we need to make more of an effort at all levels to find the next great teachers and find some ways to reward to best among us and find ways to either fix or remove the worst (with probably a good 90% representing the middle).

    With that said, this value added nonsense isn’t the magic bullet that will tell you who will be great in the classroom, most certainly not with the types of tests we now use, the available data, the timing of the tests, the large number of variables impacting the kids (home, administration, multiple teachers, multiple subjects, off days, cheating, private test prep, alternative intervention services, etc.) Baseball, by comparison, is a relatively simplistic exercise with a historic use of complex statistics and even there, it’s as much as art as a science.

    Too bad we don’t put as much effort into what does work and too bad we don’t use the obvious to determine who goes into the classroom: well spoken adults with both training and a willingness to train for a lifetime. Adults who can both write and orally communicate. Adults with a demonstrable love of their subject matter and of children. And a career path for said adults that rewards creativity and success as judged by those who are already in the profession and can make such determinations as well as by parents and students who serve as their customers.

    I wonder how Chancellor Klein would feel if we used gym teachers and fire fighters and baseball players and small business owners to determine certification and evaluation standards for those pursuing law degrees. Sure makes sense to me.

    Comment by paulrubin — February 21, 2008 @ 1:28 pm

  • [...] I’d rather focus on Leo Casey’s response on Edwize, the UFT’s official blog. In “BillyBall Strikes Out as Educational Model,” Casey [...]

    Pingback by When Baseball and Education Meet: Moneyball, the UFT and a Missed Opportunity at www.matthewktabor.com : Education and School Issues, News and Analysis — February 22, 2008 @ 4:22 am

  • Please get your facts straight. Kazmir, Fielder, and Upton were all drafted before the A’s made their first selection, Nick Swisher. Oakland didn’t have the option of selecting any of them. As for the rest, Swisher got to the majors quicker and has established more success there than either Loney or Francoeur. (Hamels may end up being better than all of them - only time will tell.) By the time the A’s made their second pick, all these players were gone. Please also note that the A’s have selected high school players at the start of the draft - they drafted a high school pitcher, Jeremy Bonderman, in the 2001 first round.

    A little investigation into Beane and the A’s would reveal the simple fact that rather than being driven by an adherence to statistics as a means to success, the A’s sought players that were underrated by other ballclubs by ANY standard. This is why the A’s of the past few years transitioned into a team that was primarily driven by pitching and defense, away from the OBP-oriented teams of prior years. Beane has kept the A’s in contention for the last decade through flexible market exploitation, not adherence to narrow ideology. Color me skeptical that public schools will ever be so creative with limited funding.

    Comment by Ars_artis — February 22, 2008 @ 5:47 pm

  • [...] a moment to recall this context, before you read Andrew Tabor’s commentary on the exchange between Kevin Carey and myself over the applicability of the statistical measures of the Oakland [...]

    Pingback by We Are Not Commodities | Edwize — February 25, 2008 @ 12:27 am

  • [...] for him.”Indeed.I mention Moneyball because of a rondelay amongst Kevin Carey, Matthew Tabor, and Leo Casey over the question of whether teachers can or cannot be an undervalued “commodity” in the sense that [...]

    Pingback by kitchen table math, the sequel: Miss A and Moneyball — February 26, 2008 @ 5:19 pm

Leave a comment

You must be logged in to post a comment.