Last week, possibly to justify the time and money I spend on fantasy baseball (two trips to New York in back-to-back years, for instance), I semi-seriously suggested that poll watchers, journalists, and FiveThirtyEight junkies should play fantasy sports if they want to deal with their addiction more rationally. I never could have predicted that Politico would fall into my trap*:

The New York Times's resident political predictor says President Barack Obama currently has a 74.6 percent chance of winning reelection. It's a prediction that liberals, whose heart rates continue to fluctuate with the release of every new poll, want to take solace in but somehow can't. Sure, this is the guy who correctly predicted the outcome of the 2008 election in 49 of 50 states, but this year's polls suggest a nailbiter.

[snip]

Prediction is the name of Silver's game, the basis for his celebrity. So should Mitt Romney win on Nov. 6, it's difficult to see how people can continue to put faith in the predictions of someone who has never given that candidate anything higher than a 41 percent chance of winning (way back on June 2) and — one week from the election — gives him a one-in-four chance, even as the polls have him almost neck-and-neck with the incumbent.

[snip]

For all the confidence Silver puts in his predictions, he often gives the impression of hedging. Which, given all the variables involved in a presidential election, isn't surprising.

So, so much wrong. But it gets worse.

What matters for Silver is that the president wins and that he ends up with a total number of electoral votes somewhere in the ballpark of whatever Silver predicts on the afternoon of Nov. 6. And even then, you won't know if he actually had a 50.1 percent chance or a 74.6 percent chance of getting there.

I'll make it easy: What FiveThirtyEight is saying right now is that, given current polling, if you ran the election over and over again, Obama would win three out of every four times. This does not mean Obama is going to win. If Nate Silver said "there's a four in six chance you'll roll below a five on this die," or "there's an eight in ten chance Derrick Rose will make this free throw," it wouldn't be controversial. No one would dismiss Silver if Rose missed, but innumeracy in political journalism is powerful.

Comparing something that happens once to the same thing happening over and over again is intuitively confusing. Unfortunately it's even more confusing if you simplify it to "what matters for Silver is that the president wins and that he ends up with a total number of electoral votes somewhere in the ballpark." This is sily because FiveThirtyEight actually charts the probability of electoral votes right there on the page. Maybe that helps visualize Silver's project a bit better:

Imagine it as a roulette wheel. Would you put all your money on Obama winning 330 electoral votes? I wouldn't; given Silver's predictions, there's only a 15 percent chance of it occurring. My gut (it's okay, you can still have instincts) tells me Obama won't. Why? Silver's model gives Obama a 60 percent chance of winning Virginia, but only by a margin of less than one percent of the vote. That's not much, and Virginia is an odd state—it has a conservative Republican governor, but it could easily go Democrat in the next gubernatorial election, and the state has a historical tendency to flip from conservative governors to moderate Democrats. And Silver's "state fundamentals" metric for Virginia (ideology, fundraising, etc) has it evenly split between Republican and Democrat. 

And Silver's numbers are based on polling—that's it. The difference between Silver and any idiot with a calculator with a functioning divide-by button is that his math skills have allowed him to develop a sophisticated methodology that weights the quality of polls by things like past performance: "each survey is rated based on the past accuracy of 'horse race' polls commissioned by the polling firm in elections from 1998 to the present." But they're still just polls: measures of what someone told a pollster they will do at a point in the future, not what they will do. A good GOTV effort could get more Obama voters to the polls; a devastating attack ad aimed at a niche the Romney campaign discovered in their research could convince them to stay at home. In a highly competitive state like Virginia, electoral strategy could swing a difference of less than one percent of the vote.

(For entertainment purposes only: As a Virginian, I feel instinctually jumpy about Obama winning. Maybe it's superstition, maybe it's like a dog sensing an earthquake. Either way, I'd put money on 300—what Silver's current forecast actually predicts—and maybe hedge it on 330.)

It doesn't account for any of the above, just like no one could have accounted for the ballot design issue in Florida in 2000. That's the difference between prediction and probability: the latter is a realistic way of looking at uncertainty, the former is… not.

What does that have to do with baseball? That's where Silver cut his teeth, and it's a wonderful playground for probability. Take the White Sox: I thought they'd be pretty good this year, better than last year, because last year Adam Dunn and Alex Rios had historically awful seasons, and I didn't think it was probable that they'd be that bad again. And they weren't. But I didn't "predict" that; I considered that Dunn, in particular, had a consistent track record (past performance), and barring undisclosed injury (known information), had a good chance of regressing to the mean, assuming the mean was somewhat worse than past performance due to the effects of aging. And knowing a bit about how individual player performance effects run scoring, I figured it could swing the Sox a few games in the other direction. (Rios I thought was more likely than not to rebound, but was way less confident of it, since his past performance is uneven.)

I didn't think they'd be competitive for first place, but then I thought the Red Sox would not be as bad as they were, causing them to start a rebuilding process, freeing up the White Sox to trade for Kevin Youkilis:

Having Youkilis on the team just for half of a season probably has given the White Sox at least one more win in the standings than they would have had with the otherwise. That may not seem like much, but that is actually a pretty big short-term upgrade compared to most mid-season trades. Sure, the White Sox are up by three games now, so Youkilis has not been the sole difference. Their lead also might get bigger in last few days of the season (it might also shrink). However, although it is clearly silly to suggest that Kevin Youkilis has been the Most Valuable Player in the American League in 2012, that trade does have a claim to being best short-term trade of the season.

And the White Sox were able to acquire Youkilis at a worthwhile price because Youk clashed with his manager, the sort of unpredictable—or at least unquantifiable—result that WAR and PECOTA and all the other stats, some pioneered by Silver and his peers, can't capture. PECOTA can tell us the probabilities of what to expect from Youkilis; WAR can give us an estimation of what we got. Neither could tell us he'd switch Sox in midseason.

This is why it surprises me when pundits get threatened by math geeks like Silver. There's still a big place for reporters and pundits, people who report and read and analyize unquantifiable information. Silver's work tells us a lot about the birds-eye-view that reporters alone can't capture. But it's just probability; it's not truth.

* Could I have said "someone will, and the odds that it will be someone from Politico are good? That, yes. Predicted, no.