"Like “pivot” and “cloud computing,” “big data” is one of those startup buzzwords that gets thrown around indiscriminately–partly because it means different things depending on the intel you’re trying to unearth and partly because it sounds like the kind of futuristic jargon that opens doors. Using machine learning to analyze big data? We can practically see the pitch deck already!”
I’m going to bite here and say “sort of”. Silver’s analysis is on the very low end of what qualifies as ‘Big Data’, but that’s sort of beside the point I want to make here. As Ezra Klein said, “The greatest trick Nate Silver ever pulled was becoming the face of aggregated polling”. Klein is mostly right in that, while Silver masterfully aggregated a lot of data, he did an equally good job of promoting his work and, in some ways, is the product of being in the right place at the right time. As the Atlantic pointed out, he wasn’t the only person who was accurately predicting the outcome of the election over an extended period of time.
He reminds me a little bit of the psychologist Steve Pinker. Like Silver, Pinker is uber-smart. But the ideas and methodology these guys have are not entirely new. Pinker writes pop-sci books that essentially aggregate existing research. That said, what makes people like Pinker and Silver special is that they take something seemingly mysterious and de-mystify it. That’s a talent very few people have. And You can sense a little bit of bitterness in Klein’s tweets, as though he feels he could have done what Silver did. And he probably could have, except he didn’t.
So yes, we should absolutely, 100% celebrate Silver’s accomplishment, but more because it represents something that reaches beyond just one dude. It’s that data has completed its near total integration into our lives, which is too often misconstrued as a bad thing. And more to the point, it’s that everyday fools like you and I can actually see how it works and know that data is our friend. Even in the face of so much bullshit punditry over the last four years, the data could not be denied. Yes, @fivethirtyeight is a game changer. But Big Data has yet to come of age and I suspect that’s where the revolution lies. And that’s what I think about that.