Home > Uncategorized > Jesus-Toast and Data-Laziness

Jesus-Toast and Data-Laziness

Once again philosophy of science is bouncing around the econo-blogosphere, and like a moth drawn to a flame I post.

The origin of this latest kerfuffle is the re-launch of Nate Silver’s 538.com.  Specifically, via Noah Smith (via Krugman), Nate Silver writes up his site’s mission statement, and both Noah and Krugman are bothered by it.   I think the point of contention are the steps of Silver’s methodology which he describes as a more scientific way to deal with data/anecdata:

  1. Collection of data or evidence.  I.e. interviews, documents or polls, etc.
  2. Organization of data.  I.e. Chronological, inverted pyramid, or in Silver’s case, descriptive statistics.
  3. Explanation.   I.e. “News analysis”, “explanatory journalism” or for Silver, running statistical tests on the data.
  4. Generalization.  This is the I-understand-something step.  You get the idea.

For me, my only problem with this (since it is an improvement over traditional journalism as typically practiced) is the description of this process as “scientific”.   If Silver views himself as a journalist doing statistics, than more power to him, this seems a reasonable place to start (I would change one thing in his list–I’ll get to that).   But if he views himself as a scientist (statistician) doing journalism, I’ll side with Noah and Krugman.

So, while I agree with the Krugman/Smith view of the scientific method, I’m willing to cut Silver a little slack, since he’s basically a journalist now and journalists have different worries than scientists.  My worry is just this: Jesus Toast.

Jesus Toast:  Seeing patterns where there are none

Have you seen this phenomenon?  People see images (usually religious, tho not always) in the browning on their toast.  It’s like a whole sub-culture and it’s usually described as a kind of miracle, a message from God, or the like.  Now, if you want to believe that God has sent you a message in the form of an image on bread, knock yourself out.   For now, I want to simply categorize the phenomenon… so I call it Jesus Toast.

For the rest of us, I think I can safely say, those images on toast are no different than images in the shapes of clouds: it is the tendency for humans to see patterns that aren’t there.  We are all “associative machines” of a sort, our intelligence is largely built on a foundation of pattern-matching.  So it should be no surprise that when faced with noise, our pattern-seeking brains will try to bring order to that.  This has been demonstrated experimentally (here is some relevant research… not quite what I was looking for, but it discusses the phenomenon).

Nate Silver’s data-laziness

Noah Smith labels Nate Silver’s data-first “theory-less” approach “data-laziness”.   As he points out, the danger with it is Jesus Toast: without an explicit theory, data is interpreted through the lens of implicit theory, such as one’s one biases.   It’s not that “people” see Jesus in their toast, specifically, it is religious people who want to believe God is speaking to them, personally, who see Jesus in their Toast.   It’s an experiment in confirming biases, not upsetting them… not exploring truth or fact.

Krugman picks up a good example already (tho not from Nate Silver, personally):

I feel bad about picking on a young staffer, but I think this piece on corporate cash hoards — which is the site’s inaugural economic analysis — is a good example. The post tells us that the much-cited $2 trillion corporate cash hoard has been revised down by half a trillion dollars… what does this downward revision tell us? We’re told that the “whole narrative” is gone; which narrative? Is the notion that profits are high, but investment remains low, no longer borne out by the data? (I’m pretty sure it’s still true.) What is the model that has been refuted?

As usual, Krugman gets right to the point.   Whose views are actually upset by this?   Well, there are those who think the whole T$2 pot of money is an illusion.   But that pot of money is still there!   Whose model requires that that pot be T$2 or greater, but never a mere T$1.5?  What does that prove?

I think it proves that the writer of the original post wants to believe that there is not corporate cash hoard.  Or perhaps they believe that they are a bulwark against the “misinformation” and incorrect “conventional wisdom”… the facts “everyone” believes, but Aha! they know better, aren’t they smart!

To Nate, here’s a suggestion:

So, Nate, here’s what you might think about, since you’re still refining/experimenting on your methodology.   Take step #4 in your process, and move it to #1.   If you do step #4 last all you will get is Jesus Toast.   Sure, Jesus could have reveled himself in your breakfast, but that’s not a very scientific way to go about things.

As someone who wrote a whole book about how Bayesian inference is great, this should be no big thing.  You start with your priors, and your priors inform the rest of your analysis.   Most importantly, your priors are theory, not data–as the critiques of your book pointed out at the time, this is a major flaw of your discussion there as well.   I’m not entirely certain you always get that distinction.

Let’s see that kind of analysis in action.  Take Krugman’s example with the corporate cash.   If your writer had started his analysis with the prior that the hoard of corporate cash was an illusion, then proceeded to try to explain it away, he’d have found that he couldn’t: there was still a large pile of cash left, too large to be consistent with the “illusion” hypothesis.   He’d then have updated his prior in favor of the corporate cash hypothesis.   Or taking the “conventional wisdom” of a large cash hoard as a prior, he’d have updated that prior (slightly) in favor of the illusion explanation, but he couldn’t reject the hypothesis.

So, maybe the 538 fox knows many things, but he hasn’t learned at least one really big thing that he really ought to understand.

Categories: Uncategorized
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: