When Models Are Confused As Reality, We Are Blind

My day job is to help analyze and model biological systems. My intellectual passion is to eventually use computers to help create knowledge that would be impossible to even comprehend otherwise. I’m pretty much as strong of a proponent of using mathematical and computer modeling to make real world decisions as you can find.

But I try my hardest to remember that models are only abstract reflections of reality and need to have their basic premises and output constantly reevaluated. We’re never going to be able to understand reality perfectly, and just because we find some discrepancies doesn’t mean that we should give up and say it’s all a waste, but on the other hand we need to remember not to treat our concepts as reality itself.

This is the big problem I have with the current state economics in general and finance in particular. They act as if their models are flawless, and if reality doesn’t do what the model said, then it was just because of random bad luck. The problem is that this isn’t true. I’m not even talking about on a philosophical level, I mean it is literally accepted that their models are based on so many flawed assumptions — and even the wrong mathematical foundations — that they only “work” in the sense that they explain what happens in the least interesting of times.

Naked Capitalism has a good introductory post about this problem, and how literally the entire fate of the world economy rested on risk models that have been known for decades to be fundamentally flawed. So whenever you hear Greenspan say this is a “once in a century” event or that it was unforeseeable, don’t believe it. My favorite excerpt that Yves quoted in the post:

We selected the six largest daily percentage changes in the Dow Jones Industrial Average during October, and asked the question of how frequent these changes occur assuming that, as is commonly done in finance models, these events are normally distributed. The results are truly astonishing. There were two daily changes of more than 10% during the month. With a standard deviation of daily changes of 1.032% (computed over the period 1971-2008) movements of such a magnitude can occur only once every 73 to 603 trillion billion years. Since our universe, according to most physicists, exists a mere 20 billion years we, finance theorists, would have had to wait for another trillion universes before one such change could be observed. Yet it happened twice during the same month.

[emphasis mine]

Yves points out that the models based on “chance” have the wrong distribution, but to make matters even worse, when financial people made all the same moves they greatly increased the chance of everything collapsing. The models themselves are wrong in how they determine risk for uncorrelated (i.e. random) movements, but with everyone going for the same instruments it makes the assets highly correlated (i.e. a movement in one asset affects other ones) which means the risk was much greater still.

Which gets me back to a more general point about models and economics. Modern economics purports to capture fundamental laws, but in reality it can only (at best) be used to describe relationships that existed during our observation periods. If we actually use our newly acquired knowledge to change policy en masse, then it changes the environment, and therefore our understanding might not be valid anymore. By implementing what we think is best, we change reality — so what is “best” might be different! [This goes for ideologies in general. I think it's impossible to be ideological because even if you get your way then there are new problems that your ideology will be ill equipped to deal with.] This is why I am so skeptical of our current ideas about intervention and all the “back of the envelope” calculations that say we will be alright if only we spend enough money and get the world to cooperate. Not only is it uncertain how accurate the calculations are, but by implementing the policies we will create a situation where the world is so highly correlated that if anything goes wrong anywhere, it could lead to complete collapse far worse than the Great Depression. Even the proponents of such policies admit that the outcome rests greatly on the developing world, and with all the political and social instability that implies.

I wish I could be more optimistic, but the reality is that we have constructed social systems that relied on false premises and there is really very little we can do to prevent things from falling apart. The best we can do is rebuild our concepts in a way that takes advantage of the knowledge we’ve gained over the past few decades, and one that is much more humble. To me this is Obama’s true challenge: it’s not about whether he can implement what we think is best, but whether he can guide us through a fundamental reevaluation of our society in general.

Addendum quote from George Orwell I got off the comments in that Naked Capitalism link that captures my view on many things about our current state:

We are all capable of believing things which we know to be untrue, and then, when we are finally proved wrong, impudently twisting the facts so as to show that we were right. Intellectually, is possible to carry this process for an indefinite time: the only check on it is that sooner or later a false belief bumps up against solid reality, usually on a battlefield.