Can human behavior be accurately predicted, or is it too complex and nuanced to quantify and discern through modeling?
Predictive modeling, the process by which a model is created or chosen to try to best predict the probability of an outcome, has lost credibility as a forecasting tool. Overly simplistic models have failed to account for the sheer complexity of human interaction and the degree to which most people behave irrationally. Most predictive economic models presume that people behave rationally most of the time, a premise which is terribly flawed but which serves as the intellectual foundation of many current economic models (See the Wall Street Journal article on this issue, here).
Predictive modeling has seeped into nearly every facet of our decision-making. It has been the cornerstone for most economic models for decades, and lately it has even attempted to predict consumer behavior. Companies like Amazon use predictive modeling to determine what consumers will purchase, and even suggest purchase alternatives based on what other consumers have purchased after reviewing the same item that the consumer initially looked at.
These models are quite good at predicting outcomes provided that the input variables, or human choices, are extremely limited. Thus, Amazon’s ability to predict what a consumer might purchase after having reviewed a particular item on its website is quite good, because the range of inputs and variables necessary to make that prediction is low.
The models, though, are poor predictors of behavior once the range of variable inputs is high. The models also fail to incorporate the nearly endless human capacity to behave irrationally, behavior which cannot be neatly reduced to an algorithmic equation. These factors underpin the failure of most economic models to predict the current housing crisis. Because models are simply mathematical equations and struggle with variability, economists developed them assuming that people would act rationally and make informed choices, and ignored extreme variability. They populated their models with rational people who would calculate the variables and make informed and optimal decisions.
These economic models failed completely in the current economic crisis due to their inability to accurately predict extreme variability. Human behavior, whether economic or personal, is populated by irrational decision-making. Human beings, according to many studies, systematically overestimate or underestimate risk. They also engage in fantasy thinking, placing unreasonable valuations on objects because of the difficulties inherent in their valuation. When engaged in fantasy mode, humans will overvalue anything from Dutch tulips to dot.com stocks to houses. Add to this irrational behavior the daisy-chain of decision making and the confluence of events that occurs in complex economies and it’s no wonder that these models bear no relation to reality and are poor prognosticators of the future.
Another study, by the journal Current Biology (reviewed by the Wall Street Journal, here), concludes that human decision making is even more complex and potentially poorer than originally thought: the study suggests that investors often move in herd-like fashion because conformity feels good. It provides pleasure. This explains why contrarian behavior is often dismissed and ignored.
The study notes that humans consistently seek to reach consensus and agreement with others, and that areas of the brain associated with pleasure react intensely when agreement is reached and others feel identically. This “pack” behavior demonstrates why contrarians are frequently rejected or maligned, and often marginalized to the fringes. Moving with the herd and rationalizing what may be a poor choice supports the notion that humans often behave irrationally and make choices that are clearly against their own self-interest.
Ultimately, these studies suggest that human behavior is not susceptible to easy compartmentalization. Western culture has traditionally struggled with the concept of dualism and the need to interpret behavior along rigid boundaries and remove all ambiguity. Eastern cultures have long accepted the duality inherent in human nature, and have never struggled in living with the muddled middle. As long as predictive models continue to evolve simplistically and along a linear trajectory, they will continue to fail and offer no relief from cyclical periods of boom and bust. If these models cannot be embedded with enough variability to mimic human behavior, then perhaps they should be scrapped as bygone relics of an age where hubris defined our thirst for knowledge.