A couple of weekends ago, I read an essay by Roger Lowenstein in the New York Times. In the article, titled “Long-Term Capital: It’s a Short-Term Memory,” Lowenstein made the point that the turmoil in the financial markets is a case of history repeating itself, and that the root cause is reliance on statistical models that underestimate the complexity and volatility of financial markets. Hmm, I thought, that sounds familiar.
Lowenstein suggested that the demise of Bear Stearns earlier this year bore an uncanny resemblance to the demise of Long-Term Capital Management ten years earlier. Both companies were led to the brink of disaster by “complacent trust in financial models” and an unwillingness to admit how overleveraged they were.
According to Lowenstein, Long-Term Capital’s strategy was “grounded in the notion that markets could be modeled. Thus, in August 1998, the hedge fund calculated that its daily “value at risk” — meaning the total it could lose — was only $35 million. Later that month, it dropped $550 million in a day.”
Explaining how the fund’s estimate could have been so far off, he said, “’Risk’ is said to be a function of potential market movement, based on historical market data. But this conceit is false, since history is at best an imprecise guide.”
Lowenstein’s conclusion is that financial models of risk are far too dependent on the historical status quo and deal with market data that is inherently correlated. When things go wrong, people panic and sell everything, not just affected stocks. The most recent upheaval in the markets due to the news on Lehmann Brothers and AIG only confirms this conclusion.
I mention this essay because it highlights the same problems we encounter when we try to model not risk but return. Econometric sales models work very well as long as the historical status quo is maintained. But when that status quo changes, the models soon become inaccurate. The growing sales predicted for next year might fail to materialize because of competitive actions. Or the stronger returns predicted from investment in search might evaporate when broadcast media are cut back.
All too often we regard the findings from models like these as timeless. They are not. They are merely estimates based on historical data. And just like they say in the stock market, “past performance is no guarantee of future results.” But then, no one seems to listen to that warning, do they?
So maybe what we need to do is include a measure of risk in our ROI models—a measure that not only takes account of how robust the model might be, but also includes the risk that the status quo will change.
We do this when forecasting the future value of a brand in the BrandZ Top 100 Ranking by taking account of the brand’s category and regional growth prospects and associated risks. Importantly, and unlike others, the BrandZ ranking takes account of consumer loyalty to a brand—that is, the risk that consumers might abandon the brand for another. Surely we can apply similar logic to ROI modeling?
The trouble is that doing so might require us to leave the comfortable world of statistics and make a judgment on how risky our assessment might be. The fact is, there are risk indicators inherent in all statistical models, but too often we gloss over them. That’s far easier than stopping to consider how correlated the explanatory data are, the degree and frequency of volatility in that data, and whether we have captured all the likely causal factors.
And then we would have to consider how likely it is that things might change in the future. What is the chance that a major competitor will relaunch their brand? What is the probability that a mega-trend will undermine volume sales in our segment of the market? What is the chance that the new marketing campaign is a bust? These are all serious questions with important implications, but they are not questions that can be easily answered.
So what do you think? Do we overestimate the robustness of statistical sales models? Is my comparison to financial risk modeling valid? And should we take more account of risk when using the results from ROI modeling? Please leave your comment below.
(4 votes, average: 4.75 out of 5)
Loading ...
Email This Post
September 25th, 2008 at 11:59 pm
hi,
yes we do overestimate the robustness of statistical sales models. some calls taken by competitive brands can impact sales and consumer perception. actions like - a key competition wishes to launch a new variant or brand or drop prices on a leading sku, can completely skew the market for all existing brands in the market.
comparisons to financial models may not entirely be justfied as a lot of politicals risks can significantly impact financial models and not necessarily impact sales models. as not all brands are internationally integrated to be impacted by currency or securities movements.
all brand marketing plans need to build in a dooms day scenario - considering situations like - what is the worst that can happen to us tomorrow either on the media scene or customer front.
pro active marketing companies can take the slighest of hints and churn out campaigns at break neck speeds. with mediums like ooh instant visibility for newer brands and ideas is more or less guaranteed.
September 26th, 2008 at 3:53 am
Timely post Nigel
I don’t think the issue arises because of a model’s inaccuracy (they all are), or whether the model accurately reflects the risk (the never can) because by definition they are but one possible numerical outcome. We presume that if the model agrees with actuals we understand the underlying structure - but in truth that result may have been achieved by a whole different process at least in parts of the human system we are trying to model. (you don’t know what you don’t know)
This becomes a more important issue because of the growing interconnectedness that can introduce new elements into the equation without (much) warning. Add to this dynamic - the desire for easy questions, easier answers, even quicker solutions and you have the fertile ground we find ourselves in.
The safety of formula driven management decision making is clearly a double-edged sword, but the most telling factor is the underlying bias of the organization and how its wetware components view the world. Most enterprises are focused on short term performance - and so we get programs that are designed to achieve that. We tend to run into problems when it comes time to better ones comparables because that’s when the underlying bias kicks in. And while each individual action might have a smaller risk component, the cumulative impact is perhaps unavoidable.
We can try to change the formula to reflect the risk - but that does not change the fact that the enterprise might pursue it regardless. Instead I think the more effective approach would be to rethink the underlying bias of the enterprise. I am not saying that we avoid risk, rather that the risk we take have greater long-term benefit.
From my corner of the soapbox - I think the wisest approach is to have a long-term value creation bias. With your indulgence, I invite your readers to visit these posts (Greed is Good, Brand Momentum, The Anatomy of a brand purchase ) for further consideration of what that entails and perhaps how to manage brand performance within a longer-term bias.
cheers
Miro
October 17th, 2008 at 9:53 am
Two thoughts on this Nigel …
Isn’t there something in Aristotle somewhere about the past not being a sufficient guide to the future?
Less facetiously, isn’t part of the role of marketing to try to break the relationships that these models define/describe? Marketeers want their campaigns to deliver better returns, their brands to provide better leverage etc … than their competitors and the past.
There’s something slightly counter-intuitive in assuming that these models will predict the future when the very purpose of creating them is to identify new commercial opportunities.