09/08/2012
I harp on one key issue in philosophy of science a lot because I get a lot of backtalk along the lines of: Everybody knows that the social sciences are a fraud. They can’t predict whether the stock market will go up or down tomorrow, so how you can say that social science data suggests that letting in a bunch of unskilled illegal immigrants today will lead, all else being equal, to lower school test scores later when their kids get into school? If smart rich guys can’t predict the stock market tomorrow, how can an evil nobody like you predict school test scores in a decade? Nobody can predict anything!
Nate Silver writes an article, The Weather Man Is Not a Moron, about how weather forecasting has improved dramatically, which it has. The forecast on the evening news is much more accurate than when I was a boy.
In 2008, Chris Anderson, the editor of Wired magazine, wrote optimistically of the era of Big Data. So voluminous were our databases and so powerful were our computers, he claimed, that there was no longer much need for theory, or even the scientific method. At the time, it was hard to disagree.
But if prediction is the truest way to put our information to the test, we have not scored well. In November 2007, economists in the Survey of Professional Forecasters — examining some 45,000 economic-data series — foresaw less than a 1-in-500 chance of an economic meltdown as severe as the one that would begin one month later. …
The one area in which our predictions are making extraordinary progress, however, is perhaps the most unlikely field [weather].
But this dichotomy between market forecasting and weather forecasting shouldn’t be all that surprising if you keep in mind that there is, theoretically, a fundamental difference between forecasting events that respond to forecasts (e.g., the stock market) v. forecasting events that don’t respond to forecasts (e.g., the weather). Theoretically, the former is resistant to improvement while the latter is not. Improving the weather forecasts is hard in an absolute sense, but the projects lacks the special kind of futility that weather forecasting does.
Hurricanes don’t respond to better forecasts by sitting down together and hashing out more sophisticated ways to fool weathermen.
In contrast, say you come up with a better way to predict whether the stock market will go up or down tomorrow. After awhile, your competitors in the stock market forecasting game will notice you are now riding around in a G6 and they will start trying to reverse engineer your method, or hire away one of your employees, or rifle through your trash. Eventually, your method will be widely enough known that the stock market won’t go down tomorrow when your method says it will, because it will go down todaybecause everybody who is anybody is already anticipating the decline that your system predicts. So, after awhile, your system will be so widely used it will be useless.
Financial economists call this the Efficient-Market Hypothesis. This does not mean that markets are more efficient than government at achieving various goals. It means that unless you have inside information, it’s really hard to beat the stock market in the long run because others will adopt your forecasting tool.
The name Efficient-Market is most unfortunate because it’s referring to the speed at which information is incorporated into forecasts, but is woozy on the accuracy of interpretation of the forecast. A phrase like Agile-Market Hypothesis might have been better.
For example, if the headline in the Wall Street Journal tomorrow morning is "iPhone Causes Brain Tumors," you won’t beat the market by sauntering in and selling your Apple stock at noonish. Markets tend to be pretty agile (i.e., efficient) at acting upon new information.
On the other hand, the market’s interpretation of information is often wrong. For example, in the mid-2000s, the news that illegal immigrants were pouring into the exurbs of California, Arizona, Nevada, and Florida to build new houses for subprime borrowers trying desperately to get their children out of school districts overrun by the children of illegal aliens was greeted almost universally as Positive Economic News. What could possibly go wrong?
Heck, a half decade later, this interpretation of What Went Wrong is largely verboten. If you read Michael Lewis’s The Big Short
carefully, yeah, you can kind of pick it up if you have an evil mind. But can you imagine a speaker at either party’s convention saying what I just said?
Is the Efficient-Markets hypothesis true? One obvious problem with it is that the Forbes 400 is full of zillionaires who beat the market long enough to make the Forbes 400. Were they just lucky? Or is the Efficient Markets Hypothesis wrong? Perhaps you can make so much money in the short run from identifying a major inefficiency, such as the recent subprime unpleasantness, that you can wind up very rich if you have the humility to then retire from placing such big bets?
Or, could it be that the Efficient-Market Hypothesis is right, and a lot of the market beaters beat the market the old fashioned way: by insider trading?
About a half decade ago, there was a lot of publicity about what enormous ROIs the endowment managers at Yale and Harvard were generating. When I looked into it, there was a correlation between endowment ROI and how hard it was to get into that college. For example, Cornell had the worst ROI in the Ivy League. I hypothesized that maybe this pattern could be explained by asking: "What would you risk to get your kid into Harvard v. What would you risk to get your kid into Cornell?" This theory was extremely unpopular, so forget I ever mentioned it.
In any case, the Efficient-Market Hypothesis embodies the crucial conceptual difference between trying to forecast the behavior of systems that respond to forecasts and those that don’t. There’s no Efficient-Weather Hypothesis. That’s because if you get better at forecasting the weather, you stay better at forecasting the weather.
Potentially, forecasting the performance of the children of new immigrants ought to be hard because the U.S. government should be using feedback from past performance to adjust policy to get the optimal mix. Illegal immigrant drywallers from Guatemala aren’t working out so well in the long run, okay, let fewer of them in. But of course, thinking about this subject is crimethink and even the stats are hatestats.
This is a content archive of VDARE.com, which Letitia James forced off of the Internet using lawfare.