Yesterday’s blog entry reported on an e-mail that I sent to academic research Wade Pfau on January 16, 2011. He responded later the same day.

Wade’s e-mail further discussed the Fisher and Statman study. He explained that for purposes of the paper he was writing he thought it would be proper to rebalance the tested portfolios each year. He asked whether I viewed annual rebalancing as a violation of the tenets of Valuation-Informed Indexing. My response is set forth below.

Wade:

I agree that rebalancing is the theoretically proper way to do things. I don’t think it is required as a practical matter. But rebalancing is theoretically better and more appropriate.

I’ll tell you the biggest problem that I aware of in testing this. There are four major cycles in the historical record (one producing a major bear in the early 1900s, one in the 1930s, one in the 1970s and one today). Each of the P/E10 levels should be (and my sense is that the data generally shows this to be so) producing worse results during the years following a bull market top than it does in the years leading up to a bull market top. For example, we are now in the low 20s. It was not a bad time to be in the low 20s when we hit the low 20s in the early 1990s. Investing heavily in stocks at the low 20s today is probably going to turn out much worse because we are probably on the way to 7 or 8 in the not too distant future.

The downside of reporting different results for the different ends of a major cycle is that you cut an already too small data set in half. So I don’t say that this should be done. But it does affect the result and not considering which end of the cycle the P/E10 level appears on causes confusion for newcomers who feel that each P/E10 level should dictate a single stock allocation.

John and I (really John — he did all the statistical work, I just asked questions and made suggestions as we went along) did not consider the effect of what stage of the cycle the P/E10 level appeared in when we developed The Stock-Return Predictor (which gives the likely return for each P/E10 level). We developed The Investor’s Scenario Surfer (which lets the investor go through a realistic, random 30-year returns sequence and see whether his choices of valuation-informed allocation levels beats a Buy-and-Hold allocation strategy) later and we did include an adjustment for the stage of the cycle in which the P/E10 level appears in that one. I don’t know how John did it but it might be possible to figure this out by looking at old articles and discussions and e-mails (he stored all his work materials on a Yahoo Finance site that was shut down, causing the materials to be lost!). The calculator forces the user to choose not only a starting-point P/E10 level but whether the P/E10 level is occurring in a “Normal” market or a “Bear” market. John applied some sort of filter so that the return sequences that pop up in the early years of the bear market scenarios are a bit more likely to be bad ones (as is true in the record, of course).

The reason why this happens is not important in terms of the statistical analysis but is highly important in terms of the big picture questions. Bull markets cause massive misallocation of resources (because all investors make bad spending choices, thinking they possess more lasting wealth than they do in fact possess). What’s really going on is that bull markets are causing economic crises (the Buy-and-Hold theory is that it is economic developments that are causing stock price changes, but it appears to be the other way around — stock price changes are causing economic developments). I think that part of the reason why the general phenomenon is not widely appreciated today is that it requires a reversal of thinking on so many different fundamental points.

I am not trying to cause confusion here. I’m just saying this as a follow-up to your point about the late 1990s being the best time for Buy-and-Hold. The core principle here is that all overvaluation is the product of investor self-deception (in a world of rational investors, prices would always be set properly). So the biggest bull markets are going to be at one and the same time the most dangerous time to invest in stocks and the time when theories that downplay valuations are going to become most popular. We couldn’t have had the huge 1990s bull without Buy-and-Hold having first become the dominant theory. This is one of those cases where the observer is causing the results of his experiment by watching things. Buy-and-Holders saw only half of the story and their inability to see the other half caused millions of others not to believe that the other half matters, which caused data to be generated that for a time seemed to vindicate the findings of the Buy-and-Holders.

To become totally objective about this, someone would have to stand outside of bull and bear markets and do the tests. But we are of course always in one or the other or on our way to one or the other. We are on our way to a time when only
studies taking extreme bear positions will be considered in line with the data (because the data of that moment will make stocks look the worst they have ever looked in history).

Anyway, I think it would be a theoretical plus to have an adjustment for whether the P/E10 level occurred in the bull phase of a secular bull/bear cycle or in the bear phase of it. But I have doubts as to whether such an adjustment can be included as a practical matter (because of the limited data set).

Another statistical problem that John encountered is that the data seemed to behave differently pre-1920 and post-1920. The effect of valuations has generally been stronger in the post-1920 years (John has an article where he discusses
reasons what might be the reasons for this). John’s practice was to use data from 1921 forward when possible and to incorporate the earlier data only when it was needed to generate statistically meaningful results (for example, he needed a larger data set when we made 60-year predictions in The Stock-Return Predictor).

Rob