ETF ratings and rankings are useful tools to evaluate funds and help us select better actively managed funds and better index funds. But ratings and rankings should be part of a larger framework that will provide more useful absolute and relative evaluations of funds against an appropriate benchmark index fund. A strong framework will permit comparisons of funds across categories to uncover the strengths and weaknesses of an individual fund in detail.
Even if a fund rating calculation considers a fund's ability to do better than its peers during a bear or bull market, the performance measurement that dominates most fund ratings is a single performance number for each fund for each year or some other period. A breakdown of how and why the performance of the fund was achieved in that period is a better guide to what the future might hold for that fund than a simple historic return calculation or longer term comparison of returns among a group of funds. It is essential to look beyond ratings and into the fund for better ways to identify funds with superior prospects.
For instance, ETFs do not need to hold cash balances to meet cash redemptions, and the ETF portfolio manager who wants to manage cash aggressively can be invested down to the fund’s last few dollars each day without unusual cash flow management problems. In fact, however, many ETF portfolio managers do not invest their cash aggressively and show a pattern of returns that suggests the manager consistently holds excess cash. During years when the market rises, these funds lag the performance of their benchmark index because interest on cash balances is not keeping up with the return on positions in the index. The same funds will look too good to be true in a year like 2008 when their cash balances earned modest positive returns while portfolio positions performed poorly.
Fund ratings also explicitly recognize the limitations of past performance as a predictor of future investment results, and then proceed to focus on just such performance comparisons. In the final analysis, the dominant focus of most fund service ratings is on comparisons of a fund’s historic performance to peer group performance. One problem with past performance is that it is a "noisy" measure of the value added or subtracted by a fund's investment process. Effective fund evaluation will attempt to break down the components of performance to separate the wheat (value added elements and costs) from the chaff (noise).
S&P does not star rank funds strictly relative to other funds in their category. This means that a disproportionate number of 5-star equity fund rankings might be in technology, for instance. The fact that the fund rankings in this sector were higher than in other sectors might or might not be a reliable indication that overweighting the sector was a good idea.
A desirable feature of the S&P service is that it is possible to examine and recombine ranking inputs in various ways. For instance, if you are relatively unconcerned about risk characteristics when the market environment is good, you can estimate the effect if you removed risk considerations from the ranking. Obviously you cannot do that with precision, but a fund that was top-rated on everything but risk would move up in rank if a negative risk rating was removed.
The S&P ETF and mutual fund ranking services are delivered through S&P's MarketScope Advisor platform, and may be incorporated in ETF advertisements or websites. You will want to check that the published ranking is fresh. S&P fund rankings can change frequently. In most cases, portfolio updates will be available to S&P monthly and other data items are updated frequently. Investors and advisors should concentrate on the depth of coverage S&P has in each category rather than on overall rankings.
Some of the data available from the fund rating services can be very useful, particularly data organized to answer the kinds of questions an investor or advisor might ask. However, much fund data aggregation and commentary does not explain the data assembly or analytical process.
We buy one fund rather than another because we expect the chosen fund to deliver a better return. Fund ratings are useful tools for analysis and evaluation in combination with a more comprehensive analysis but should not be relied upon solely for an investment decision. A framework that considers index tracking error calculations, fund costs including custodial fees and trading costs, taxes, and value added elements such as cash holdings and fund management will serve objectives more effectively than one-size-fits-all ratings or rankings that cannot adequately consider the diverse objectives of all investors and the widely diverse results achieved by funds investing in different market segments.