Here we consider fund ratings and rankings used to evaluate and select funds. The most widely cited analyses and ratings of ETFs are published by firms that also publish mutual fund information. It is not surprising that their ETF evaluations/ratings are usually based on some of the same criteria as their mutual fund ratings, but there are at least five important differences between typical ETF and mutual fund evaluations.
First, ETF comparisons tend to focus much more heavily on fund expense ratios than most mutual fund evaluations do though index ETF expense ratios vary less from fund to fund than mutual fund expense ratios. The emphasis on ETF expense ratios is partly the result of the availability of this metric. The expense ratio is one of the first things an investor sees when she examines an ETF’s “Fact Sheet.” The cost to trade ETF shares is important for a number of reasons, but will rarely be a make-or-break item for a long-term investor (though it will matter to the short term trader).
A second difference between mutual fund and ETF evaluation is that, as ETF trading volume has expanded, there has been increased emphasis on comparing the bid/asked spreads in the markets for various ETFs. The message investors receive from the focus on the bid/asked spread is that a narrow spread is better than a wide spread. This point is indisputable but the numbers cited for an ETF’s average spread usually understate the spread an investor will encounter when she checks a live quote. Investors appropriately want to know that a reasonable market exists in the shares of any ETF they might buy.
A third characteristic of ETF evaluation that differs from mutual fund evaluation is that analysts usually give more attention to ETF tracking error—a measure of the relative performance of the fund and its benchmark index—than investors give to this measure. Tracking error gets analysts’ attention because it is relatively easy to calculate, but most fund raters are not sure what to do with it.
Fourth, the approach most fund analysts take in evaluating ETFs and their managers is qualitatively as well as quantitatively different from the approach they take to evaluation of mutual funds. Most ETFs are index funds and index fund managers have yet to capture the imagination of fund analysts and investors in the way that some active portfolio managers have done. It is harder to put a face on an index than on a manager.
Last on the list of differences is a new initiative in ETF comparisons. At least two major fund-rating firms compare ETFs partly on the basis of ratings on the holdings in the fund portfolio. A fund with higher rated portfolio components might get a higher overall rating. The implicit assumption behind this methodology is that the process evaluating the holdings can do a superior job of selecting stocks and bonds. This idea is only as good as the securities evaluation skills behind the holdings rankings.
In its holistic approach Standard & Poor’s has gone beyond the introduction of fundamental holdings analysis to incorporate other data. In addition to incorporating its STARS stock rankings for equity funds and credit analysis, quality assessments, data from a quantitative fair value calculation for most of the securities the fund owns and information on the level of risk the fund has been taking; S&P uses information on the costs associated with purchasing and holding the fund’s shares. The inputs incorporated in S&P’s mutual fund and ETF evaluations are shown in Exhibit 1. Note that the mutual fund and ETF inputs are not identical within any of the three major ranking categories.
|PERFORMANCE ANALYTICS||PERFORMANCE ANALYTICS|
|S&P STARS||S&P STARS|
|S&P Fair Value||S&P Fair Value|
|3-Year vs Peers||S&P Technical|
|1-Year vs Peers|
|RISK CONSIDERATIONS||RISK CONSIDERATIONS|
|S&P Quality Rank||S&P Quality Rank|
|S&P Credit Rating||S&P Risk Assessment|
|Manager Tenure||S&P Credit Rating|
|Sharpe Ratio||Standard Deviation|
|COST FACTORS||COST FACTORS|
|Expense Ratio (Net)||Expense Ratio (Gross)|
|Sales Load||Price to NAV|
Source: Standard & Poor’s (2009) and MarketScopeAdvisor (January 26, 2010)
The weightings will not necessarily be consistent across even broadly similar funds because the amount of information S&P has will not be consistent from fund to fund. One of the great virtues of S&P’s ETF rankings is that the downloaded version shows the percent coverage of some of the inputs. When data are limited, the category may be omitted or underweighted or eliminated entirely. For instance, many ETFs did not have rankings in January 2010.
The holdings data and the fund classification data S&P get from Lipper are well designed, highly regarded, and the best available for their purposes. S&P’s equity research service is highly regarded; but its STARS coverage is adequate only in North America and only for large- and mid-cap stocks. For the bottom half of the Russell 2000 and smaller companies in the United States and for foreign stocks, S&P has relatively few STARS rankings. For example, on the iShares S&P/TOPIX 150 Index ETF, a fund based on an S&P branded index of 150 of the largest companies in Japan, STARS coverage is only 20 percent.