• Print
  • Default text size A
  • Larger text size A
  • Largest text size A

Understanding rating methods

  • Wiley Global Finance WILEY GLOBAL FINANCE
  • Exchange-Traded Funds
  • Facebook.
  • Twitter.
  • LinkedIn.
  • Google Plus
Please enter a valid e-mail address
Please enter a valid e-mail address
Important legal information about the e-mail you will be sending. By using this service, you agree to input your real e-mail address and only send it to people you know. It is a violation of law in some jurisdictions to falsely identify yourself in an e-mail. All information you provide will be used by Fidelity solely for the purpose of sending the e-mail on your behalf.The subject line of the e-mail you send will be "Fidelity.com: "

Your e-mail has been sent.

Here we consider fund ratings and rankings used to evaluate and select funds. The most widely cited analyses and ratings of ETFs are published by firms that also publish mutual fund information. It is not surprising that their ETF evaluations/ratings are usually based on some of the same criteria as their mutual fund ratings, but there are at least five important differences between typical ETF and mutual fund evaluations.

First, ETF comparisons tend to focus much more heavily on fund expense ratios than most mutual fund evaluations do though index ETF expense ratios vary less from fund to fund than mutual fund expense ratios. The emphasis on ETF expense ratios is partly the result of the availability of this metric. The expense ratio is one of the first things an investor sees when she examines an ETF’s “Fact Sheet.” The cost to trade ETF shares is important for a number of reasons, but will rarely be a make-or-break item for a long-term investor (though it will matter to the short term trader).

A second difference between mutual fund and ETF evaluation is that, as ETF trading volume has expanded, there has been increased emphasis on comparing the bid/asked spreads in the markets for various ETFs. The message investors receive from the focus on the bid/asked spread is that a narrow spread is better than a wide spread. This point is indisputable but the numbers cited for an ETF’s average spread usually understate the spread an investor will encounter when she checks a live quote. Investors appropriately want to know that a reasonable market exists in the shares of any ETF they might buy.

A third characteristic of ETF evaluation that differs from mutual fund evaluation is that analysts usually give more attention to ETF tracking error—a measure of the relative performance of the fund and its benchmark index—than investors give to this measure. Tracking error gets analysts’ attention because it is relatively easy to calculate, but most fund raters are not sure what to do with it.

Fourth, the approach most fund analysts take in evaluating ETFs and their managers is qualitatively as well as quantitatively different from the approach they take to evaluation of mutual funds. Most ETFs are index funds and index fund managers have yet to capture the imagination of fund analysts and investors in the way that some active portfolio managers have done. It is harder to put a face on an index than on a manager.

Last on the list of differences is a new initiative in ETF comparisons. At least two major fund-rating firms compare ETFs partly on the basis of ratings on the holdings in the fund portfolio. A fund with higher rated portfolio components might get a higher overall rating. The implicit assumption behind this methodology is that the process evaluating the holdings can do a superior job of selecting stocks and bonds. This idea is only as good as the securities evaluation skills behind the holdings rankings.

In its holistic approach Standard & Poor’s has gone beyond the introduction of fundamental holdings analysis to incorporate other data. In addition to incorporating its STARS stock rankings for equity funds and credit analysis, quality assessments, data from a quantitative fair value calculation for most of the securities the fund owns and information on the level of risk the fund has been taking; S&P uses information on the costs associated with purchasing and holding the fund’s shares. The inputs incorporated in S&P’s mutual fund and ETF evaluations are shown in Exhibit 1. Note that the mutual fund and ETF inputs are not identical within any of the three major ranking categories.

Mutual Fund ETF
PERFORMANCE ANALYTICS PERFORMANCE ANALYTICS
S&P STARS S&P STARS
S&P Fair Value S&P Fair Value
3-Year vs Peers S&P Technical
1-Year vs Peers  
RISK CONSIDERATIONS RISK CONSIDERATIONS
S&P Quality Rank S&P Quality Rank
S&P Credit Rating S&P Risk Assessment
Manager Tenure S&P Credit Rating
Sharpe Ratio Standard Deviation
Standard Deviation  
COST FACTORS COST FACTORS
Expense Ratio (Net) Expense Ratio (Gross)
Sales Load Price to NAV
Turnover Bid/Asked Spread

Source: Standard & Poor’s (2009) and MarketScopeAdvisor (January 26, 2010)



The weightings will not necessarily be consistent across even broadly similar funds because the amount of information S&P has will not be consistent from fund to fund. One of the great virtues of S&P’s ETF rankings is that the downloaded version shows the percent coverage of some of the inputs. When data are limited, the category may be omitted or underweighted or eliminated entirely. For instance, many ETFs did not have rankings in January 2010.

The holdings data and the fund classification data S&P get from Lipper are well designed, highly regarded, and the best available for their purposes. S&P’s equity research service is highly regarded; but its STARS coverage is adequate only in North America and only for large- and mid-cap stocks. For the bottom half of the Russell 2000 and smaller companies in the United States and for foreign stocks, S&P has relatively few STARS rankings. For example, on the iShares S&P/TOPIX 150 Index ETF, a fund based on an S&P branded index of 150 of the largest companies in Japan, STARS coverage is only 20 percent.

  • Facebook.
  • Twitter.
  • LinkedIn.
  • Google Plus
Please enter a valid e-mail address
Please enter a valid e-mail address
Important legal information about the e-mail you will be sending. By using this service, you agree to input your real e-mail address and only send it to people you know. It is a violation of law in some jurisdictions to falsely identify yourself in an e-mail. All information you provide will be used by Fidelity solely for the purpose of sending the e-mail on your behalf.The subject line of the e-mail you send will be "Fidelity.com: "

Your e-mail has been sent.
Article copyright 2011 by Gary L. Gastineau. Reprinted and adapted from The Exchange- Traded Funds Manual, 2nd Edition with permission from John Wiley & Sons, Inc. The statements and opinions expressed in this article are those of the author. Fidelity Investments® cannot guarantee the accuracy or completeness of any statements or data. This reprint and the materials delivered with it should not be construed as an offer to sell or a solicitation of an offer to buy shares of any funds mentioned in this reprint.
The data and analysis contained herein are provided "as is" and without warranty of any kind, either expressed or implied. Fidelity is not adopting, making a recommendation for or endorsing any trading or investment strategy or particular security. All opinions expressed herein are subject to change without notice, and you should always obtain current information and perform due diligence before trading. Consider that the provider may modify the methods it uses to evaluate investment opportunities from time to time, that model results may not impute or show the compounded adverse effect of transaction costs or management fees or reflect actual investment results, and that investment models are necessarily constructed with the benefit of hindsight. For this and for many other reasons, model results are not a guarantee of future results. The securities mentioned in this document may not be eligible for sale in some states or countries, nor be suitable for all types of investors; their value and the income they produce may fluctuate and/or be adversely affected by exchange rates, interest rates or other factors.
601154.2.0