Original article appeared in Harvard Business School’s Working Knowledge

What Does an ESG Score Really Say About a Company?

21 JUL 2021|by Kristen Senz

A key gauge in the $30 trillion sustainable investment market provides a murky picture of corporate social responsibility. Research by Anywhere Sikochi and George Serafeim probes the underlying factors.

Receiving more information can clarify the complex, but not when it comes to environmental, social, and governance (ESG) scores.

A recent study shows that the more information a company discloses about its ESG practices, the more rating agencies disagree on how well that company is performing along these dimensions. According to the research, a 10 percent increase in corporate disclosure is associated with a 1.3 to 2 percent increase in ESG score variation among major ratings providers, which all interpret and process disclosures differently.

With more than $30 trillion in sustainable investment capital on the line, the stakes are high for companies and investors. Institutions, such as asset managers, pension funds, and endowments, often rely on ESG ratings to make investment decisions. Divergent scores hurt firms, investors, and markets, the research findings suggest, and these effects appear to be worsening over time.

“IF I SAY THERE WERE 20 PEOPLE OUT OF A MILLION IN MY COMPANY THAT WERE HURT DURING THE YEAR, WOULD YOU SAY 20 IS TOO MANY?”

“People are being sold on money being invested responsibly by using these ratings that nobody really understands,” says Harvard Business School Assistant Professor Anywhere “Siko” Sikochi, who co-authored the paper Why Is Corporate Virtue in the Eye of the Beholder? The Case of ESG Ratings with HBS professor George Serafeim and Dane Christensen of the University of Oregon. “That’s where the danger is in having all these different ratings not being aligned in some way.”

Rating agencies rely heavily on corporate reporting—a company indicating that it has an anti-discrimination policy on the books, for example—in calculating ESG scores. When companies are more specific about the impact of their ESG policies, such as by reporting how many discrimination complaints they faced during the year, these disclosures tend to stir disagreement among the raters, says Sikochi.

“If I say there were 20 people out of a million in my company that were hurt during the year, would you say 20 is too many? Maybe you’d think, 20 is actually pretty good, because it’s 20 out of a million. I think that’s where the issues are,” explains Sikochi. “There is work that needs to be done from the rating side. Companies may not be keen on disclosing some of these outcomes, because this is where most of the disagreement is happening.”

The muddy landscape of ESG scores

ESG disclosures aren’t formally regulated in the United States, but the House of Representatives recently passed a legislation package that would require public companies to disclose quarterly and annual ESG metrics.

Meanwhile, the Securities Exchange Commission (SEC) is considering disclosure rules for climate risk, board diversity, and other issues. Groups like the Sustainability Accounting Standards Board (SASB) have created frameworks that companies can use to guide disclosures in various industries, but adoption is voluntary.

Dozens of ratings providers use corporate disclosures to formulate ESG scores, although four rating agencies—MSCI, Sustainalytics, RepRisk, and new entrant Institutional Shareholder Services (ISS)—currently dominate the market. Each agency relies on its own analysts and algorithms to synthesize disclosures of ESG metrics, such as a company’s carbon emissions, board diversity, or safety policies, into separate environmental, social, and governance scores, which are then consolidated to one ESG score.

The impact of disclosure

Using data on firms from 69 countries from 2004 to 2016, Sikochi, Serafeim, and Christensen analyzed ESG disclosure scores from Bloomberg and ESG performance data from MSCI, Thomson Reuters, and Sustainalytics. To isolate the effect of increasing disclosures on score disagreement, they compared results across countries, where some had adopted broad disclosure mandates.

“We find that after a country or stock exchange implements mandatory ESG disclosure requirements, the affected firms increase their ESG disclosures and experience greater ESG rating disagreement,” the authors wrote.

“WE’D RATHER SEE THAT A COMPANY HAS PLANTED TREES, INSTEAD OF SEEING THAT THEY HAVE POLICIES ABOUT NOT DESTROYING TREES.”

By moving from the 25th to the 75th percentile in terms of ESG disclosure, firms saw the spread between their best and worst ESG scores widen by as much as 31 percent. More disagreement among rating agencies leads to higher short-term stock return volatility and larger stock price swings, the researchers write. Sikochi and his colleagues also found that score disagreement among major ratings providers is growing over time, suggesting an increasing effect on markets.

Focusing on outcomes, not box-ticking

Sikochi says the research findings point to the need for greater debate about ways to regulate disclosure that would make scoring more predictable for firms and useful for markets. He and his co-authors argue that ESG outcomes are a better way to measure success in environmental protection, social responsibility, and corporate governance than by checking off a company’s written policies.

“If they’re being rated on their policies, then companies are just going to have lots of policies in place and disclose those policies, but I don’t think the world will be a better place,” says Sikochi. “We’d rather see that a company has planted trees, instead of seeing that they have policies about not destroying trees.”

In areas where science can bolster best practices, such as with carbon emissions, industry norms have begun to emerge. But for outcomes and specific issues that are inherently more subjective, Sikochi says, ratings agencies and regulators will need to develop a shared understanding to guide disclosures and increase consistency in how the information is used.

The future of ESG scores and disclosure regulation

Just as standardized accounting practices evolved over time, Sikochi says, he expects that a more uniform system for assessing ESG factors will eventually take shape. He thinks such a system might involve credit rating agencies, which he has studied in depth.

“If credit rating agencies get involved with ESG ratings, are they going to take the same frameworks or the same thinking they use with credit ratings and apply those to ESG? When they rate corporations, they provide a formula and a grid, so firms can actually predict what their rating is going to look like,” he says. “I’m wondering if they’re going to do the same thing with ESG.”