Comments on OMB's Draft Report on Regulatory Accounting

Mr. John Morrall III Office of Information and Regulatory Affairs NEOB, Room 10235 1725 17th Street, N.W. Washington, D.C. 20503 Dear Mr. Morrall: On behalf of Citizens for Sensible Safeguards, a broad-based coalition of consumer, labor, environmental, and other public interest groups, I am writing to provide comments on OMB's draft report to Congress on the costs and benefits of federal regulations. As you know, we strongly oppose the idea of regulatory accounting. Rulemaking decisions are made on a case-by-case basis, as they must be, and throwing all of the government's diverse regulations, from environmental standards to economic controls, into the same pot has little practical utility for public policy. Even worse, because of the vast analytical uncertainties involved in such an exercise, and the limitations of monetization generally, it is more likely to create confusion than to shed light on the regulatory process. Indeed, OMB's two completed reports, as well as the draft report, tend to back this up. "We still believe that the limitations of these estimates for use in making recommendations about reforming or eliminating regulatory programs are severe," OMB states in its second report. "Aggregate estimates of the costs and benefits offer little guidance on how to improve the efficiency, effectiveness, or soundness of the existing body of regulations." Yet despite such warnings, the Administration has, for the second straight year, included the regulatory accounting rider as part of its budget. This seems impossible to reconcile with OMB's statements. If the Administration truly believes what it is saying in OMB's reports, then regulatory accounting should be abandoned. Such an aggregate study is simply not a wise use of executive branch resources. As for the specifics of the draft report, we believe it is flawed in at least three significant ways: First, OMB does not adequately explain the various assumptions used in monetizing benefits. When presenting monetized figures, it is crucial to explain the assumptions used in arriving at those figures. Indeed, OMB's own Best Practices document makes this clear: "Where benefit or cost estimates are heavily dependent on certain assumptions, it is essential to identify these assumptions explicitly." OMB has explained how it arrived at its aggregate numbers, identifying the various studies and RIAs employed. But OMB has never explained the assumptions that underlie cost-benefit analysis for individual rules, even though OMB itself has gone back to monetize agency RIAs. This is especially important when dealing with such controversial assumptions as discounting for future lives saved and willingness-to-pay, which deflate benefits. Second, OMB should not rely on the dated Hahn-Hird estimates. OMB should abandon its use of the Hahn-Hird study, along with Hahn's 1996 update. Although published in the 1990s, these estimates are actually based on much earlier work, and include a number of gaping holes. In fact, the Hahn-Hird study misses some of the most significant environmental benefits of the last 20 years. By failing to capture this data -- and by failing to include benefit estimates for five of the seven areas for which they estimated costs -- the Hahn-Hird study understates benefits, and, as a result, so does OMB's report. Third, OMB understates the benefits of federal regulation. OMB's monetization of benefits coupled with the use of the flawed Hahn-Hird data, and the limitations of OMB's report generally (for instance, its failure to capture adaptive effects), have the effect of significantly deflating "net benefits." Moreover, the presentation of "net benefits" (benefits minus the costs) is misleading, as it excludes nonmonentary benefits -- which, at least rhetorically, OMB says are important. Explaining Monetization Despite society's ideal of protecting all members of society, including children, from the array of health, safety, and environmental threats, we recognize that society's resources are limited, that priorities for actions to protect health must be set, and that evaluation of benefits plays a key role in that process. We also recognize that there are acknowledged limitations in available methodologies (i.e., willingness-to-pay and value of statistical life) for monetizing nonmonetary benefits. In our view, these limitations are so acute that monetizing health benefits can make economic analyses less useful to decision-makers, and less transparent to the public, than analyses that simply describe nonmonetary benefits (quantitatively to the extent feasible). Nonetheless, if OMB is going to monetize benefits, it must explain the methodologies and assumptions, both implicit and explicit, that underlie its estimates. Specifically, we are most concerned that OMB has provided no explanation of willingness-to-pay methodology and the practice of discounting for future lives saved: Willingness-to-Pay. The use of willingness-to-pay methods may be profoundly affected by socioeconomic characteristics. Specifically, lower-income individuals may report lower willingness-to-pay, not because they actually place a lower value on a particular benefit, but because they are implicitly trading off such payments against basic necessities. We believe OMB should generally avoid basing willingness-to-pay functions on highly constrained respondents. Indeed, OMB should acknowledge in its report that all WTP values are based on income constraints, and that utilization of the WTP methodology implicitly involves a major policy choice on the "correct" level of wealth. Before promoting WTP based on generally accepted monetization of nonmonetary benefits, OMB should evaluate which, if any, willingness-to-pay estimates are indeed "generally accepted," and how such acceptance developed. Willingness-to-avoid methodologies should also be discussed in the report. Willingness-to-avoid is assessed by asking respondents how much money they would have to be paid in order to accept a particular risk -- for example, increased risk of harm to their children's health resulting from an environmental factor. While in theory willingness-to-pay and willingness-to-avoid should be the same, WTA is generally higher than WTP by a factor of two to ten or more. We believe it is significant that, in the research conducted to date, there is no sum of money that parents are willing to accept for an increased risk to their children's health. Discounting. One disturbing element of the draft report is its discounting of future lives saved. At least in some instances, lives saved are first monetized and then a discount factor is apparently applied to the monetized value -- a process that OMB does not explain. In response to our comments from last year, OMB stated, "Discounting is a generally agreed practice in the economics profession and required by the Best Practices document and an OMB circular." But this does not answer our chief complaint: that OMB has failed to provide transparency and context to its estimates, as in fact, its own Best Practices document requires. The report should specify the discount rates and value-of-life that OMB is using, and there should be an acknowledgment and discussion of the associated policy implications, which are enormous. For example, take OSHA's lockout/tagout regulation. OMB estimated that it costs $70.9 billion for each premature death it prevents, while OSHA, which did not discount future benefits, estimated this cost as between $190,000 and $1.2 million. Yet despite the enormous impact that discounting can have on estimates, OMB does not point out that it is discounting in its report -- as it does for OSHA's methylene chloride rule, for example, which is misleadingly labeled an "agency estimate" even though OSHA nowhere adopts a cost-per-life-saved. OMB's assumptions should be expressly articulated, rather than left for the reader of the report to try to untangle, particularly considering the immense policy implications for regulatory decision-making. The combination of monetizing and then discounting benefits means that regulations are virtually pre-ordained to appear cost-ineffective. But this result is not grounded in reality; it is merely the mathematical consequence of OMB's assumptions, many of which are at best open to fundamental question. Indeed, Congress -- through the Occupational Safety and Health Act, the Clean Air Act, and many other laws -- has committed us to engage in long-term protection, despite the implications of discounting. Unlike dollars, lives cannot be put in a bank account in which they earn compound interest. There is no a priori reason to believe that people facing imminent death from chronic disease in 2020 will be any less dismayed by that prospect than people facing such a fate today. Indeed, insofar as the future lives saved are those of people who are children in 1998, the opposite is true -- people often value their children's well-being more highly than their own. Presumably, this attitude would extend to assuring that their children, upon reaching adulthood, will not contract chronic diseases that could have been prevented by expenditures today. Further, while the argument can be made that discounting is justifiable because any rational individual would prefer to save a life today rather than tomorrow, it must be noted that regulatory expenditures simply do not compete directly against alternative ways of saving lives. Put another way, money saved by foregoing regulatory safeguards is by no means guaranteed to end up being spent on other life-protecting outcomes. It is simply false to imply that there is a direct trade-off between environmental protections and, for example, better emergency response services. OMB discusses none of this. The Hahn-Hird Estimates OMB relies heavily on the Hahn-Hird estimates. But these estimates have significant problems, many of which OMB acknowledges. In fact, they are so significant that OMB should throw them out all together.
  • First, although published in 1990, the Hahn-Hird estimates are based on much earlier studies by various analysts. Such dated analysis tends to deflate net benefits, as it misses adaptive effects that occur over time.
  • Second, the Hahn-Hird estimates cover only a limited scope of regulation. Because of data limitations, Hahn-Hird did not account for any regulatory benefits in five of the seven areas for which they recorded costs (in the case of OSHA regulations, they assumed negligible benefits, which is hard to sustain in light of current evidence, based on prior studies). Assuming these areas produced positive benefits, the Hahn-Hird study understates the aggregate benefits of regulation, as does OMB's report.
  • And third, even for the areas Hahn-Hird covered, significant holes in the data remain. For instance, they did not capture the enormous health benefits associated with reduction in levels of airborne lead, or the growing body of work that led to the recent regulation of fine particulates.
In 1996, Hahn updated the 1990 analysis, but many of the same weaknesses remain. Moreover, Hahn's estimates were not based on the government's numbers, despite the title of his report (Regulatory Reform: What Do The Government's Numbers Tell Us?). Instead, where an agency refused to monetize for the value of a life, Hahn employed his own preferred measurement. More startling, Hahn completely threw out two rules in which he did not agree with EPA's conclusions. These rules showed substantial net benefits, but in a footnote, Hahn weakly argued that we were likely to cure the type of cancer the two rules addressed in the next 20 years, and so the benefits were likely to be zero. This drastically shifted the aggregate results. Yet despite these glaring problems, OMB employs the ratio of benefits to costs derived from Hahn's 1996 update to calculate the benefits for regulations between 1987 and 1994. For example, to come up with benefits for environmental regulation from this time period, OMB multiplied its cost estimate by 1.4, Hahn's benefit-cost ratio for environmental regulation. OMB acknowledges this is crude -- and this would be true even if the Hahn's data was reliable. But the data is fraught with problems, and significantly understates benefits, as explained above. For this reason, it should not be used to calculate benefits for regulations between 1987-1994. OMB says it knows of no other way to calculate benefits over this time period. We're not sure either, but using the Hahn-Hird data in such a way shouldn't even be an alternative. It is absurd that OMB is willing to use the Hahn-Hird data in such a way, yet will not incorporate data from EPA's new study, The Benefits and Costs of the Clean Air Act 1990 to 2010, which is the product of years of work, millions of dollars, and intense independent peer review of both the economics and science. In fact, Freeman, whose data Hahn and Hird heavily relied, was one of the peer reviewers. OMB's explanation for why it won't include the study is insufficient. Surely, if OMB wanted to, it could make the necessary adjustments to incorporate the data. It seems more likely that OMB is unwilling to use the study because it is uncomfortable with its conclusions. If this is the case, it is likely that no level of rigor will ever be sufficient to offset such bias, and the process of assessing costs and benefits in such a way cannot be useful. Understating Benefits As explained above, benefits in OMB's report are deflated in a number of specific ways related to its methodology. But apart from these specifics, the very nature of a monetized, aggregate cost-benefit analysis -- whatever the methodology employed -- is likely to understate the benefits of regulation for several reasons:
  • First, many benefits simply cannot be monetized. Agencies often evaluate benefits using qualitative or quantitative factors that are difficult to monetize. For example, as OMB briefly notes, equity considerations are not monetized, but are often of paramount importance in regulatory decision-making. OMB should also discuss the difficulty in monetizing other benefits, particularly the value of reducing non-fatal illnesses or injuries. Costs, on the other hand, are more easily stated in monetary terms. This analytical discrepancy is only accentuated when you attempt to monetarily add up all federal regulation at once and can produce numbers that are greatly misleading.
  • Second, using past RIAs to estimate today's regulatory impacts, as OMB does, deflates net benefits. As OMB points out, industry learns to adapt to regulation and reduce costs over time through technological advancements, "learning by doing," and other factors. The RIAs used by OMB -- some more than a decade old -- were conducted before any adaptive effects could take hold, and as a result are likely to overstate costs. OMB has stated that adaptive effects can also deflate benefits. But while this may be true in some cases, the available evidence suggests greater reductions in costs than benefits over time. Thus, not taking adaptive effects into account is likely to reduce "net benefits" in an aggregate study.
OMB does acknowledge some of these deficiencies. Yet in presenting its monetized figures, OMB includes the category of "net benefits" in Table 3, calculating benefits minus the costs. While we appreciate the note indicating that this does not include nonmonetary benefits, we believe the data should not be presented in this way. OMB can present monetized costs and monetized benefits, but given that many benefits are not captured, or are understated, by monetization, the presentation of "net benefits" is likely to be misleading. Monetized benefits and costs alone cannot provide the "net benefits" of regulation. Conclusion After all of OMB's statements about uncertainty and the various methodological leaps of faith -- the use of the Hahn benefit-cost ratios in particular -- it seems more clear to us than ever that regulatory accounting is not a worthwhile endeavor. Unfortunately, it does not seem to be as obvious to Congress. The House has already passed regulatory accounting legislation (H.R. 1074), and Sen. Fred Thompson (R-TN) is pushing his own version (S. 59) as well. The Administration has rightly opposed these bills. Yet the regulatory accounting rider is still included in the proposed budget. This is not helpful. It gives tacit endorsement to the idea of regulatory accounting, and only encourages those in Congress to move forward with free-standing legislation. The Administration should stand by the verdict of OMB's reports: "It is difficult, if not impossible, to estimate the actual total costs and benefits of all existing Federal regulations with accuracy." Given this, OMB should recommend that Congress drop the regulatory accounting requirement no matter what form it takes. It is misleading, places too great an emphasis on monetization, and is of no value to policy-making. Sincerely, Gary D. Bass Chairman, Citizens for Sensible Safeguards
back to Blog