On May 18, the U.S. Environmental Protection Agency (EPA) became the first agency to release its plan to ensure that federal spending information from the agency is current and of high quality. While the plan details the agency’s current quality procedures, it seems lacking in several areas. Data quality plans for federal spending information were mandated by the Open Government Directive (OGD), but most have not been made public.

The current economic difficulties and the large government programs designed to counter them, including the bank bailout and the Recovery Act, have brought federal spending under increased scrutiny. Transparency of government actions is only one component needed for effective oversight of this spending. Ensuring the quality of the data being made available to the public is another vital aspect. Currently, challenges to data quality include the duplication of data, missing transactions, inaccurate data, and untimely data. These problems and their associated risks vary by agency and depend on variables, such as the types of data a particular agency collects, and thus require agency-specific solutions.

Plan Requirements

The OGD, issued on Dec. 8, 2009, instructed each agency to create a data quality plan that enhances the transparency of how the agency spends federal funds. Office of Management and Budget (OMB) Deputy Director Jeff Zients issued a memorandum on Feb. 8 outlining a timeline for public release of the plans, as well as what information should be contained in the plans. Data quality plans were due to OMB by April 14. OMB was to work with agencies to improve their plans and finalize them by May 14.

Zeints’ memo requires agencies to be accountable for the quality of federal spending information that is publicly disseminated through venues such as USAspending.gov and similar websites. The memo prescribed a data quality framework for the agency plans, requiring them to define measures for assessing the quality of data and its collection, reporting on data quality, identifying risks that could misstate or misrepresent data, and monitoring data quality. Further, the memo requires compliance with the public participation and collaboration elements of the OGD by mandating a communications strategy to engage the public in the solicitation of feedback via websites and social media.

The Zeints memo also requires specific plans to address spending information submitted for inclusion on USAspending.gov and evaluation of information on grants, loans, contracts, and other forms of assistance. Agencies are required to discuss compliance, review, and monitoring of data quality and identify if improvements are necessary. If improvements are needed, the agency is also supposed to establish a timeline and milestones for implementation.

OMB clarified these instructions in a subsequent April 6 memo. The second memo notes that by Oct. 1, USAspending.gov will begin disclosing information about recipient and sub-recipient reports on use of federal awards. Currently, only agency award information has been available through USAspending.gov, so the public has only been able to obtain information about who the prime recipient has been, but not how they have used the money. The new Oct. 1 system is premised on the experience from disclosure under the Recovery Act. This new procedure of recipient and sub-recipient reporting will increase agency responsibility for data quality. Zeints wrote that "the goal is to move toward 100% of awards data being reported on time, complete, and accurate (free of error) by the end of the fourth quarter of FY 2011, with interim milestones." For each metric of timeliness, completeness, and accuracy, agencies are required to achieve a 10 percent improvement each quarter beginning in FY 2011.

EPA’s Plan

EPA is the first agency to publish its plan. EPA’s plan complies with many of OMB's requirements. EPA makes significant effort to explain its governance process and structure. In particular, the agency has developed a Spending Information Quality (SIQ) working group to develop the plan and integrate it into a long-term agency strategy to ensure the integrity of federal spending information. The plan also describes in detail the procedures and systems employed by the agency to address data quality. For instance, the plan explains in depth the existing audit process for the Federal Procurement Data System (FPDS) and includes the results of the latest audit in the plan’s appendix.

However, the plan fails to identify substantial changes needed to comply with the transparency, collaboration, and participation elements of the OGD. The plan appears to be lacking in three main areas: plans to improve accuracy, recipient and sub-recipient reporting, and mechanisms for public participation.

The EPA document provides little detail on how the agency will accomplish the data accuracy goals stated in the OMB memos. EPA reports that the accuracy of key contactor spending data from FPDS is between 86.5 percent and 96.5 percent. While this may seem to be a relatively strong degree of accuracy, it falls short of OMB’s goal of error-free reporting to be met by the last quarter of FY 2011. The agency briefly announces the impending implementation of a new electronic acquisition system that is expected to improve the accuracy of FPDS data. However, EPA does not describe how the features of its new acquisition system will address the known deficiencies.

Additionally, the agency fails to provide any quantifiable metrics for the deficiencies of assistance awards data in the Financial Award Assistance Data System (FAADS). EPA does identify FAADS deficiencies, including missing data, misspelled names, incorrect use of codes, and mistyped numbers from recipient-submitted information. However, the plan provides no proposals to address these issues or any process underway to develop a solution.

EPA also does not present concrete milestones to fix problems that are identified by the agency, as required by OMB guidance. According to the Zeints memo, OMB is to monitor progress through "potential" dashboards that will be publicly available. To do this, the memo states it will require periodic updates to the plans and use portions of the plans to facilitate the measurement of progress. However, it is unknown how OMB will be able to monitor the progress of EPA and other agencies if the plans lack concrete goals and milestones to implement improvements.

EPA’s plan contains little discussion of the likely difficulties in ensuring strong data quality for the newly expanded data from recipients and sub-recipients. The plan does note, however, that current federal acquisition regulations require that agencies recognize the lack of "privity of contract," or a direct relationship, between themselves and sub-contractors. EPA claims that these rules would prevent the agency from placing any requirements, such as reporting on funds received, directly on sub-contractors. The EPA indicates that forthcoming requirements for sub-recipient reporting will be included in future contract language and passed on to sub-contractors in the agreements between them and prime recipients. However, the agency says that the lack of a direct relationship between the agency and sub-contractors will prevent the government from directly confirming submitted data with sub-recipients. How EPA plans to deal with this apparent data quality hurdle is not specified in the plan.

Beyond the brief acknowledgment of the difficulties reviewing or confirming new recipient and sub-recipient data, the EPA plan does not identify any specific expected data quality problems, nor what steps could be taken to minimize and track these problems. The problems with inconsistency in spelling, codes, format, and completion of data fields that the agency noticed in the data submitted to FAADS are likely to also be problems in forthcoming recipient reports.

The EPA plan also does not comply with the OMB guidance on agency communications efforts. EPA states that the agency sought internal "EPA participation." However, the OGD defines participation in terms of public engagement, not simply the engagement of agency employees. The plan makes no mention of public collaboration but only notes that the agency's spending data will be made available to the public by OMB through a dashboard. There is not enough information in the plan to indicate any attempt to involve the public in the data quality process.

There are several ways EPA’s plan could include more participation and collaboration. The agency could commit to receiving outside input for future versions of the plan, such as the agency does with its Open Government Plan. Similarly, EPA could utilize the same IdeaScale collaboration tool used to solicit public input to develop its Open Government Plan. Further, outside participants could provide an independent source of data quality auditing. Many corporations track spending information and may have solutions to data quality problems.

Currently, it is unknown how other agencies will implement OMB guidance, as the plans are not yet public despite the expiration of the May 14 deadline. However, since the plans are approved by OMB prior to release, EPA’s plan may be representative of the scope and content that can be expected from other agencies.