OMB Director Peter Orszag released a memo to the heads of executive departments and agencies this morning announcing a new initiative in the FY 2011 budget process that seeks to bring increased emphasis and resources for program evaluation within agencies. The memo outlines three parts to this initial effort:

1) Posting more information about federal evaluations online: OMB will begin working this fall with federal agencies to expand access to information about program evaluations. The goal will be to "make researchers, policymakers, and the general public aware of studies planned or underway" that examine if a program is making the grade or evaluate the effectiveness of other approaches and strategies that achieve the desire outcomes.

This is a great first step for the administration and should encourage agencies to make evaluations and studies used to make decisions open to the public - something that has been a problem at times in the past. Further, it could potentially become as useful as a project at the Department of Health and Human Services that allows the public to search all government funded clinical trials (see ClinicalTrials.gov). While the memo does not explicitly say OMB will develop a central repository for all government program evaluations, allowing for the capacity to search evaluations across agencies and department that focus on a central goal or issue (alleviating homelessness for instance) would allow for better coordination and communication between agencies that run programs with similar objections. Hopefully the resulting online disclosure will be fully searchable and comprehensive.

2) Inter-agency Evaluations Working Group: OMB is also going to re-constitute an inter-agency working group of evaluations experts from the Domestic Policy Council, the National Economic Council, and the Council of Economic Advisers. This working group will help build agency evaluation capacity by developing a network of experts inside and out of the federal government, share best evaluation practices from across the federal government, develop techniques for using evaluation and performance data to continually drive improvement, and potentially develop government-wide guidance for program evaluation practices.

There are some key points made within the language of this section of the memo that are big improvements over previous performance efforts. First, it is clear that OMB and the Obama administration understand a one-size fits all approach will not succeed. The memo references that different evaluation methods and structures are necessary for different types of programs and objectives the government is trying to achieve and that agencies need flexibility within government wide evaluation guidance to "adopt practices suited to their specific needs." Second, there is an acknowledgment, albeit subtle, that many agencies do not have the resources or in-house expertise to develop a "strong, independent evaluation offices." More on availability of additional resources in a second (see #3 below). Finally, in both the first section of the memo and the second, OMB opens the door to potential collaboration between government and outside experts to design and implement robust program evaluation studies. This could potentially allow those outside the government who are responsible for implementing government programs, as well as experts from academia or other sectors to work more closely in designing evaluations that measure the right type of things.

3) Voluntary FY2011 Evaluation Initiative: The third aspect of the memo is a voluntary program that invites interested agencies to submit additional information along with their FY 2011 budget materials to win additional funding for high-priority evaluation activities. OMB plans to award funding to 20 "rigorous program evaluations across the Federal government or [to agencies] to strengthen agency evaluation capacity." Agencies must submit additional information to OMB that includes an assessment of evidentiary support for budget priorities, new proposals for rigorous evaluations, an assessment of agency capacity to conduct rigorous independent evaluations, and the identification of statutory impediments.

This third section seems kind of like a pilot program since it is being implemented in a limited way initially. Only 20 awards will be made to agencies and the scope of performance measurement activities this will support is restricted to "social, educational, economic, and similar programs" that support life outcomes of Americans. The memo specifically mentions that four very large areas - procurement, construction, taxation, and national defense - will not be considered except on a case-by-case basis.

This limited scope is not necessarily a bad thing - nor is the fact the initiative will begin as voluntary for agencies. In addition, this section, and the memo overall has a more collaborative tone than previous performance improvement efforts and gives agencies more of a central role in developing their own evaluation systems. These are all positive developments. Yet in order to help assess the impact of the pilot, I hope OMB will make all the information from agencies that volunteer for this initiative, as well as the 20 awards OMB decides on available on the Internet.

Bottom Line:
This memo is not a comprehensive plan to replace the PART, but it does seem to hint at the beginnings of that process while seeking to correct many of the mistakes made in the past. Orszag gives a little more insight about the scope and goals of this initiative in a blog post that describes how this initiative will be used by OMB down the road:

This is a first step. The agencies participating in this initial effort will serve as demonstration projects through which we can test approaches to improve program effectiveness and efficiency, share best practices, and further improve performance. After assessing this initiative in FY2011, the Administration will be better positioned to implement government-wide evaluation metrics.