Fatally Flawed Stimulus Report Ignores Subrecipients
by Sam Rosen-Amy, 4/2/2010
The great thing about the Recovery Act is that it provides us with a great deal of data on hundreds of billions of dollars of federal spending. Anyone can go to Recovery.gov, the stimulus tracking website, download data for their state or the entire nation, and see each and every report submitted by recipients of the funds -- all 230,000 of them. Sifting through that amount of data can be like drinking from a fire hose, but it's an important feature of any spending transparency system. Anyone can take the data and do their own analysis, greatly expanding the uses of the data.
The downside is that, like the recipient reports, there is no guarantee that every analysis will be high quality. A recent Mercatus Center report that uses the Recovery Act data shows is one such example. The report, written by Veronique de Rugy, purports to show that, instead of going to congressional districts with high unemployment, Recovery Act funds are disproportionally awarded to districts represented by Democratic members of Congress. It says that, according to the fourth quarter recipient reports, Democratic districts received 2.65 times the amount of awards that Republican districts received, a staggering assertion labeled as a "fact." Unfortunately de Rugy's analysis is based on a flawed methodology and a strangely constructed data set.
The primary problem with du Rugy's methodology is that it ignores subrecipient reports. Instead, she chooses to analyze only prime recipient award data. Discarding the subrecipient awards hamstrings the analysis, since it will account for only the places which originally received Recovery Act funding, not where the money actually went. De Rugy argues that focusing on the prime recipients is important from a political economy view, but much to the analysis' detriment, this focus substantially misrepresents the amount of funds each congressional district received and why they received it.
A lot of the Recovery Act spending isn't directly spent by the federal government (for the purposes of this post, we're only talking about the discretionary spending in the Act, not the tax cuts and mandatory spending like Social Security, both of which make up about two-thirds of the Recovery Act spending). Instead, the federal agencies give money to the states, which then distribute the funds to other entities to carry out the work. For instance, to pave a highway, the federal Department of Transportation would give money to the New York Department of Transportation, which would then hire a contractor to do the work. So if you just look at the prime recipients, who are the first people to touch the money after it leaves the federal government, you'll find a lot of state agencies getting large amounts of money. In fact, in the third quarter, 46 out of the top 50 recipients of Recovery Act funds are either states or state agencies.
These state agencies, of course, aren't spread across the state; they're usually clustered in the state capital. Therefore, any examination of just the prime recipients will show large sums of money going to the state capitols, since they are receiving funding which will be distributed across the state to pay teachers, hire police officers, and pave roads. Again, another quick look at the data proves this point: in the third quarter, the eighteen districts with the most Recovery Act funding are all in state capitals.
The fact that these districts are essentially pass-through districts, with Recovery Act funds flowing out to the rest of the state, fatally skews de Rugy's findings. Nate Silver over at 538.com noticed this problem as well, and tried redoing the analysis to correct for these districts. After discarding the districts in state capitals, Silver found that the political disparity fell from a 265 percent difference to only a 31 percent difference, a giant reduction. Silver pointed out that this smaller disparity would likely disappear as well if he controlled for variables such as unemployment, race, poverty level, age, and other demographic variables.
Silver, however, ignores all the money that the state capital districts received but will find its way to other areas of the state. He points out that this handful of districts received some $118 billion, while the other 357 districts received only $48 billion. Redistributing that $118 billion would drastically reshape the spending landscape, and would have an enormous effect on the analysis. Of course, the only way to analyze how that money is distributed is by using the subrecipient report data. These reports will show another level down the chain, and help specify where exactly the money is being spent.
Granted, including the subrecipient reports in the analysis complicates it, since including them will lead to double counting of funds, but there is a remedy for this problem. We here at OMB Watch employ such a remedy in the Recovery section of our online federal spending database, FedSpending.org. Here, we've created a new data field called "Net Amount Retained" that indicates how much money stayed in a particular location or with a particular recipient.
Lastly, de Rugy appears to have used a peculiar data set. Comparing her data set (available here) with one downloaded from Recovery.gov (available here, although to match the report's data set, I removed all third quarter reports as well as subrecipient and vendor reports), I found that de Rugy's data set had about 4,000 fewer recipient reports, and $12 billion less in prime recipient awards. I have no idea why these reports are missing, but their absence invites further questions about the analysis.
Because of these missing prime reports and the absence of subrecipient reports, the de Rugy report is rather flimsy. It's unfortunate that the Recovery Act's rich (albeit flawed in some respects) data set have not gone under other analyses as complex as de Rugy's, and it's a shame that this particular analysis was so deeply flawed. However, much to de Rugy's credit, in a response to Silver's critique, de Rugy said that she will be reworking her numbers to take the state capitals into consideration. I'm looking forward to the next iteration of this report with the hope that it will provide a better understanding of the Recovery Act data.
Image by Flickr user jby1982 used under a Creative Commons license.