Questions Surround Handling of FirstGov

FirstGov.gov-- the government’s first attempt to coordinate electronic information across federal agencies for public use -- has recently experienced a significant increase in the number of users. As a result of the demand for information since September 11, the FirstGov site received 7 million hits during the month of September, up from an average of 1 million hits per month since its beginning. This surge in popularity comes on the heels of recent press allegations that FirstGov’s development has been fraught with government mismanagement, abuse of taxpayer money, possible competitive advantage, and questions of ownership. The FirstGov tale began with a “donation” to the government. In June 2000, Dr. Eric Brewer, co-founder and chief scientist at Inktomi Corp., set up a nonprofit organization, known as Fed-Search, to construct and maintain a governmentwide search engine for the federal government, free of charge. This donation, however, was never meant to be permanent; rather it was to serve as a catalyst, which, through the creation of FirstGov, it has proven to be. In 2003, Fed-Search is scheduled to dissolve, leaving open the question of who will carry on the work of FirstGov’s search engine. In June 2001, the General Services Administration (GSA), which oversees FirstGov, asked for advice on how to proceed with the contracting of a new provider for the search engine. From outside appearances, Inktomi would seem to be at a competitive advantage to receive such a contract as a result of the Fed-Search donation. Fed-Search, with money personally donated by Brewer, contracted with Inktomi, Brewer’s company, to do the programming that powers FirstGov’s search engine -- programming that is proprietary and not owned by the government. If the government were to contract with another vendor, the programming that currently powers the FirstGov search engine would be scrapped. An August article in Government Computer News pointed to this, reporting that “a new search engine vendor would have to start from scratch.” However, David Binetti, who heads Fed-Search, disputes this characterization. He told OMB Watch that Fed-Search has committed to handing over equipment and an instruction manual on how to build the database necessary to power the search engine (minus the Inktomi programming) to whoever wins the contract. With such a road map, he estimated a new search engine could be built in a matter of weeks. All of this leaves a number of questions up in the air as GSA moves to resolve the issue of the FirstGov search engine. Will anything be lost in the (possible) transition from one search engine to another? How did a donation that began with good intentions end up with someone other than the government owning the index of government information? Could Eric Brewer, as co-founder of Inktomi, end up profiting from his own donation? Did the donation, deliberately or not, have strings attached? Will GSA be compelled to contract with Inktomi based on cost, since Inktomi has already set up, and is currently running, the search engine? A Bit of Context In order to make it easier for the public to access government information, President Clinton issued a memorandum in December of 1999 directing the heads of executive departments and agencies to promote access to government information organized not by agency, but by the type of content or service people may be seeking. In response to the memorandum, GSA took the lead in setting up an online portal for indexing and cataloguing links to the vast number of publicly accessible government documents online. The resulting product, FirstGov.gov, was launched on September 22, 2000. FirstGov allows citizens to search more than 47 million U.S. Government web pages by topic, rather than by agency, and allows access to this information 24 hours a day, 7 days a week. It can search 500 million documents in less than a quarter of a second and handles millions of searches a day. In June 2001, FirstGov expanded its search engine to include sites from all 50 states and Washington, DC. For such efforts, the FirstGov site has received numerous awards. In spite of this success, the complexity of the technical side of FirstGov has caused a great deal of confusion and uncertainty, as well as some critical and sometimes inaccurate media surrounding FirstGov. All parties involved have been put on the defensive. There has been a resulting lack of transparency at GSA, which should be the player in this process that reports and answers to the public. However, GSA employees have been reluctant to share details about the process through which FirstGov was created. The Players The goal of getting the site up and running in 90 days, set by president Clinton in his first ever webcast to the nation on June 24, 2000, left no spare time in the undertaking of a project as large as FirstGov. Under pressure to get it done on time and with the extremely low launch budget of $680,000, GSA had to make many quick decisions to get the first government web portal of its kind up and running. GSA went through a formal bidding process and contracted GRC International to build the "front-end," or the user interface of FirstGov, which it designed in only 30 days and continues to maintain. Brewer -- also a professor of computer science at the University of California, Berkeley -- offered to provide the search engine capability to the government for free. Brewer's company, Inktomi, produces search engines, as well as information comparison and directory building tools, among other products. In making such a donation, Brewer stated that he wanted to give something back to the government. Part of his early work as a masters student was funded by a DARPA grant (Defense Advanced Research Projects Agency) -- the central research and development organization for the Department of Defense. That research contributed to the technology he used to co-found Inktomi. In March 2000, Brewer wrote a detailed proposal to the government with his ideas for the future of e-government. After writing this proposal, he set up a nonprofit foundation, Federal Search Foundation (commonly known as Fed-Search) in June 2000 to house the search engine. Fed-Search's mission is "not simply to build a government search engine, but rather to catalyze an Internet-enabled government." Fed-Search houses Inktomi technology to gather information from all public federal government documents posted online in order to promote a more modern, more open, and more efficient government resource -- free and open to the public. The services that Fed-Search is donating are referred to as the "back-end" of the site. Fed-Search provided all of the hardware, software and bandwidth and contracted with Inktomi to secure access to the software and services necessary to build and operate the search engine. But because Brewer wanted Fed-Search to be a catalyst, it was set up to dissolve in 2003, ensuring that the government would then contract with a vendor to take over the management of the search engine. Issues At the Heart of the Matter All parties involved with the building of FirstGov last year agree that things happened very quickly. Because of the very short time-frame, Fed-Search did not go through a formal bidding process -- it used Inktomi's services because Brewer knew exactly what Inktomi had to offer, that it could do the job, and do it in the short time-frame available. Neither Fed-Search nor GSA used the customary bidding process -- one involving competition and public bidding -- for contracting services for FirstGov's construction, which should raise concerns. http://www.autonomy.com">Autonomy Inc., a company that provides relevancy software to sort through the items returned in a search and limit them to those most relevant, was awarded a contract through GRCI, and was paid $700,000 for software that was not used because Fed-Search ended up donating the software. It appears that $700,000 of taxpayer money was paid to Autonomy, Inc., for software that was not used, and in effect wasted, because GSA prematurely awarded the contract. A September 2000 memo of understanding between GSA and Fed-Search specifically states that if "Inktomi or some other party operating under license from Inktomi is the awardee on the successor procurement, the servers and associated hardware at the Fed-Search data center will remain intact and connected to Inktomi. If the awardee on the successor procurement is using a different technology for the search engine, the connections to Inktomi will be disconnected and the servers will be purged of software and data prior to donation to GSA." Upon dissolution, Fed-Search will turn over its servers to the Federal Government -- but not the data on the servers. Upon reading this line in the memo, it sounds as if something will be lost if there is a transition from Inktomi to another vendor. This is not the case according to Binetti, the president and CEO of Fed-Search. Binetti explained to OMB Watch that nothing would be lost because any incoming vendor will have the hardware on which to house the database, and the instruction manual showing how to build it. This manual, which Binetti calls the Critical Instruction Manual, will allow any vendor to rebuild the database so that no one will be starting from scratch. Binetti assures that "every potential vendor starts from the same advanced position." If a new vendor is awarded the successor procurement the database will be destroyed, but nothing will be lost, Binetti explained, because the database is currently reconfigured every two weeks by crawling agency websites. Aside from the procurement issues of FirstGov, some argue that the Fed-Search index of government information is not actually public information -- an issue OMB Watch raised in testimony over a year ago. Industry representatives, such as David LeDuc of the Software and Information Industry Association, object to the fact that the data in the FirstGov index is inaccessible to companies that run other search engines unless those companies pay for it. LeDuc argues that other companies have a right to re-disseminate government information however they choose, and for free. However, Fed-Search asserts that it currently owns the database and index, and does not legally have to give the public access. Currently, other companies may pay Fed-Search to be Certified Partners in order to access the Fed-Search index. If the government owned the index, this sort of practice could imply a breach of the Paperwork Reduction Act (PRA). The PRA explicitly calls for government agencies to "enhance public access to and dissemination of, information, using electronic and other formats" and also directs agencies to "encourage a diversity of public and private sources for information based on government public information," and "in cases in which the agency provides public information maintained in electronic format, provid[e] timely and equitable access to the underlying data (in whole or in part)." The PRA also states that agencies should not (1) "restrict or regulate the use, resale, or re-dissemination of public information by the public"; (2) "charge fees or royalties for resale or re-dissemination of public information," (3) or "establish user fees for public information that exceed the cost of dissemination." It appears, however, that since Fed-Search is the entity charging for access to its index, there is no breach of the PRA. Although the government (or the public) owns the information on its own sites, the index that makes that information easy to search and retrieve clearly does not belong to the public. This brings us back to our initial questions -- if Fed-Search owns the index now, who will own it next year after the procurement process takes place? What implications will Fed-Search's ownership of the current index have on that procurement process? And perhaps most important, why doesn't the government own the index that makes its information searchable and retrievable? Back to Executive Report
back to Blog