Background About the NPT Project OMB Watch has been involved in a three-year effort called the Nonprofits' Policy and Technology (NPT) Project to provide and identify opportunities for nonprofits to: learn about and utilize newer technologies for public policy activities; improve communication and coordination between technology and public policy professionals in the nonprofit sector; and to increase the accessibility to -- and comfort level with -- these tools. One element of the NPT project is to research and present a number of different models that might work for a "sector-wide" communications system that would allow a wide range of non-profit organizations to disseminate information across the Internet through state-of-the-art tools. These tools would ideally help users to categorize, filter, organize and deliver that information in a timely manner. The result would be to find or develop such tools that would also allow participants in such a system to selectively receive the information based upon their own personal preferences. The NPT Project is examining filtering technologies to address the timely and efficient distribution of policy information to stakeholder groups. One of the main concerns is stemming the potential for "information overload" that is a problem for many organizations. During the course of the project, we have examined a number of different information systems, including commercial software products like PointCast as possible models. One software product that has come to our attention is called BackWeb. Usually used in a corporate "intranet" environment, BackWeb enables an organization to distribute small "teasers" via the Internet. Such teasers usually contain links back to web pages or additional information. The teasers themselves are organized into various categories (called channels) and software that runs on the recipient's PC allows one to select (or filter) content by individual interest as well as to customize how frequently new information is delivered. Users can elect to view content in the form of a scrolling "newsticker," a title bar newsfeed that runs under any open window on the user's desktop, or as a screen saver. BackWeb has been customized by a number of large corporations in very innovative ways -- Compaq, for example, uses BackWeb to automatically distribute software updates to people that purchase their home PCs. Need Content provision and information sharing in the nonprofit sector today is "democratic," in that any individual with telephone, fax, e-mail, or Internet access, can deliver messages of almost any type to anyone at any time. For recipients of those messages, there is a problem. There is not always an easy way to verify the reputation of the information source, the intentions of the sender, the quality and purpose of the actual content, and the ability to respond or take action on the content. Moreover, with regards to e-mail, there is great potential for receiving too much information, too many copies of the same information from different senders who don't coordinate their dissemination, or not receiving the information as all (given incompatibility among systems, accidentally deleting important information from an unfamiliar sender, etc.) Nonprofits are also limited in the types of content that can be easily distributed among organizations, particularly grassroots and small organizations. Many legacy e-mail systems cannot easily accommodate large file attachments, and not all nonprofits have access to the Web for all staff. An overwhelming number of nonprofits utilize dialup Internet connections, yet the quality of the connection may be erratic. Moreover, for those nonprofits which are fortunate enough to maintain advanced information distribution systems, the internal structures are not always conducive to the urgent needs of collaborating organizations and coalition partners. In some cases, the "technology people" become the de facto "content staff", as they increasingly determine how information is distributed, formatted, arranged and gathered. Time sensitive and/or mission–critical information, most frequently "action alerts" regarding important policy activities are currently disseminated via e-mail lists and faxes. However, given the growing ubiquity of e-mail and faxes, recipients sometimes miss important information, because they are not always able to sort out high from low-priority information. This problem stems an ability to capture the recipient's attention in such a way that they actually encourages them to notice the information and take action on it while it holds their attention. All of these factors above limit the range of quality content, and quality information sources available electronically to nonprofits, especially the information distributed by their "opposition." In recognition of these needs, the NPT Demonstration Committee evaluated a copy of the BackWeb Technologies BackWeb Foundation 5.0. The intent was to see if the advanced "push" mechanism model could be applied to nonprofits around a given issue area. Information would be drawn from organizations actively engaged in that issue, and balanced with general nonprofit information. The Demonstration During the summer of 1999, the NPT Project at OMB Watch conducted a demonstration of BackWeb, coordinated through its Demonstration Committee. Given the sophistication of the product, and the limited administrative resources, the decision was made to incorporate the basic system installation necessary to show off the widest range of features to interested organizations, with overwhelming them with options. Because BackWeb is more commonly deployed in corporate intranet environments, we wanted to assess the tool in a context that more applicable to the public policy needs of nonprofits. The demonstration was therefore structured with the assistance of the Invest In America Campaign steering committee (http://www.ombwatch.org/ia) to focus on federal budget issues. The system was set up in consultation with BackWeb staff support, and demonstrated over a period of 45 days. The first 30 days were devoted to internal testing among the NPT Demonstration Committee and OMB Watch staff to learn the workings of the system. An invitation was then extended to a number of outside organizations to evaluate the BackWeb client. Approximately 15-20 testers, representing of range of organization structures, were included in the outside tester group. In some instances, multiple persons within an organization tested a client. Specifically, NPT Project was looking at BackWeb's ability to: (1) coordinate a wide range of information types and sources (2) prioritize information for easy analysis and decision-making (3) target the appropriate information for the appropriate viewers (4) deliver an easy to use product (5) allow senders to disseminate information easily and efficiently Testers were asked to download a copy of the customized BackWeb client, set up the software and allow it to connect periodically to the Internet to retrieve content, and think about how it might be used in a public policy context. Follow-up discussions were then conducted with each tester to determine: · ability to install BackWeb Client · ease and frequency of client use · thoughts on the hierarchy of channels · opinions on the type and format of content delivered BackWeb Initial Feedback Tool It was necessary for us to experiment with a few different clients, before we determined the correct settings. Our testers were able to install the client application easily, due to the small size of the Lite Installer package. The installer program came in handy, as it could "remember" where an installation left off in the event of a system crash, and helped facilitate the upgrading of most existing clients to the version required for the demonstration. A good number of potential testers, however, were not able to participate because there were no updated Macintosh clients available for testing that were compatible with the version of the BackWeb server software we were running. Actual amount of time spent using BackWeb and the frequency with which people checked the tools varied. Most people would check first thing in the morning to see if anything new or of interest arrived, and then actively check 2-4 times a time for about 5 to 10 minutes. Most activity occurred roughly for the first four days, and then dropped off to maybe once a day, unless a "teaser" (flash) were triggered. At least half of the testers experienced a system freeze or crash, or significant drain on resources. This was despite adjustment in the frequency and capacity settings of downloaded information. At least two users on a dialup modem connection found that BackWeb clients consumed all available modem channels, preempting all other download/upload tasks. There was also concern that BackWeb clients preferred the use of the Internet Explorer browser (if it was installed but not active on a user's hard drive) even when Netscape was the user-selected default. Additionally, testers wanted the ability to create their own customized channels, much like a more advanced "bookmark" that could monitor selected websites for new content, and deliver that content through the BackWeb client. The system we tested required users who were running a Windows 9x/NT operating system, with approximately a 6-10 MB of free hard drive space. There were questions as to whether BackWeb would be developed as more of a Web-based tool versus a stand-alone client, or at least a stand-alone client that would ultimately require less disk space, and allow for a wider range of users. Content The content channels for the BackWeb clients were determined by the Invest in America Steering Committee, with added input from OMB Watch staff. The majority of the channels represented the most frequently updated news and information pages from selected nonprofit organizations that work on federal budget issues. We also included sites of general interest to nonprofits, and nonprofit and mainstream media pages. The first reaction from all testers was that there was a flood of potentially too much information. This was mainly due to our decision to have all clients subscribed to all channels as the default. Users could unsubscribe from the channels they did not wish to receive. Moreover, not every tester understood or took advantage of alert, filter, "show only new content," or search function that could be applied to limit the amount of delivered content In general, users did find BackWeb to be a very useful content aggregation and synthesizing tool, especially given its ability to draw information from organizations and groups that many people might not regularly tap as resources. This could be a potentially useful way to raise the visibility and participation of organizations overlooked or left out of major policy deliberations, by drawing more attention to the quality of the content versus the name of the groups that produce the content. The quality of the content varied wildly from channel to channel. This was mainly because the content delivered was only as good as what each organization provided, and was in turn dependent upon our ability to define the most applicable capture rules used to draw content from specific web pages. Indeed, one unexpected result of our BackWeb demonstration was our discovery of BackWeb's channel creation and content validation rules regarding web pages. This provided us with a glimpse of the inconsistency among organizations regarding page design and formatting. Another issue was that not all organizations update their pages on a regular schedule, or in the same manner. We found that very few would update on a regular daily, weekly, or monthly schedule, and for those that did, there was a tendency to continue to add all new items to a page in which older items were also included. In some cases this meant that the date something was posted was not the same date that BackWeb encountered the item as "new". More accurate definition of the content capture rules would most likely alleviate this issue. Anecdotally, at least two testers noted that BackWeb delivered important content that was actually used for policy work, before they received similar information from either e-mail or the Web. Format of Content No individual delivery mechanism really stood out from the others as being better. Most people stuck to the basic client interface, the newsticker, or title bar. Most important was the ability to click directly to the specific Web page from which content was drawn. There was uncertainty regarding how useful this might be if some users checked their client later than other users and a page were removed. Testers did note that having the ability to have content organized by the source was helpful, but that there should be the ability to select among different categorization schemes. BackWeb places the burden on the content provider to make sure information is presentable and useful. As such, there is bound to be inconsistency among organizations providing information via the Web, such that users may see content without any attendant summary information. The Publishing Wizard, while somewhat slow, was a feature some testers found useful, particularly because it forces the sender to determine the value and target audience for certain content. More importantly, it forces senders to provide a title, shelf life, and summary information on the content they send, which makes it easier for recipients to determine what to do with the information they receive. The expiration date, in particular, helps to minimize that outdated information would be sent to users. For content that was published through the BackWeb system, most testers, while not sanguine about the exact visual and audio content used as "teasers," did see great potential in the ability to customize how certain groups of recipients can be targeted for certain content. Even more impressive was the ability for the system administrator to actually track not only where and how successfully content is delivered, but whether the content is read or acted upon once it is delivered. There were questions raised, however, about the accessibility of this content to persons with disabilities, especially those with visual impairments, and those with attention deficit disorder, for whom the multimedia content and alert mechanism might prove difficult. General Comments There was real interest in the features of BackWeb that might force standardization of the content that individual organizations publish on the Web, especially if there a common posting interface that would lay out the format and suggested guidelines for updating web pages. On the other hand, there was concern that this might put more of a burden on organizations. There was also skepticism concerning how many nonprofits could be convinced to publish things like action alerts and time sensitive information, organization news, or longer documents through this system as opposed to reaching people with e-mail as they do now. For those testers and organizations interested in learning more about the product, there was concern that this might be too expensive for one organization to implement and maintain. There was, however, interest in talking through the idea of implementing BackWeb as either a subscriber-based service or as a system whose cost is shared among different supporting organizations, as long as it was affordable and it were accessible to a wide range of nonprofit content. Nonprofit groups choosing to implement this and similar solutions will need to give some thought as to what makes the most sense in terms of the deployment model. Given the high-level of central administration necessary to make this run, this is a not a tool that allows easily for unmoderated posting of information. It requires careful consideration and assessment on all manner of issues, including: · a comprehensive understanding of all information flows and needs · logical groups of recipients for certain types of content · how to incorporate new and diverse information sources (either by gauging subscriber response and suggestions, or determining a top-down manner) BackWeb requires a commitment to devote significant technical and people resources to ensuring its successful deployment in a nonprofit public policy environment. More importantly, it requires a realistic understanding of what this tool can do and cannot do. It cannot generate quality information, it can only distribute the information in a manner consistent with its administration.