A set of interactive and collaborative Internet technologies that is already making a mark on American political and popular culture is also beginning to play an important role in the military and intelligence technology constellation. Collectively known as Web 2.0, these Websites and technologies provide richer user experiences, allow for the combination of information from different sources displayed in various formats, and facilitate interaction, collaboration and information sharing.
Videographers post their work on Youtube for quick and widespread dissemination. Internet surfers flock to Wikipedia, an Internet encyclopedia written and updated by the general public, for information on a wide variety of subjects. Presidential candidates have established Myspace pages to attract contributors and to establish communities of supporters. College students have been known to spend hours every week updating their Facebook profiles and interacting with other site members.
Not surprisingly, interest in the collaborative capabilities these Websites offer has migrated into usage in the military and intelligence communities through some of their more Internet-savvy employees.
“A number of our agency analysts and technology engineers follow innovation on the Internet pretty closely,” said Lewis Shepherd, chief of the requirements and research group at the Defense Intelligence Agency (DIA). “Through general Internet usage at home, they began to see analogies between what knowledge workers were doing on the Internet and what we in the intelligence community could do.”
In 2004, DIA began to experiment with beta versions of open source Web 2.0 software on its classified networks. The first such as effort was a wiki—a Website that allows visitors to add, remove, edit and change content—and that has evolved into the use of multiple wikis and numerous other Web 2.0 tools.
“The goal is to improve the ability of our Directorate for Information Management and our CIO to provide information that our analysts and warfighters need in a more timely and more usable fashion,” Shepherd explained. “We find increasingly that Web 2.0 applications improve speed, decrease latency and improve the collaborative capability of analysts.”
Meanwhile, the military services are using Websites like Myspace and Facebook to establish virtual recruiting stations, according to Anthony Bradley, an analyst at technology research firm Gartner. “The Marines have also posted recruiting videos on Youtube,” he said, “but it’s not clear whether this is an official or a grassroots effort. Web 2.0 technologies can help the services build much more robust two-way communications channels with potential recruits, in contrast to the static Websites they have used until now.”
Web 2.0 refers to a set of Internet phenomena such wikis, blogs, RSS feeds and mashups—as well as behindthe- scenes enabling tools—that provide users with more dynamic, interactive and collaborative functionalities than earlier Internet experiences. Gartner divides Web 2.0 into subcategories.
While Web 2.0 tools started out as a consumer phenomenon on the Internet, they are now being adapted by enterprises in what Gartner terms Enterprise Web 2.0. “Enterprises are using the categories of Web 2.0 and directing them toward their enterprise goals,” Bradley explained. “These may include enhancing interactions with employees inside the enterprise firewall as well as customers, contractors and stakeholders on the outside.”
Enterprise Web 2.0 is being facilitated to a certain extent by new versions of enterprise applications. “These applications often include Web services that allow data, and sometimes logic, to be exposed,” noted Paul Wlodarczyk, vice president of solutions consulting at software firm Just Systems.
Web 2.0 technologies allow users to create their own small-scale programs because browsers have been enhanced with tools that create a programming environment, allowing users to combine information from different sources and display them in varying formats. “These can be handled by the line of business,” Wlodarczyk added. “You don’t have to go back to IT to develop them.”
This puts more control of content and its presentation in the hands of users. “IT deals with big applications,” explained John Crupi, chief technology officer at software company Jackbe. “If a user wants two or three small things, IT can’t handle it because it won’t scale.”
At DIA, the Web 2.0 phenomenon has been implemented hand-in-hand with the agency’s serviceoriented architecture, known as ALIEN, or All-source Intelligence Environment. “ALIEN includes semantically enhanced and richly metadata tagged data based on a common metadata standard on structured and unstructured data from multiple databases,” Shepherd said. “We have found that with this common data format and Web 2.0 tools, we get a big bang for our buck with much richer data right out of the box.”
DIA recently implemented an open source application called Gallery for sharing images. “It took less than three days from start to deployment,” Shepherd said.
DIA’s first Web 2.0 effort was to create a wiki, with the idea being to do for the intelligence community what Wikipedia does for the general public: provide an encyclopedia that would allow the distributed expertise of intelligence community analysts and specialists to be reflected in its pages. “They can author and edit content at any time in a very user-friendly way,” Shepherd said.
That wiki, known as Intellipedia, is now maintained by the chief information officer of the Office of the Director of National Intelligence on behalf of the entire intelligence community.
“We found that wikis could be used in a lot of different ways,” Shepherd said. “We now have multiple wikis, all commonly searchable and some maintained by communities of interest.” For example, a wiki known as the Defense Intelligence Integration Guide is open to government engineers as well as contractors and support personnel. Contractors also have access to the site, Shepherd noted, but are not authorized to alter the content.
“Intellipedia is a shining example of a successful widespread enterprise implementation of Web 2.0,” commented Gartner’s Bradley.
The military intelligence community is also making use of mashups, a Web 2.0 technology that allows the collection of data from a variety of sources and their visualization in multiple ways. A mashup is a Web 2.0 application that rapidly provides presentation layer integration of information from multiple sources, explained Bradley. In the consumer world, mashups have been used to combine classified ads for apartment rentals and a map application to provide the user with a graphical presentation of where apartment vacancies are located.
In military and military intelligence environments, human intelligence feeds could be mashed with other intelligence information and mapping tools to deliver better information visualization for enhanced situational awareness, according to Bradley. “Mashups can also be used as the presentation layer mechanism to create common operational pictures,” he added.
The intelligence community is using mashups to display data provided by the National Geospatial-Intelligence Agency from Google Earth and the Arc GIS Explorer service from ESRI, according to Shepherd.
Creating mashups requires the deployment of middleware technology that extracts data from databases and allows them to be displayed on an Internet browser in a variety of formats. For over a year, DIA has been creating mashups for intelligence briefings using Presto framework from Jackbe. DIA has also used Presto to build “a number of portal-like applications and user interfaces enabling users to add mashup visualizations to their own personalized Web environment,” said Shepherd.
“Presto allows databases to be exposed as a service,” explained Jackbe’s Crupi. “The user can combine various elements of data together and display them in tabular, graphical or other formats.”
DIA is using the technology to design intelligence briefings for Pentagon decision makers, Crupi added. “For the user, it provides the ability to create new services out of the box and interact with that information.”
Web 2.0 mashups are also being used to develop integrated battlespace management and situational awareness systems, according to Lou Hutchison, vice president for public sector sales at Nexaweb, which provides Enterprise Web 2.0 technologies.
“Previously these were ‘swivel chair’ applications,” Hutchinson noted, referring to intelligence gathering and dissemination applications used to provide information in-theater. “They need to be rolled up into enterprise class applications.
“Traditionally what would happen is that an analyst would have to deal with a multitude of applications in order to satisfy that individual out in the field,” he added. “Our technology allows the analyst to utilize all of these applications without compromise to their robust functionality and gives the analyst the ability to deal with those systems in an Internet browser regardless of where the user sits.”
Supplying an enhanced user experience requires providing a back-end infrastructure that interacts with enterprise applications, according to Hutchinson. “You need to make them richer, not only on the front end but behind the scenes as well,” he said. “This requires not only dealing with user interfaces, but also with the heavy lifting and plumbing of critical messaging and data integration requirements.”
Mashups are to be distinguished from portal applications by the interactivity and flexibility that mashups offer, according to Just System’s Wlodarczyk. “A portal is a composite application that pulls sources of information onto one piece of glass,” he said. “Portals provide an integrated view into content, but there is no interactivity and no process integration.”
Mashups, by contrast, represent a distinct subclass of composite applications, said Wlodarczyk. “Mashups provide an interactive interface, and those interfaces can be customizable. This provides real and measured benefits to enterprises because it allows end users access from their browsers to disparate information systems without having to download. The information is more up to date and you are better able to provide end users the information they need.”
DIA also makes heavy use of RSS feeds, a format for syndicating the content of Websites. RSS feeds could be used to relay the latest news on a given subject or to notify users of recent changes to a wiki or other content site.
“There is a lot of interest in RSS feeds created off specific data sources so that databases receive automatic updates,” Shepherd said. DIA analysts also generate RSS feeds from search engines and receive information as updates to Web pages, wikis or e-mail in their inboxes. “That way they never have to run the search again.”
DIA’s use of open source Web 2.0 tools means the agency must take some precautions before letting those applications loose, especially on classified networks. The agency has deployed a sophisticated accrediting system which allows programmers to vet new software quickly. DIA also has deployed a tool that allows programmers to drag and drop new applications into the agency’s programming environment once it has been approved.
DIA’s rapid prototyping environment, known as Lax X, keeps unaccredited software safely separate form live data systems. “Lab X is the rapid prototyping environment where we post beta versions of software that are not quite ready for prime time,” Shepherd said. “The applications are fully vetted and tested without causing any damage to data or to the infrastructure.”
Lab X recently went live for users of the top secret JWIX network.
Enterprise Web 2.0 applications generally don’t experience the same free-for-all atmosphere associated with wikis and blogs on the public Internet because they are inside the enterprise firewall, noted Gartner’s Bradley. “Military and intelligence wikis should not allow, or should significantly limit, allow anonymous postings,” he said. “This approach should minimize nebulous, suspect or inappropriate postings. Also, enterprise wikis don’t have the same security problems from with phishers and spammers because access to the wiki is constrained.”
While the use of Web 2.0 technologies may just be getting under way, the next Internet iteration, known as Web 3.0, is already hurtling up the turnpike, according to Shepherd. “We, like many others, are scratching our heads over what Silicon Valley calls Web 3.0,” he said. “Web 3.0 is a more semantic web. We are heavily investigating semantic technologies and are using many semantic tagging engines and extracting software.
“The value of Web 2.0 mashups will be improved in Web 3.0 if semantic meanings are baked right into the data,” Shepherd continued. “XML tagging is becoming more and more semanticized, and we are watching with keen interest what is being done. We are eager to see more advances in semantic approaches.”
DIA has deployed a suite of 13 different commercial metadata extraction and tagging services in order to inject greater semantic meaning into its data. “Some of these tools are data normalization pieces, some are semantic identity and extraction tools, and some are tagging and data structuring pieces,” Shepherd explained. “They perform in-line data markups and set up semantic databases that identify the semantic relationships among the entities.”
This extraction and tagging “factory,“ as Shepherd terms it, works constantly on the agency’s data traffic to produce semantically enabled in-line markup of data and text. DIA is also exploring visualization and other analytic tools that can provide rich value relationships among entities.
“We haven’t invested in any one company,” Shepherd noted, “but are eagerly watching the advantages companies are bringing to the new semantic era.”
The semantic Web 3.0 will further enable the Pentagon’s dream of a “push” information paradigm, in which analysts, decision makers and warfighters will automatically be provided the information and intelligence they need without having to search for it. “We now still rely on the pull paradigm, in which users are composing and performing complex queries in their hunting and gathering efforts,” Shepherd said. “What users prefer are systems smart enough to semantically understand raw text, correlate multiple types of information and compose relationships across all sources of information.”
Such a system would provide analysts, commanders and warfighters with prepackaged, actionable intelligence packets. “Once the system understands peoples, places, things and events mentioned in texts and the relationships between them, it will be able to route real time information to where it is needed,” Shepherd said.
“This will provide value especially for warfighters who have no time and no bandwidth to search across large databases,” he added. “We don’t want to have to train warfighters in information retrieval. We just want to provide them with the information they need, correlated from across all sources.” ♦