Piecing the Information Together
Written by DAVE MOHR
ROBUST GEOSPATIAL PROCESSING TECHNOLOGIES PROVIDE SOLUTION FOR MULTI-SOURCE FUSION CHALLENGES.
Multi-Source Fusion is the latest term for the age-old military challenge of gathering and synthesizing data sets from multiple intelligence sources so they can be analyzed to provide useful information pertaining to situational awareness and enemy intention. The issue of multi-intelligence fusion, however, has assumed added importance in this era of digital acquisition and delivery when data flows into military operations centers in enormous volumes and varieties.
The multi-source data challenge is especially relevant to the analysis of geospatial data. Geospatial intelligence analysts today are faced with trying to drink from several fire hoses at once as they search, filter and associate the vast assortment of images, maps, vectors and assorted data made available to them in near-real time from diverse sources. Much of this data is collected by satellites, aircraft and ground observations and transmitted directly to the operations center. Due to the volume, the dissemination of this data is automated and usually provides a rich metadata context to perform spatial and non-spatial data indexing. Other data sources are derived from less automated origins, such as message traffic. These often are free-form and lack geospatial information.
Many analysts now routinely utilize combinations of readily accessible commercial and military remote sensing data sets that run the gamut from high-resolution panchromatic to regional synthetic aperture radar images. The addition of non-geospatial data such as electronic signals or message communication presents additional analysis challenges.
For analysts, the crucial priority in multi-intelligence analysis is verifying with a high degree of certainty that each data set relates to common geographic points. Otherwise, identical ground points and features that should overlap will not, which can prohibit or confuse the analysis process. Such geospatial synthesis of raster images, vector graphics, elevation models and other data requires complex photogrammetric capabilities.
While modern geospatial intelligence analysts are experts in the interpretation of these and other types of data, they are usually not trained in the complexities of photogrammetry. Fortunately, advanced image processing systems have been designed to automate, or at least partially automate, the critical steps in the multi-intelligence fusion workflow.
SEARCHING AND FILTERING
As developers of the RemoteView geospatial processing and analysis software, Overwatch Geospatial Operations has integrated tools required for multi-intelligence fusion into a single user interface which has been designed specifically for fast and flexible use for geospatial intelligence analysts as well as military analysts in operations centers anywhere in the world. This has created a seamless and customizable process that allows the analyst to find the right data sets, visualize and analyze them, and present the results for reporting purposes.
The first step in the process—searching for imagery that suits the needs of a particular project and covers the correct area of interest—is perhaps the most daunting due to the number of imaging assets now available. Just taking into account commercial platforms for instance, an analyst may be able to choose from multispectral Landsat, high-resolution IKONOS and QuickBird, and synthetic aperture Radarsat imagery covering nearly any point on Earth. The key to finding and accessing the applicable data set lies in the metadata.
Metadata refers to the non-picture data that accompanies the imagery and documents the collection information. It can contain details of the scene location coordinates, acquisition date, look angle, cloud-cover percentage and numerous other image characteristics and acquisition conditions. This metadata is the focal point for finding the common ground among imagery data. Although complete metadata standardization has not yet been achieved among databases, most image archives in both the commercial and military sectors use metadata as indexes that can be searched to locate an image by geographic coordinates, dates, resolution or a combination of criteria.
Of course, image data alone does not tell the whole story, and analysts must search for non-geospatial data that may or may not have metadata attached. Even if metadata exists for these pieces of information, it is typically different from imagery metadata. The solution is a metadata search framework consisting of multiple tools designed to convert formats and extract data index elements even if they are not tagged as metadata.
A common example is the problem of referencing electronic messages to a geographic location. Although latitude and longitude coordinates may not be attached to the messages, place names, target identifiers, phone numbers or postal codes might be embedded within the data. In this situation, a data format or reference system conversion and possibly a second-order query must be applied to correctly determine the geographic location relating to that data. The second-order query may involve extracting a place name from the message data and then searching another database to convert the name to geographic coordinates.
The metadata framework offers built-in flexibility to select and utilize search tools applicable to any network-connected database, assuming there is open database connectivity (ODBC) or Web access. There is no single solution to the data-searching challenge. Ideally all necessary information would be consolidated into a single, all-encompassing database. But in reality this approach is not feasible. A full-featured metadata framework enables each end user organization to quickly customize query workflows that apply to the data archives that it most often accesses without the need for the data normalization required in other products.
Geospatial searches must also include intelligent filtering so that analysts can narrow their criteria based on specification ranges. Analysts are concerned with the accuracy of selected images and must be certain that multiple data sets can provide the required accuracy before they are fused. Intelligent filtering is capable of comparing a new data set with another data set of known accuracy from the same area. This comparison can determine whether the new imagery has the spatial accuracy to satisfy the requirement of the application at hand.
Despite the volume of data that may be pouring into an operations center, the analyst invariably has a need for mission-critical information that is not contained in an internal database. Overwatch has solved this problem by developing Web-based search and retrieval tools that enable the analyst to extend the query to other databases. As long as these databases are Web-accessible and the appropriate security access is granted, these tools will find the right data sets, retrieve them and bring them back into the processing system for fusion and analysis.
VISUALIZATION, ASSOCIATING AND ANALYSIS
A cornerstone of geospatial visualization is the merging of images. This merging process requires data sets to be georeferenced in such a way that features in each data set line up precisely with each other. The robust geospatial processing software in RemoteView combines multiple images using hundreds of ground points in two dimensions (2-D,) regardless of file format, resolution, pixel size or datum, and create a single image on the fly. This automated conversion and 2-D georeferencing literally take seconds or tens of seconds instead of hours, making image incompatibility a thing of the past.
However, the modern military analyst must operate in the three-dimensional (3-D) world, and 3-D feature extraction in the geospatial processing environment is rapidly becoming a necessity for precision targeting and other applications. 3-D feature extraction in remote or inaccessible parts of the world has been a major stumbling block where precise 3-D feature extraction was required. Areas such as active battle zones, covert regions, or remote location all share the absence of ground control points required to perform 3-D precision positioning. The integration of a technique called multiple-image ray intersection into a wizardbased workflow now makes it possible for even the novice user to extract precise 3D positions from imagery without ground control provided that sufficient camera model information is available.
Once the images have been merged, rectified and positioned in three dimensions, analysts are ready to interpret the composite imagery, identify features, overlay previously extracted features, and provide the assessments they have been trained to make. While automated feature detection and identification have received much press and may indeed be included in the future, there is no substitute for human beings in the multi-intelligence fusion workflow.
Today, full-featured geospatial processing systems offer wizardbased image enhancement and manipulation tools—such as landcover classification and change detection—that a user can invoke with the click of the mouse. This simplified workflow relieves the analyst of the burden of understanding the exact mechanics of processing algorithms, allowing him or her to focus on interpreting the scene that has been visualized.
Following interpretation of multi-intelligence data, the analyst must present the results in a report to commanders. Often the preference of the command or the needs of the situation will determine the format in which the assessment will be delivered. To accommodate this, robust geospatial processing packages include a variety of output methods to easily incorporate fused images, annotations and assessments into common documents such as Microsoft Word, Excel and PowerPoint.
Overwatch has taken presentation one step further and leveraged the expertise of the Web programming community to use Javascript and PHP, widely-used Open Source and general-purpose scripting languages especially suited for Web development to display geospatial information in a dynamic Web environment that can be accessed by as many end users as needed. When situational awareness is an issue in a war zone, analysts are increasingly turning to these Web-based tools to create dynamic Web documents. These dynamic reports can now be created in minutes by an intelligence analyst and posted to a common network to provide a complete understanding of the combat environment to a broader community at speeds unimaginable just a few years ago.
VISUALIZING THE FUTURE
Multi-intelligence fusion enables decision makers to transition from situational awareness to situational understanding. The fusion of images helps these individuals to visualize the situation on the ground in near real time. The addition of photogrammetric rigor to analysis increases the integrity of the intelligence. The critical addition of other non-geospatial data sets—achieved by accessing remote databases—lets the decision makers put the situation on the ground into a perspective as it relates to their mission, giving them an understanding of events and activities that could not be derived from a single data source. ♦