Currently, TV series abound with stories about crime scene investigators who carefully piece together diverse clues into a mosaic that predicts the guilty parties. A common theme involves a picture that is taken from a stationary video camera, or a cellular phone call to a specific telephone number, which in turn provides that final piece of evidence that ties the parts into a coherent picture. The urban infrastructure allows for the collection of this type of data.
Several pressing challenges that face the world today, food supply, availability of clean water, cheaper power supplies, global warming, can be addressed by what IBM refers to as a Smarter Planet, a planet that is carefully wired together with a supporting cast of hardware and software to rationalize and optimize decision making. This concept involves collecting volumes of data of various formats, applying special analytic refinements to transform the data into information, and distributing that information to decision makers. By this initiative transportation can become faster and safer; energy can be distributed cheaper and more efficient; and, food can be processed faster and distributed along with the necessary information needed to quickly roll back or even prevent a food contamination crisis.
The highways of Singapore exemplify the implementation of the Smarter Planet concept. Cameras monitor traffic flows and sophisticated software scans the streaming video to determine when an automobile is stopped. Only then does a human gets involved to decide whether the motorist stopped because of a mechanical problem. If the human operator determines a mechanical problem is the reason for the stop, the operator dispatches a tow truck that arrives within 15 minutes. This service is a benefit to the citizens of Singapore, but the real reason to justify the expense of the smart highway is to prevent traffic disruptions.
A corollary to this smarter planet drive is swifter, more precise, law enforcement with serious implications to operations conducted within an urban environment. Why the emphasis on an urban environment? According to The World Bank in 2007 more people lived in cities than rural areas (see Resources). Estimators predict that by 2050 around 70 percent of the human population will live in cities. Urban environments are characterized by an infrastructure of transportation, communication, and commerce.
An array of sensors, such as cameras (still and video streaming) and RFID readers that are connected to central command centers by networks of cables and wireless routers where the data collected can be stored and analyzed when needed helps make this infrastructure smarter. As a result, you can expect to have your picture taken many times while you walk the streets of London, or drive the highways of Singapore, and even enter a taxi in San Francisco. This trend will not stop at the edge of the urban environment. According to an article in the Los Angeles Times, even in the rural Amish community of Lancaster, Pennsylvania, there are reported to be 165 cameras carefully monitoring the day to day life. (See Resources for a link to the article.)
All foreigners who visit Beijing are required to carry the National Identity Card. This card has an RFID chip that is embedded with all the personal information the local authorities know about the visitor, making the information available to any official with an RFID reader within range. Of course, there is no technical reason why the data collected by strategically placed RFID readers cannot be transmitted either by cable or wireless to a central command post focused on visitor management.
Given the pace of scientific advancement, new sensors might become available to more precisely and rapidly identify people of interest. The great Science Fiction writer Philip K. Dick in his Minority Report gave us a glimpse of what this infrastructure might provide to the authorities in the future by foreseeing iris scanners sensitive enough to identify anyone who walks through the urban jungle.
What I refer to as the urban surveillance infrastructure provides persistent surveillance and the continuous collection of information. This information can come is several different formats and is stored for a time during which it is available for retrieval, with no known target in mind, and can be used to assist local authorities in surveillance operations. It is a capability, that is, it provides an ability, always available but only used when needed. The strength of this capability varies from one urban environment to another. In some cities the capability is weak; in others weak but improving; and, in others, it is already strong and getting stronger.
In the urban jungles, the urban surveillance infrastructure forces a whole new challenge to the planning and execution of the mission. And using the history of technological advances as a guide, the challenge will become more prevalent and demanding in the future. This challenge requires a new look at tradecraft and the potential for new mission planning tools to assist the officer. A conceptual base for a new planning tool can be gleaned from trip planning aids, such as Google Maps and MapQuest. To this base you might add extensions to a geographic information system (GIS) workbench virtual map to position geographic points of importance, such as:
- The location and coverage of the urban surveillance infrastructure sensors
- Prior meeting sites
- Local hotspots (police stations)
- Locations to avoid (banks, churches)
- Prior routes to and from meeting sites
Modern GIS workbenches also provide for some intriguing add-ons, such as three dimensional renderings of buildings, which can help create a planning environment in which the officer can conduct a virtual dry run of the mission in cyberspace. It can also help during the execution of the mission by virtually tracking an officer and correlating his or her movements and location with certain warning and indicator flags, for example by activating a specific cell phone within a certain radius of the officer's position. Aids like these might greatly enhance both the officer's and the agent's safety and the successful completion of the mission.
Unfortunately, all urban environments are continually changing. For example, installations of new sensors, cellular towers, and roads coupled with on going maintenance of existing facilities make it expensive and laborious to build and maintain a usable virtual map. What is needed is a strategy of building virtual maps only for those cities that already have a strong, mature, urban surveillance infrastructure in place. We can further refine a list of cities by selecting only those cities that have a suspected competency in using a mature urban surveillance infrastructure for surveillance operations. Accordingly, it is essential to develop a framework for assessing the competency of an urban surveillance infrastructure.
There are three aspects of the urban surveillance infrastructure as a continuous integration tool for surveillance: the underlying technology enabler, the local authority's tradecraft, and the local authority's organization. All three and how they interface with each other are equally important. And although it is my belief that the most sophisticated technology does not make an incompetent organization competent or improve poor tradecraft, we can focus on two areas: the underlying technology enablers and the interface between the organization aspect and the technology as measured by the service level agreement (SLA). I identify seven coarse grained components that together provide the architectural scope of the urban surveillance infrastructure. We can measure the completeness of the urban surveillance infrastructure by the presence of these components and their integration with each other. It is possible to decompose some of these components into finer grained components to provide a more precise assessment. The SLA, a common information technology construct, is an agreement between the technology providers and the users (the surveillance organization) on how the system behaves. The SLA areas of most interest are security, availability, disaster recovery, and the timeliness in which the urban surveillance infrastructure can provide information to the users.
- The urban surveillance infrastructure provides that data in a format suitable for forensics. The data is archived for long periods and is available to the local authorities for reconstructing past operations, the look back that allows them to see forward.
- The urban surveillance infrastructure provides data in a real time or near real-time fashion so that local authorities can use the infrastructure in an on going surveillance operation.
By assessing the three aspects of the urban surveillance infrastructure you can measure the overall competence of the system. That competence is the value it can actually deliver to a surveillance team. I classify the competence into four types: identify, locate, track, and predict.
How accurate the urban surveillance infrastructure identifies an individual depends on the types and fidelity of the sensors.
How precise the urban surveillance infrastructure locates the position of an individual depends on the degree of accuracy of the geographic position that is supplied by the sensor.
How accurate the urban surveillance infrastructure determines the route that an individual takes.
How accurate the urban surveillance infrastructure predicts the next destination of an individual.
Each type requires and expands the previous type. It seems logical that you must firmly establish a competency to identify a person of interest before that person of interest can be located. The competency of tracking a person of interest from one geographic location to another is meaningless unless the geographic locations are available through GPS or triangulation. And, for prediction, one must sometimes consult an archive of tracks to predict future tracks.
Conceptually, the urban surveillance infrastructure can be decomposed into seven architectural logical building blocks. A logical building block is an artifact that is used by system designers to build the framework for a complex, sophisticate system. How does one eat an elephant? One bit at a time. The same principle applies to complex system development. Decompose the system into components or blocks which, in turn, you can further decompose into more refined components. Ultimately, you can develop and use these components independently.
The seven logical building blocks are:
Increasingly sophisticated devices that capture an amazing variety of data exist. The prices of these devices are dropping while their performance and capabilities are increasing. Cameras are a good example. They are becoming less expensive and their reliably improving to the point that you find them most urban settings. RFID readers, as another example of a sensor, are becoming more stable, sensitive, and accurate. One can deploy them at key urban settings to read the data that is transmitted by, for example, a National Identity Card.
The availability of data storage is becoming less of a constraint on system development. A petabyte (1 petabyte is 1,024 terabytes) is becoming the more common unit of storage for large enterprises and governments; and because of the advances in virtualization the management of these storage pools is less problematic. Before, no matter how valuable a picture, video, or voice recording piece of data, if there wasn't enough space available to save it then there was no way to retrieve it later when needed. Current storage technology enables us to store a wide array of information in several formats suitable for rapid retrieval, on demand, very cost effectively. Thousands of pictures can be stored for pennies per picture at a central site where they are available when needed.
High speed wired and wireless networks are becoming more ubiquitous and pervasive. They provide access to data and decision support logic to make informed decisions from virtually anywhere. Even areas that were inaccessible a few years ago are now more accessible to data and support with impressive data transmission rates. Landlines, which are expensive and inflexible, are being replaced by wide area wireless routes that provide greater access and flexibility.
- Information management
Modern information management disciplines and tools provide an effective layer of abstraction over the vast sources of data, by correlating relationships between data to produce real information and knowledge in a format suitable for rational decision making.
Modern entity analytics and fraud detection systems provide unparalleled methods to highlight an individual acting outside some norm that is described in business-like terms. An excellent example is a smart surveillance system, such as the IBM Smart Surveillance System, which can be directed to alert a watch officer whenever a person in a video view puts down a package or suitcase and leaves the area.
- Complex event processing
Advances in software development allow for system behavior to be dynamically controlled by a set of discreet events that occur over a set period. Policies can be put into effect that dictate what response is given for a specific set of events. These policies can be modified quickly to support changes in the operating environment. An example is when someone crosses a border checkpoint, marries a citizen within a week, and checks into a hotel the next day, alone. Another example, in this case an absence of an event, is when a person enters a country and fails to check into a hotel or fails to purchase anything with a credit card within a predetermined period. Each event or absence of an event, by itself is not note worthy. But as a sequence over time the totality of events become interesting.
- Geographic information workbenches
Information that is coupled with a geographic location and displayed on a map provides a clarity that text alone cannot impart. Trip planning tools that display the routes as road segments can also display the location of relevant points, such as police stations, prior meeting places, and other points.
The use of these seven logical building blocks can provide the framework for assessing the maturity of an urban surveillance infrastructure. There is a dependency to these logical building blocks; while some can be independently developed and implemented, others rely on preexistence. Figure 1 shows this dependency in an architectural layered view.
Figure 1. Urban surveillance infrastructure architectural building blocks
The foundation is the standard operating environment that we find in an urban operating environment. It consists of the officers and cars on the ground, the command post, and the operational doctrine, all the items that the local authorities use for surveillance. The essences do not change because the urban surveillance infrastructure is only a technology enabler, which increases the effectiveness and productivity of the standard surveillance model. But the urban surveillance infrastructure does help the local authorities gain a better understanding of our tradecraft, and allow them to do more with less. Fewer officers and cars on the ground might be required for the same level of coverage, reducing expenses that are associated with a surveillance operation.
The base urban surveillance infrastructure consists of an integrated set of logical building blocks, the sensors, the network, sufficient storage, and the software necessary to manage the large pools of data the sensors collect. This scenario includes some sensors that are in strategic places, for example, high definition cameras at a checkpoint, which capture the image of a person and link the image with personal information such as name and age. The camera is connected by the network so the image is stored in a centralized database. Because many people cross that checkpoint, the storage requirements for the images is large and requires an indexing method for quick retrieval if and when needed. As the person travels around the city, other sensors record data, again such as cameras that save the image and compare it to images stored at the centralized site. This base urban surveillance infrastructure allows for the identify competence.
You can augment the base urban surveillance infrastructure by having sensors record the geographic location of the data at the time of capture. For example, the cameras that are used can have a GPS record the position or in the case of data that is captured at a known location, for example an airport, the logic to add that information to the data. This feature allows for the next level of competence, locate.
You can further extend the base urban surveillance infrastructure by adding a geographic information system building block. This extension can help track the movements of the person by displaying the events as the person traveled the city on a virtual map.
You can achieve the greatest competence prediction by using the software tools that analyze the vast pools of data and give some insight to their meaning, and that allow for processing discreet events that happen over time. For example, this person visited the city three times in the past six months. During each visit, he took three similar routes to three different Italian restaurants. The complex event processing software picks up on the past visits, while the analytics provide the probability that the person might take one of the three prior routes to an Italian restaurant. It plots the projected route on the GIS workbench.
By putting the completeness of the integration of the logical building block along one axis of a diagram, and use by the local organization along the other provides a diagram of the maturity of the urban surveillance infrastructure as a tool for surveillance. There is a strong, direct correlation between the completeness of the urban surveillance infrastructure and the surveillance capability as shown in Figure 2.
Figure 2. Urban surveillance infrastructure maturity model
Assessing of the risk of the urban surveillance infrastructure is more complex than determining the completeness and strength of integration of the underlying technology components. A city may have all the components that are implemented, seamlessly integrated, and operational, but technology is merely an enabler that can, if used properly, increase the productivity and effectiveness of the human. If the local surveillance teams are not trained to use the infrastructure in their operations, or if the local mission processes are directed at another purpose, for example population control, then the real threat to our operations is greatly minimized.
I suggest an approach that takes into accounts the following three aspects of the urban surveillance infrastructure. Grade each aspect between 0 and 1 and then multiply to provide a possible score of between 0 (no use) to 1 (high use).
Initially, tradecraft can be scored either a 0, it cannot use the urban surveillance infrastructure, or 1, the tradecraft can use the urban surveillance infrastructure.
- Urban surveillance infrastructure architecture
Each logical building block is graded, followed by how integrated each block is with each other within their respective layer, and followed by how integrated each layer is with each other layer. The assessment, at least initially, is simple. Does the building block, for example the sensors, function, coverage, and reliability to do what they are supposed to do? The score is between 0 and 1 with 3 midpoints.
- 0 indicates the absence of an urban surveillance infrastructure
- .2 indicates that the urban surveillance infrastructure can be used to identify an individual only
- .5 indicates it can be used to locate an individual
- .8 indicates it can be used to track an individual
- 1 indicates it can predict future movements of an individual
Because identify requires the base layer (sensors, network, storage, and information management tools) for an urban surveillance infrastructure to receive a .2 all components must be in place, operational, and integrated. To get a score of .5, requires the addition of geolocation. See Appendix A for the essential elements that are required for assessing the urban surveillance infrastructure technology architecture.
This score is assessed by how strong the SlA between the technology providers and the surveillance organization is.
A combined score, with a range 0 through 1 to assess the value a city's urban surveillance infrastructure provides the local surveillance team, would be practical. A score of 0 states that the city, for all practical purposes, does not have an urban surveillance infrastructure and the surveillance threat comes solely from the local teams that are operating in a manually. Conversely, a score of 1 indicates that the urban surveillance infrastructure is architecturally sound with all seven of the logical building blocks integrated, the local team is trained on its use, and the local authorities have the appropriate mission processes to use it.
The steady improving price performance of technology is providing the basis for an urban surveillance infrastructure to support law enforcement and more. And as the price performance and function of this technology improves the urban surveillance infrastructure will become stronger and more prevalent. This infrastructure will consist of sensors, for example cameras and RFID readers, operating 24x7, feeding a steady stream of pictures, video, or other biometric information to a central data center with enough storage space to store and retrieve them quickly, and efficiently. It will include advanced analytical tools, such as the IBM Smart Surveillance System, that can be programmed to detect unusual and suspicious behavior. It will include advanced event processing tools that present the unusual and suspicious events to a human, thus allowing a smaller staff to focus exclusively on those events that are important. This infrastructure will be backed by a network of wired and wireless nodes to swiftly and expeditiously route the data to the right team at the right time.
The surveillance infrastructure will provide the capability to track an individual with only minimum human intervention. By itself, it can be used as a forensic tool to determine the prior movements of a case officer; and, with targeted surveillance, it can be used to facilitate and enhance the surveillance team's job.
Table 1. Appendix A: Essential elements of information that is used to assess urban surveillance infrastructure architecture
|Competence||Sensors¹||Networks||Storage||Information management||GIS||Analytics²||Complex event processing|
|Identify||Coverage fidelity, reliability, types (camera, video, RFID), geographic location provided (build in GPS, cell towers)||Completeness, speed, reliability||Number of images that are stored, how long images are saved||Are virtual pools supported||Facial recognition|
|Score||0 - .5||.6 - .8||.9 -1.0|
¹Sensors that are used here pertain solely to the urban surveillance infrastructure. As such they are fixed assets, unlike a tracking and location device that is attached to an automobile.
²Facial recognition and locating a data geographic locations through triangulation from cell towers can help to identify and locate a person of interest.
- The World
Bank: Explore the data related to world population.
- "Lancaster, Pa., keeps a close eye on itself" (Los Angeles Times,
21 June, 2009): Read the article about how a web of security cameras
monitors the city of Lancaster.
- IBM Smart
Surveillance System: Find more information about this offering
that provides video based behavioral analysis capabilities.
- IBM Smarter Cities: Learn more about infrastructure solutions for
smarter cities, which are designed to encourage economic development,
modernize infrastructures, and create a new urban model.
- "IBM Intelligent Operations Center" (developerworks, March 2012):
Read a good introduction to IBM Intelligence Operations Center.
- developerWorks on
Twitter: Join today to follow developerWorks tweets.
- developerWorks podcasts: Listen to interesting interviews and
discussions for software developers.
demos: Watch demos ranging from product installation and setup for
beginners to advanced functionality for experienced developers.
Get products and technologies
- IBM Smarter City Solutions on Cloud: IBM Intelligent Operations
Center on IBM SmartCloud offers a straightforward, user-based subscription
service at a single price that includes all costs, including hardware,
software, maintenance, support, and networking.
evaluation versions: Download or explore
the online trials in the IBM SOA Sandbox and get your hands on
application development tools and middleware products from DB2®,
Lotus®, Rational®, Tivoli®, and WebSphere®.
developerWorks profile: Create your profile today and set up a watchlist.
community: Connect with other developerWorks users while exploring
the developer-driven blogs, forums, groups, and wikis.
John Andersen is the Chief Architect for the IBM US Federal CTO group. He works directly with the IBM Federal Clients within the Intelligence Community (IC) and Federal System Integrator partners to provide technical leadership across the domain of complex enterprise systems technology and architecture. John represents IBM within the Industrial Technology Initiative, a forum that allows the US Intelligence Community access to US Industrial Research and Development expertise. For his work on the US Government’s Technology Advisory Panel, a panel designed to help the IC make better decisions on the usage of technology for their mission, John received the prestigious Agency Seal Medal, the highest award given to non-agency personnel. Currently, John oversees several very complex analytic initiatives that span vast pools of data and employ state of the art software and hardware systems, such as Extreme Analytics using Entity Resolution coupled with structured data, and IBM Watson with large pools of unstructured data. Additionally, he is the Solution Architect for a complex architectural design project for a Commercial Cloud implementation for the US Government.