Enable mapping and geospatial analytics in SoftLayer cloud

Using Esri's ArcGIS map application

Cloud-based infrastructure is becoming the next generation IT platform for managing hardware resources, reducing costs, and optimizing infrastructure. The core characteristics associated with cloud infrastructure have intrinsic relationships with geospatial technologies - namely, dynamic resource allocation to meet the robust demands of geospatial professionals, related domain sciences, and the large amount of compute-capacity necessary to perform analytics on terabytes of spatial data. This article introduces GIS, geospatial analytics, and the Esri ArcGIS and Esri Maps application and how it deploys on IBM SoftLayer Cloud.

This article has been revised for IBM SoftLayer Cloud.

Share:

Philip Heede (pheede@esri.com), ArcGIS for Server Product Manager, Esri

Philip Heede is a product manager at Esri, a global leader in GIS. Philip focuses on server-based products, bringing cloud computing to GIS and spatial analysis. For over 8 years, Philip has dedicated his time and attention to furthering new frontiers for GIS as a consultant and technical lead in Esri's Professional Services division before joining the Product Management team to bring that real-world experience to the core software development organization. Philip holds a Master's degree in Computer Engineering from the Technical University of Denmark.



26 August 2014 (First published 05 September 2012)

A Geographic Information System (GIS) allows anyone to store, manipulate and visualize data displayed on a map for the purpose of intelligent spatial analysis. A GIS can reveal patterns in data, identify relationships and trends, and allow for predictive modeling to understand and plan today for what may impact a business or community tomorrow.

Discover the value that GIS can add to applications, and the value that the cloud brings to this landscape as a new modality for geospatial analytics.

What is GIS?

Maps communicate. Similar to a photograph or a painting, maps can visually punctuate concepts and ideas -- they can record history, model changes, and envision the future. Interpreting patterns and trends depicted by a map can transcend language and cultural barriers and can be a powerful tool for critical decision-making.

Although there are many definitions of a GIS, for the purpose of this article, a GIS is technology that provides a dynamic mapping system to store and access spatial data for the purpose of performing spatial analyses, querying, and geoprocessing.

There are many different kinds of maps and many different kinds of problems that are solved using maps. Planning growth for a city, providing services to a community, managing public infrastructure, developing or evolving a business, or improving emergency response operations are all examples of problems that are better solved using maps. A GIS allows users to compile and communicate large amounts of information, and through visualization, easily reveal relationships and patterns in user data. GIS maps are built on a formal GIS information model that describes geographic features, founded on map layer concepts; this model can be deployed on the web, to desktop apps, through servers, and to mobile devices.

Traditionally, a GIS was implemented with local data in an on-premise software and hardware environment. But increasingly, GIS is being delivered via cloud infrastructure. With the advent of cloud platforms, consuming and sharing data is quickly becoming democratized, making geospatial analysis and GIS accessible to anyone. Since information increases in value with its ubiquity, the platform to deliver intelligent maps to more users and communities adds value not only to those deliverables, but also to the underlying GIS that has been used to generate those offerings through, say, web mapping applications.


Geospatial analytics in action

To underscore the value of geospatial analytics for web developers, let's examine an example of a web mapping application leveraging geospatial technologies.

Esri's Operations Dashboard for ArcGIS is a web application that displays different content in a map context, allowing any type of user – technical, consumer, and managers – to filter, aggregate, and display situational data, while not overwhelming them with minute details causing information overload. This can convey meaning and understanding in an easy-to-understand, dynamic map form. Through the powerful system of a GIS, critical decisions can be made that can benefit the business, the customer, citizens, and constituents.

In Figure 1, the Operations Dashboard web application provides a geospatial context for city incidents and events. Features such as bus routes, demographics, key public facilities, weather, and traffic relate to the events and incidents. Through the visualization of this data, city officials can:

  • Understand real-time events and incidents through maps
  • Track trends and see if incidents are increasing or decreasing
  • Engage one common operational picture, accessible 24x7 through any browser
Figure 1. An example GIS-enabled operational dashboard
An example GIS-enabled operational dashboard

This web mapping application can be consumed through a web browser on the desktop, as well as, on mobile devices such as smartphones and tablets. Users of the application can view the following through a map context:

  • Current events and incidents flowing from a variety of citywide sources, e.g. such as data derived from the IBM Intelligent Operations Center (IOC) solution or real-time data flowing through the ArcGIS GeoEvent Extension for Server, and others
  • Demographics/population affected by the events
  • Critical facilities, such as nearby schools and day care facilities; critical resources, such as shelters and hospitals; and critical infrastructure assets, such as subways
  • Estimated travel time from incidents, plus performance metrics
  • Social media data such a Twitter and YouTube video feeds

Now let's address each of the components on which this mapping dashboard was developed.

ArcGIS for Server

Esri ArcGIS for Server provides the capability to create, manage, and distribute GIS services over the web to support desktop, mobile, and web mapping applications. In addition, ArcGIS for Server provides users with a scalable GIS server platform that can be deployed on a single machine to support small workgroups, or it can be distributed across multiple servers for supporting enterprise applications.

ArcGIS for Server simplifies access to GIS services for developers, GIS professionals, mobile workers, as well as knowledge workers without any GIS experience. With ArcGIS for Server, users stay in control of their content through centralized management of spatial data, including imagery.

Map services

The map service is the way that users make maps available to the web using ArcGIS for Server. Users can then consume the map service in web applications, mobile applications, ArcGIS for Desktop, ArcGIS Online, and other client applications. One common use of a map service is to show operational (business) data on top of basemap tiles from ArcGIS Online or Bing Maps. Many other ways to use a map service include:

  • To serve dynamic maps
  • To serve tiled cache maps
  • To serve features
  • To serve network analysis functions such as routing and geocoding
  • To serve maps or features through KML
  • To serve imagery or rasters
  • To serve maps for mobile devices

Mapping APIs

ArcGIS mapping APIs are available for a wide variety of platforms and languages. For the web, a modern HTML5/JavaScript API is the primary choice. For mobile platforms, Esri SDKs are available for Android, iOS, and Windows™ Phone. For traditional desktops and other devices there are SDKs for .NET, Java®, Mac OS X, and Qt. IBM software products that employ Esri APIs and SDKs include Maximo, TRIRIGA, i2, and SPSS.

Through these different languages, developers are able to leverage the mapping and geospatial analysis web services through their favorite language and its known strengths. For additional ease of use, a large number of configurable application templates are available from Esri, as well as application builders to help even non-developers quickly create custom web mapping applications. This variety of APIs has built-in capabilities to consume ArcGIS for Server map services, making web, desktop, and mobile application development easier.

Mapping content

ArcGIS Online is a complete, cloud-based, collaborative content management system that lets organizations manage their geographic information in a secure and configurable environment, ArcGIS Online Map and Geoservices is one of the features of ArcGIS Online that enables developers to leverage a large variety of base maps in their applications. ArcGIS mapping APIs and SDKs have built-in functions to leverage the ArcGIS Online maps, enabling developers to use the base maps as a backdrop in order to provide context for their own content.

For example, developers who want to add a base map that represents worldwide streets to a JavaScript web application using ArcGIS API for JavaScript and ArcGIS Online basemaps, can do that using a function as follows:

var map;
require(["esri/map", "dojo/domReady!"], function(Map) {
    map = new Map("mapDiv", {
      center: [-56.049, 38.485],
      zoom: 3,
      basemap: "streets"
    });
});

For a complete list of samples that include using different types of map services, widgets that enable spatial analysis, and application templates, the ArcGIS for Developers site is rich with information to help developers and users expand in that area. It includes information on all that is needed to build an application, and how to leverage existing application templates through customization.


Back to the Esri Maps application

In the Esri Maps web application, one of the geospatial analytics features used is calculating a service area based upon a street network and estimated drive times on each street segment. In this analysis, ArcGIS for Server calculates every point on the street network that can be reached from a location in a certain number of minutes. If you request a 5 minute service area, ArcGIS Server calculates all the streets that can be reached from a specified center point and how far a car can get from there in 5 minutes and display a polygon on the map showing that area. This ArcGIS API for JavaScript sample demonstrates how simple it is to implement this on the client side. Once an ArcGIS for Server service area web service has been published the JavaScript application can leverage that directly as a task. The following code snippet highlights that part of the sample:

Click to see code listing

map = new Map("map", { 
  basemap: "streets",
  center: [-122.447, 37.781],
  zoom: 15
});

map.on("click", mapClickHandler);

params = new ServiceAreaParameters();
params.defaultBreaks= [5];
params.outSpatialReference = map.spatialReference;
params.returnFacilities = false;

serviceAreaTask = new 
ServiceAreaTask("http://sampleserver3.arcgisonline.com/ArcGIS/rest/services/Network/USA/NAServer/Service Area");

Service area is an example of geospatial analysis that can be leveraged in ArcGIS. Many spatial analysis tools are available as web services, each of them are capable of doing quite complicated operations that take into account the spatial aspect as a critical part of its calculations.

In our example, the application is implemented as a JavaScript web application. It leverages multiple web services to enable its features. For mapping capabilities the application leverages the ArcGIS API for JavaScript and map services powered by Esri's ArcGIS Online and ArcGIS for Server. The application uses ArcGIS for Server map services to show facilities, demographics, response resources, transportation, and weather information near incidents and events.

Figure 2 highlights the architecture for deploying ArcGIS for Server and the Esri maps web application.

Figure 2. Deployment architecture

ArcGIS for Server can publish content based on a variety of dataset types and formats. ArcGIS for Server can publish Datasets from a file geodatabase, which is a file format for geospatial content. If the datasets do get many updates and require synchronization, it's recommended to keep them in a relational database, which can then optionally be enabled as an Enterprise Geodatabase for Esri's ArcGIS for Server in order to provide even more advanced capabilities. ArcGIS for Server supports a wide variety of RDBMS such as IBM DB2®, IBM Informix®, Oracle, SQL Server, and PostgreSQL. There is also read-only support for Data Warehouse databases such as IBM Netezza® and Teradata with SAP HANA support being introduced with the upcoming ArcGIS for Server 10.3 release.

An alternative view highlighted on the architecture diagram is that of a Geospatial Data Analyst using ArcGIS for Desktop. Using ArcGIS for Desktop an analyst is able to create and manage geospatial content; they're also able to execute very sophisticated spatial analysis.


Deploying ArcGIS for Server on IBM SoftLayer

ArcGIS for Server can be deployed as a single machine; it can also be setup in a multi-machine configuration. This can be useful to meet increased capacity needs, or to provide redundancy for high availability. ArcGIS for Server architecture can be deployed in cloud environments very efficiently. This section covers how ArcGIS for Server can be deployed in the IBM SoftLayer environment.

IBM SoftLayer is IBM's public cloud that is available in many locations around the world. Ease of use with cloud infrastructure capabilities is available for users who want to leverage GIS mapping and analysis power. Unique features of SoftLayer, such as the ability to provision bare metal servers and QuantaStor Storage Appliances, makes it a great choice for users to run even their most CPU and I/O intensive tasks in a cloud environment.

You need a SoftLayer account and an ArcGIS for Server license (trials are available) before proceeding with the following steps. SoftLayer provides the capability to provision bare metal servers, virtual machines, or a combination of both in a private cloud. For their own setup, users should carefully consider the trade-offs in cost versus performance, functionality, and manageability between bare metal and virtual machines.

The following steps include setup that applies to either bare metal or virtual servers. In some cases references to creating an image can be found, which applies in full to virtual servers as well as bare metal servers subject to a few restrictions with regard to operating system support. In those cases where a bare metal configuration does not support creating an image, the setup would include manually installing the software on the new bare metal server rather than provisioning a new machine based on a previously created image.

  1. Use the SoftLayer portal to request a new machine: Click Device List > Order Devices.
    Figure 3. Order SoftLayer products and services
    SoftLayer products and services window

    Click to see larger image

    Figure 3. Order SoftLayer products and services

    SoftLayer products and services window
  2. Select the machine type you want to provision: Bare metal server, bare metal instance, virtual server, or public cloud instances.
    • Bare Metal Server: Fully customizable, takes 2-4 hours to be setup to your exact specifications
      1. Select the CPU type.
      2. Configure the detailed specifications of the bare metal server. The machine should use a supported operating system. For example Microsoft Windows Server 2012 R2 or RedHat Enterprise Linux 6. There are many options to choose from with regards to CPU, RAM, storage, backup functionality, etc. This is where you can customize the bare metal server to suit your exact needs. For example, storage bays can be configured in RAID or JBOD configurations with SSDs or hard disks.
        Figure 4. Configure the bare metal server
        Configure the bare metal server
    • Bare Metal Instance: Fewer options, almost immediately available
    • Virtual server: Classic single tenant virtual machine, limited options, almost immediately available

      Configure the virtual server according to your needs: data center location, CPU, RAM, storage, etc. Like with the bare metal server setup, the machine should use a supported operating system, for example, Microsoft Windows Server 2012 R2 or RedHat Enterprise Linux 6.

      Figure 5. Configure the virtual server
      Configure the virtual server

      Click to see larger image

      Figure 5. Configure the virtual server

      Configure the virtual server
    • Public Cloud Instances: Multi-tenant, almost immediately available

Customizing ArcGIS for server installation

  1. Remote Desktop or ssh to the instance when it's ready.
  2. Download the setup files to a local location and run the ArcGIS for Server setup executable and install the software. You can install both Desktop and Server and use the license files to authorize.

    Before running any other steps, it would be good to create a backup if you've configured a suitable backup solution, or create an image if using a virtual machine or supported bare metal server configuration.

  3. After the backup or image has been successfully created set up a second server either from an image or create and install the software on a second bare metal server. Start that process by copying the spatial (map) data.

    Spatial data can be located in one of several locations:

    • Locally on a disk directly attached to the machine. This should be sufficient if you plan to have one running machine with ArcGIS for Server at any time.
    • Remotely on a shared drive that is common to all machines. This is the setup needed if ArcGIS for Server is deployed in a multi-machine distributed configuration or if multiple individual ArcGIS for Server sites need access to common datasets.
    • In a relational database such as IBM DB2, IBM Informix, Oracle, SQL Server, or PostgreSQL. Setup of such a database is outside the scope of this document, but can easily be consumed from ArcGIS for Server if available.

    For performance sensitive or for very large datasets consider using SoftLayer's iSCSI or QuantaStor storage options.

  4. Set up the spatial data in the location that is planned. Start ArcGIS for Desktop, and load the mxd file from that location. An mxd file, or map document, is a file that includes references to the spatial datasets used and how they're displayed. If the spatial data is in a remote location, make sure to open the ports of both machines where it's deployed and where it will be accessed. Also, make sure the Windows Firewall opens those ports if running on a Windows operating system.

    Make sure to use a network location that continues to be available, even if the ArcGIS Server machines are not. This guarantees that new machines start from images ready to configure and run if setting up a virtualized or cloned environment.

    Figure 6. Setting up spatial data
    Setting up spatial data

    Click to see larger image

    Figure 6. Setting up spatial data

    Setting up spatial data
  5. Start the ArcGIS Server Manager through http://localhost:6080/arcgis/manager if connecting locally from the machine itself (otherwise use the hostname or IP of the machine) and choose Create New Site.
  6. Assign an Administrator account through setting a user name and password.

    ArcGIS for Server Configuration files can be located in one of two locations:

    • Locally on a disk directly attached to the machine. This is sufficient if you plan to have one running machine with ArcGIS for Server at any time.
    • Remotely on a shared drive that is common to all machines. This is the setup needed if ArcGIS for Server is deployed in a multi-machine distributed configuration.

    Make sure to use a network location that continues to be available, even if the ArcGIS for Server machines are not. This guarantees that new machines start from images ready to configure and run if setting up a virtualized or cloned environment.

  7. Using ArcGIS for Desktop, publish a map service to this newly setup ArcGIS for Server site using the data and mxd located in the previously chosen location: Click File > Share As > Service.
    Figure 7. Publishing a map service
    Publishing a map service

    Click to see larger image

    Figure 7. Publishing a map service

    Publishing a map service
  8. At this point, you can create another image. Give this image a name that corresponds to the service published on it. After creating the image, you should be able to see two images in the SoftLayer Customer Portal: Devices > Manage > Images.
    Figure 8. Manage images option in SoftLayer
    Manage images option

Starting an instance from a customized image in a virtualized environment

You can start an instance from an image that contains a pre-configured service. In this case each instance you launch from the image contains the definition of the service and is ready to use. After you login to the manager you are prompted to login, rather than create a site.

  1. After logging into the ArcGIS Server Manager, the same services you published before are already running. This is a good way to restore a one-machine site configuration.
  2. For multi-machine site configuration, use the same image to add more instances, or recover one of the instances. To setup a multi-machine site configuration, launch a new instance from the image. When the instance is ready, you can remote desktop to it, and this time login to the administrator API through the URL, http://localhost:6080/arcgis/admin (or the hostname or IP if logging in remotely). Login using the same account you use to login to manager and select deleteSite from the supported operations.
    Figure 9. For multi-machine site configuration
    Screen shot of the ArcGIS Server Administrator directory

    Click to see larger image

    Figure 9. For multi-machine site configuration

    Screen shot of the ArcGIS Server Administrator directory
  3. By deleting site, the machine is ready to join another site. In this case the site to join is that of the first instance. Go and start the ArcGIS Server Manager, login, and click Join Existing Site.
  4. Enter the IP address of the first instance that contains the main site then proceed to finish after that. For this operation to work, make sure the firewall on the instance is open on the ports used by ArcGIS for Server. This is primarily tcp/6080 and tcp/4000-4003 for simple default setups.
  5. When the new instance joins the first instance they will both be running in the same site forming a cluster. You can confirm that by checking the ArcGIS for Server Manager on either or both machines by going to the Site tab and selecting Machines from the side menu.
    Figure 10. Confirming the join of the new and first instances
    Confirming the join of the new and first instances

    Click to see larger image

    Figure 10. Confirming the join of the new and first instances

    Confirming the join of the new and first instances
  6. This approach can be used to add more instances to this running cluster when needed. It also can be used to replace an instance if it needs to be replaced at any time. In addition, simple automations can be built into the image to enable the same changes mentioned in those steps using the ArcGIS for Server REST based Admin API.
  7. The deployment of a web application can be done on one of the server machines using its IIS web server if running Windows, or it can be deployed on another server such as Apache or nginx if running on Linux. This depends on the architecture of the deployment and the requirements it will address.

Conclusion

Geospatial analysis and mapping can transform applications — not just from a user experience perspective and ease of use, but also from a feature advantage perspective. Understanding the impact of location as part of any information, and taking location and geographic information into account as information is analyzed, can reveal new insight that was otherwise hidden and hard to understand. Being able to run the usual analysis and add the enriching factor of location and geographic feature analysis transforms a partial vision into a complete understanding. The Esri maps web application described is an example that yields new insight simply through rich map visualization of data.

Additionally, leveraging ArcGIS for Server in the cloud is a critical aspect of making such features available while maintaining control over the resource utilization and optimizing it based on the deployment requirements. ArcGIS 10.2 for Server was built to run efficiently in cloud environments as the architecture was designed to work in a distributed virtual or bare metal environment enabling users to get the best performance and scalability. ArcGIS for Server can be successfully deployed in the IBM SoftLayer Cloud to support building applications that leverage its mapping and geospatial analytics capabilities.

Developers who want to leverage mapping and geospatial analysis in their applications can find a very rich platform in Esri products that offers the needed capabilities in the cloud environment.

Acknowledgements

  • Julio Olimpio – Esri, IBM Strategic Alliance Manager
  • Michael Belikoff – Esri Systems Analyst, ISSO
  • Wolfgang Kulhanek – Lead Cloud Architect, IBM Executive IT Specialist

Resources

  • Get started with powerful mapping applications very quickly with ArcGIS APIs and SDKs and application templates that leverage ArcGIS Online and ArcGIS for Server.
  • Focus on the mobile platform and build applications specific to mobile devices using APIs for iOS, Android, and Windows Phone.
  • Data specialists can leverage ArcGIS for Desktop for data management and creation, in addition to generating more data from very sophisticated analytics.
  • Get started today by leveraging the variety of APIs and content available from Esri through the ArcGIS for Developers plans starting at no cost.
  • To develop with ArcGIS Server, get access to the full software suite through the Esri Developer Network (EDN), which is available through an annual subscription and provides a cost-effective way to license Esri ArcGIS products and tools for development. Trial licenses for ArcGIS for Server and ArcGIS for Desktop are also available. Whether you're a web, desktop, mobile, or server developer, ArcGIS for Developers and EDN have the resources you need for building GIS applications and solutions that can be easily deployed in your environment, and in the cloud.

Comments

developerWorks: Sign in

Required fields are indicated with an asterisk (*).


Need an IBM ID?
Forgot your IBM ID?


Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.

 


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name



The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.

 


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks


  • Bluemix Developers Community

    Get samples, articles, product docs, and community resources to help build, deploy, and manage your cloud apps.

  • developerWorks Labs

    Experiment with new directions in software development.

  • DevOps Services

    Software development in the cloud. Register today to create a project.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

static.content.url=http://www.ibm.com/developerworks/js/artrating/
SITE_ID=1
Zone=Cloud computing
ArticleID=981079
ArticleTitle=Enable mapping and geospatial analytics in SoftLayer cloud
publish-date=08262014