Introduction to Manual Explorer in IBM Security AppScan Enterprise 8.7
Faster, FIPS PUB 140-2 compliant, and browser friendly
IBM Security AppScan Enterprise Edition V8.7 is an enterprise solution for web application and web services security. It offers advanced security testing and risk management in addition to scanning for various types of security vulnerabilities. Compared to other web application security software, IBM Security AppScan Enterprise can:
- Find the latest web application security vulnerabilities and update the pattern matching rules of vulnerability databases, which rely on the IBM X-Force research and development team.
- Provide professional fix recommendations that are designed and written by IBM security experts.
- Generate more than 40 security compliance reports, including PCI Data Security Standard, Payment Application Data Security (PA-DSS), ISO 27001, ISO 27002, HIPAA, and more.
- Provide centralized user management compatibility, and support multi-user and multiple programs that scan at the same time.
The new Manual Explorer tool is included with Security AppScan Enterprise. This article explains the need for Manual Explorer, how to install and configure it, and how to review the output.
The need for a manual explorer
Automated explorer tools can significantly improve your scanning efficiency, but they can't explore all content and URLs in web applications. You need a manual explorer to uncover more URLs and content that might not be discovered by an automatic scan. A manual explorer is useful if:
- There is a specific order involved in accessing a particular web page.
For example, with online shopping a user must submit an order before going to the payment page and then to the confirm order page. When testing the confirm order page with Security AppScan Enterprise, you need to add the order and pay the order first.
- Web applications use anti-automation mechanisms such as pages that require entry of verification codes or answers to questions.
- A specific value is required when scanning a specific page.
- For some dynamically generated URLs, orphan pages or Flash
links cannot be found by automated exploration tools.
- You want to scan just a few web pages and don't expect a global scan.
The new Manual Explorer tool
The original Security AppScan Enterprise Manual Explore plug-in did not comply with the Federal Information Processing Standards (FIPS) PUB 140-2 standard. FIPS is a set of standards developed and published by the U.S. National Institute of Standards and Technology Commission (NIST), which describes document processing, encryption algorithms, and other IT standards for use within non-military government agencies and by government contractors and vendors who work with the agencies.
FIPS PUB 140-2 covers security requirements for cryptographic modules and replaces FIPS PUB 140-1 (published in 1994). To adapt to a wide range of password module applications and environments, the standard defines four levels of security. Security Level 1 provides the lowest level of security and Security Level 4 provides the highest. The latest version of FIPS PUB 140-2 was published in 2002.
Increasingly, software companies are getting FIPS 140-2 standard compliance certification. The certification can greatly help a company to improve their product quality and move toward internationalization. All software sold to government agencies in the United States as well as the regulated industries must comply with this standard. When users enter their user name, password, and other confidential information during scanning, the information might be visible to a third party. Therefore, all components and tools with encryption capabilities within Security AppScan Standard and Security AppScan Enterprise V8.7 must comply with the FIPS 140-2 standard.
To enforce FIPS 140-2 compliance, in Security AppScan Enterprise you can click Edit in the General Settings window (from Enterprise Console Settings on the Administration tab). To edit the Enterprise Console Settings, check the box labeled "Enable enhanced security." This disables non-compliant features, such as the Manual Explore plug-in. (Make sure you have enabled FIPS 140-2 on your system before editing the settings.)
Convenience and lower maintenance costs
The new Manual Explorer tool is more convenient and has lower maintenance costs. Because the original Manual Explorer was a browser plug-in, developers had to develop plug-ins for each browser and continually maintain updates, which creates a large workload. The new Manual Explorer is compatible with Firefox, Internet Explorer, and other browsers; there is an easy configuration to help you select which browser to use. The new Manual Explorer tool also has better performance than the previous version.
Editability and reuse
Explored data generated by Manual Explorer is editable and reusable. It can be saved as an .htd file that you can view and edit with Traffic Viewer. Traffic Viewer is a powerful HTTP debugger that you can use for exploring URLs and for troubleshooting.
Installing and using Manual Explorer
Manual Explorer can be downloaded after creating a content scan job. If Security AppScan Enterprise is installed on the local server, you can also install Manual Explorer by double-clicking ManualExplorerSetup.exe, which is in the AppScan Enterprise\WebApp\downloads directory. The Generic Service Client (GSC), Glass Box, and Traffic Viewer tools are also under this directory.
If you prefer a quick scan, from the Administration tab select User Settings and check the box labeled "Use the browser plugin in browser to record URLs in QuickScans" rather than "Import data flow file," which is checked by default. Otherwise, you cannot import HTTP data flow.
Follow the instructions below to use Manual Explorer in a regular scan.
- From the IBM Security AppScan Enterprise main window, select Scans then click Add to create a content scan job.
- From the What To Scan window, expand Manual Explore, as shown in
and click Add in the Manual Explore section.
Figure 1. Manual Explore
- Select the button for "Use manual explorer tool or AppScan Standard
explore data file" as shown in Figure 2.
Figure 2. Import manual explore data
- Click Download to download ManualExplorerSetup.exe and then install it.
- After the installation is complete, launch Manual Explorer. You can
configure it by selecting File > Preferences. Use the
following information, as shown in Figure 3, in the
Preferences window. Click Done.
- Browser: The default browser set by the user.
- Preferred proxy port: The default is 9999. Manual Explorer will select another port automatically if the default port is in use.
- Trace log level: Error, Warning, Information, and Verbose. The default is Error.
Figure 3. Manual Explorer configuration
If Manual Explorer generated a log file, click the View Log link, as shown in Figure 3, to view the details.
- After configuring the Manual Explorer, you can start to record
the URLs. Make sure the browser set in Manual Explorer has
Click Record... on the Manual Explorer main window to start recording the URLs. A browser opens where you can enter the starting URL and start to explore.
If you want to explore websites prefixed with https, a warning window entitled Untrusted Connection will display, as shown in Figure 4. Click Yes.
Figure 4. Untrusted Connection
- After the exploring is complete, save the explored data as
an .htd file and close the browser. Return to the Manual Explore
window in Security AppScan Enterprise and import the .htd file.
The Manual Explorer tool
will filter out dozens of URL paths such as pdf, jpeg, gif, tar, doc,
m4p, and so on. From the
Manual Explore URLs window (in Figure 5), you can still remove unnecessary
URLs from the Manual
Explore URLs list during importing.
Figure 5. Manual Explore URLs
As you explore the site, Manual Explorer collects and records parameters such as sessions, cookies, and hidden field values automatically. You can remove unnecessary values by selecting items from the Manually Explored Auto Form Fill Fields window, as shown in Figure 6.
Figure 6. Manually Explored Auto Form Fill Fields
After the import has completed, from the Manual Explore window check the "Test URLs as an ordered sequence (multi-step operation)" box, shown in Figure 1, to explore URLs that require a specific order to access.
After the import has finished, you can also configure the scan job according to your requirements. Select a security test policy and run the content scan job after saving. If you just want to test manually explored URLs, the Starting URLs field is not required. The Starting URLs field is required when you scan without manually explored URLs.
In this article, you learned about the features of the new Manual Explorer tool as well as how to install and configure the tool. The Manual Explorer tool addresses some drawbacks of the earlier plug-in and can provide lower maintenance costs.
- Enable IBM SWG Government Standards FIPS 140 Guidelines: Read about IBM SWG Government Standards FIPS 140 Guidelines.
- FIPS PUB 140-2, SECURITY REQUIREMENTS FOR CRYPTOGRAPHIC MODULES: Learn more about this standard, which will be used by Federal organizations when these organizations specify that cryptographic-based security systems are to be used to provide protection for sensitive or valuable data.
- IBM Security AppScan Enterprise V8.7: Get information about new features, installing, upgrading, and migrating.