Make PaaS your vulnerability testing ground

Building a cloud security testbed is difficult; explore various concepts and models

Evaluate, integrate, and define various security testing concepts in differing scenarios with the author; you can also explore a sample user PaaS testing environment structure as a basis for a security testing model.


Judith M. Myerson (, Systems Engineer and Architect

Judith M. Myerson is a systems architect and engineer. Her areas of interest include enterprise-wide systems, middleware technologies, database technologies, cloud computing, threshold policies, industries, network management, security, RFID technologies, presentation management, and project management.

03 June 2014

Also available in Chinese Russian Japanese

Let's examine using Platform as a Service (PaaS) as a vulnerability testing ground by using different security testing concepts in different scenarios. In this article, I will show you how to evaluate, integrate, and define any security testing concepts, then further explain the information by linking concepts to scenarios.

But before we start with the more difficult concepts, let's take a look at a user PaaS model structure to gain familiarity with the topic.

User PaaS model structure

The user PaaS model structure falls into three types:

  • The application development lifecycle process that tracks an application on the PaaS from requirements through design and coding and security testing to deployment stages.
  • The risk-management lifecycle tracks the processes of mitigating risks from identifying assets to implementing cost-effective safeguards. Risk is the likelihood a threat agent will take advantage of one or more vulnerabilities.
  • The business process lifecycle allows the developer to control and protect applications in each PaaS stage. As part of this cycle, the developer uses spreadsheet, word processors, billing and other business tools.

Let's take a simplified look at how these lifecycles are interrelated to one another on security testing:

  • In the risk-management lifecycle, PaaS testers identify risks to application development, then create risk-based approach to security testing.
  • In the application development lifecycle, testers apply the risk-based approach.
  • In the business process lifecycle, the testers use spreadsheets and documents to record the results of the risk-based security testing including vulnerability assessments.

Now let's look at a generic security testing model.

Security testing model

Identifying the weakest link, conducting penetration testing, and relying on standards and frameworks are insufficient methods to discover and detect these three types of security issues:

  • Security flaws at the PaaS application design level
  • Security bugs at the PaaS application implementation level
  • Resource outages at the PaaS platform level

A better solution would be to set up a security testing model using PaaS as the vulnerability testing ground. The model consists of the following phases:

  • Discovery stage
  • Automated vulnerability scan
  • Vulnerability assessment process
  • Security assessment process
  • Penetration test
  • Security audit
  • Security review

The discovery stage phase is the foundation of the security testing model; it's the building block of the successive phases. By using this model, gaps can be found in any phase and analyzed before proceeding to the next phase. This process is repeated until it reaches the security review phase.

If the tester finds security testing issues in any phase, she can return to an earlier phase to repeat the testing process with the new information, then proceed to the next phase.

The tester should set her own limit on number of times she should return to any previous stage. She should document on the results of new or improved security tests in each phase until she reaches the limit.

Now let's look at each of these stages and couple them with scenarios.

Discovery stage: SaaS upgrade

The discovery stage is the first stage in security testing when using PaaS as the vulnerability testing ground.

What it's about

You start by identifying what applications are being developed and tested on the PaaS. Then you attempt to detect or discover the application's potential vulnerabilities, such as:

  • Logic flaws in the application
  • Slow responses to user queries
  • Improper configuration of database connections

You can often find this information in documentation, including codes and logs.

Problem scenario

You discover that an application that runs well on-premise does not run as well as a Software as a Service (SaaS) application. The application's attack injection vulnerabilities (caused by processing invalid data) show up while running the application in the cloud.

Solution scenario

Review the code in the on-premise application, then conduct simulated attack injections while in the PaaS testing environment.

Automated vulnerability scan: IaaS upgrade

Following the discovery stage, look for known security issues by using an automated vulnerability scan tool (or several) to match conditions with known vulnerabilities.

What it's about

This type of tool automatically sets risk levels with no human intervention needed to verify or interpret the levels. The tool can be supplemented with credential-based scanning that helps to remove some common false positives by using supplied credentials to authenticate with a local accounts service.

Problem scenario

The Infrastructure as a Service (IaaS) on which the PaaS runs is upgraded with the addition of virtual machines that may come with hidden vulnerabilities.

In a virtualized environment, isolation and security zones enabled through the use of firewalls, routers, switches, and IPS devices can be created. The trouble is that the virtual machines (VMs) float around within the IaaS, and it can be difficult to get your system's security rules to consistently follow these machines.

Solution scenario

Get security policy management tools and determine that processes governing the management of VMs are in place. This ensures that changing the location of VMs will trigger replication of required security functions to the new location.

Seek out virtualization-aware solutions. Virtualization-aware, or VM-aware system and network components identify VMs that help your centralized management console tool track, monitor, and update these VMs with correct and changing policies. This in turn helps you manage network security policies, with the VMM/Hypervisor for added visibility and control.

Vulnerability assessment process: multiple host vulnerabilities

Built upon the discovery process and vulnerability scanning phase, the vulnerability assessment process stage is used to identify security vulnerabilities.

What it's about

The vulnerability assessment process places the findings on security vulnerabilities into the PaaS environment under test. An example would be removing common false positives from the report and determining risk levels that should be applied to each report finding.

Problem scenario

A PaaS developer's cloud-resource optimization application contributes to false positives in the reports a user has successfully accessed the PaaS, when in reality the application has failed. The failure causes the PaaS platform to shut down completely. The application doesn't identify failures or implement short timeouts. It negatively affects performance data needed to determine service-level agreement (SLA) availability guarantees.

The PaaS developer did not build simple services comprising a single host; the single host would allow him to create replicated service instances that can survive host failures. Instead, she built them comprising multiple, dependent hosts she discovered too late could not survive host failures.

Let's take a look at how the developer has built multiple dependent hosts. If the developer has a billing application that consists of business logic components — A, B, and C — she can compose a service group like this:

In (A1, B1, C1), (A2, B2, C2), (An, Bn, Cn), where n is the number of component type representing the number of a service group category:

  • for service category 1:
    • A1 is the logic to find service code.
    • B1 is the logic to insert service category into the bill.
    • C1 is the logic to check the accuracy of ZIP codes.
  • for service category 2:
    • A2 is the logic to find service code.
    • B2 is the logic to insert service category into the bill.
    • C2 is the logic to check the accuracy of ZIP codes.
  • for service category n:
    • An is the logic to find service code.
    • Bn is the logic to insert service category into the bill.
    • Cn is the logic to check the accuracy of ZIP codes.

Due to long timeouts, the user is able to access the system when the system's health is failing.

Solution scenario

To fix the problem so the system will not fail, the developer decomposes the components into independent pools like this: (A1, A2,...An ), (B1, B2...Bn ), (C1, C2, ...Cn).

This allows her to create multiple redundant service copies at healthy data centers. This means if component A1 fails or slows down, all other components A2-An in the same independent pool will not fail. Likewise, the second independent pools of components (B1-Bn) and the third pool of components (C1-Cn) will not fail.

As a result, the user is able to access the healthy systems. False positives do not appear in the report findings on vulnerability assessment process.

Security assessment process: Threshold-level injection vulnerabilities

The security assessment process builds upon the vulnerability assessment process by adding manual verification to confirm the level of exposure that could occur in the production phase.

What it's about

The security assessment process does not include the exploitation of vulnerabilities to gain further access. It could be verification in the form of authorized access to a system to confirm system settings and involve examining logs, system responses, error messages, codes, and validation of numerical analysis.

A security assessment widens the coverage of the systems under test. It does not cover the depth of exposure that a specific vulnerability could lead to.

Problem scenario

When the holiday buying season crunch hits, the system detects higher workload demands. In response, the system quickly creates additional instances of resources to balance workload demands dynamically. This requires additional users and a higher number of data requests to access the system.

Resources not needed are never freed up due to threshold-level injections, eventually leading to a system crash. Accessing the system to manually confirm the level of exposure that could occur in the production phase has not been done.

Solution scenario

Manually examine logs, system responses, error messages, and codes to verify authorized access to the system. Manually verify that a threshold policy exists and outline what cloud computing service consumers and providers should do in developing the following policies:

  • Resource threshold policy to ensure that consumption is balanced dynamically for applications in the cloud below or at the threshold level.
  • User threshold policy to ensure users can access concurrently the application up to the limit specified in a user license from the provider below or at the threshold level.
  • Data request threshold policy to ensure that data requests to the application can be processed immediately below or at the threshold level.

Penetration test: LDAP/AD injection vulnerabilities

The penetration test simulates the PaaS under attack. It builds on the previous stages and tests the exploitation of found vulnerabilities to gain further access to the PaaS and IaaS.

What it's about

Penetration testing tests the ability of an attacker to gain access to confidential information, affect data integrity or availability of a service, and the respective effect on the PaaS.

Each test compares one another on the tester's ability to use their problem-solving abilities employing a range of penetration testing tools in finding vulnerabilities that would not be identified by automated tools. Penetration testing also tests the ability of a defender to detect the attacks and response appropriately.

According to NIST SP 800-115 (Resources), most vulnerabilities exploited by penetration testing include:

  • Misconfigurations
  • Buffer overflows
  • Incorrect input validation
  • Race conditions
  • Incorrect file and directory permissions

A penetration tester should be able to determine when to stop before a further action will cause damage to the PaaS and the underlying systems. It could lead to loss of system availability or exposure to sensitive data.

Problem scenario

Exploitation of LDAP/AD injection vulnerabilities results in a system crash. Hackers take advantage of the application's failure to neutralize characters that have special meaning in LDAP.

Closely resembling SQL injection, LDAP injection occurs when LDAP statements are constructed with user-supplied data that is not verified in the application. This results in executing arbitrary commands such as granting permissions to unauthorized queries and altering contents within the LDAP tree.

Solution scenario

Explore using top-notch external penetration testing tools to test for LDAP and SQL injection vulnerabilities. The tool should cover other types of security vulnerabilities such as Cross-Site Scripting (XSS), improper authentication protocols, and improper database connection configurations. Look for good internal penetration testing tools to test whether LDAP and security policies are adequate in protecting against accidental introduction of LDAP/AD injections.

Also, find good wireless site penetration testing tools to test for wardriving (searching for vulnerable WiFi networks while in a moving vehicle) and man-in-the-middle attacks. Ensure that wireless security protocols are sufficiently secure, and access points and clients (mobile devices, laptops) are correctly configured. Proper configuration can be used to prevent impersonation of a rogue laptop in an attempt to intercept client-access point communication that could be used to inject LDAP/AD malware. Use your mobile laptop or stationary desktop to discover unsecured wireless networks that belong to you.

Security audit process: Threshold noncompliance

A security audit is a systematic evaluation of the security of a company's information system by measuring how well it conforms to a set of established performance criteria.

What it's about

Security audits are often used to determine how well information systems have been complied with legislative regulations, industrial standards, and security policies. Making use of the earlier security tool approaches we've covered, a security audit assesses the security of information systems, information handling processes (classified vs. unclassified), and user practices.

Problem scenario

An audit reveals threshold levels are not in place despite the earlier security tool approaches to correct the problem. The audit finds that when threshold levels are not set, a developer has no way of knowing:

  • If resource is excessively consumed.
  • If the number of developers attempting to access the PaaS has reached the limit specified in a user license.
  • If data requests the developer sends exceeds the threshold level.

Solution scenario

An auditor should recommend what developers should do in developing the following policies:

  • Resource threshold policy to ensure that resource consumption is balanced dynamically for applications in the cloud below or at the threshold level.
  • User threshold policy to ensure that can access concurrently the application up to the limit specified in a user license from the provider below or at the threshold level.
  • Data request threshold policy to ensure that data requests to the application can be processed immediately below or at the threshold level.

The threshold policy should address numerical analysis vulnerabilities and how to handle them. In short, if truncation and rounding errors exceed the acceptable bounds, the metrics obtained will show loss of significance. PaaS developers most likely have more expertise than the provider in developing on numerically stable algorithm that solves a well-conditioned problem. This will ensure that the threshold policy will be established before you reach the security review phase.

Security review process: Compliance verification

Security review involves the process of verifying that industry or internal security policies have been applied to system components or product.

What it's about

Security review is typically completed through gap analysis or reviews of the code or by reviewing design documents and architecture diagrams. It does not use the other security tool approaches — vulnerability assessment, security assessment, penetration test, or security audit.

Problem scenario

During a security review, an organizational unit did not fully comply with some aspects of NIST Special Publication 800-115 on Technical Guide on Information Security testing and Assessment.

Solution scenario

Review documentation to determine whether technical and security aspects of policies and procedures are current, and review logs to ensure that security controls are logging information on the use of PaaS and whether the organization is adhering to its log management policies. Examples of log information include:

  • Authentication server or system logs on successful and failed authentication attempts
  • Intrusion detection and prevention systems logs on malicious activity and inappropriate use
  • Security logs recording information on known vulnerable services and applications
  • Threshold logs recording the level of user thresholds, data requests, and resource thresholds

Automated audits tools should be available to reduce review time of most log types.


In planning for security testing, consider best practices for resolving security testing issues in each phase of the model. You need to set a limit on number of times you could return to a previous phase to resolve testing issues before proceeding to succeeding phase. Build a team of developers, managers, business analysts, and make it easier for them to do their jobs conducting security testing in each stage of its lifecycle.




  • Get involved in the developerWorks community. Connect with other developerWorks users while exploring the developer-driven blogs, forums, groups, and wikis.
  • Try IBM Bluemix™— an enterprise-grade cloud development platform.


developerWorks: Sign in

Required fields are indicated with an asterisk (*).

Need an IBM ID?
Forgot your IBM ID?

Forgot your password?
Change your password

By clicking Submit, you agree to the developerWorks terms of use.


The first time you sign into developerWorks, a profile is created for you. Information in your profile (your name, country/region, and company name) is displayed to the public and will accompany any content you post, unless you opt to hide your company name. You may update your IBM account at any time.

All information submitted is secure.

Choose your display name

The first time you sign in to developerWorks, a profile is created for you, so you need to choose a display name. Your display name accompanies the content you post on developerWorks.

Please choose a display name between 3-31 characters. Your display name must be unique in the developerWorks community and should not be your email address for privacy reasons.

Required fields are indicated with an asterisk (*).

(Must be between 3 – 31 characters.)

By clicking Submit, you agree to the developerWorks terms of use.


All information submitted is secure.

Dig deeper into Cloud computing on developerWorks

  • developerWorks Premium

    Exclusive tools to build your next great app. Learn more.

  • Cloud newsletter

    Crazy about Cloud? Sign up for our monthly newsletter and the latest cloud news.

  • Try SoftLayer Cloud

    Deploy public cloud instances in as few as 5 minutes. Try the SoftLayer public cloud instance for one month.

Zone=Cloud computing, Security, DevOps
ArticleTitle=Make PaaS your vulnerability testing ground