Tuesday, 29 July 2014

Planning and Managing a Penetration Test Project

In a previous post on this blog I have highlighted the main changes introduced by PCI DSS 3.0 vs. the previous version 2.0. One of the most relevant is the Requirement 11.3 which obligates organizations that process, store, or transport
credit card data to implement a methodology for web application Penetration Testing. This is a recurring commitment—not once and done. This testing must be performed when there is significant change and at the very least yearly.

The requirement now specifies that penetration testing activities (internal and external) must follow an "industry-accepted penetration testing methodology," such as the specifically referenced NIST SP 800-115,Technical Guide to Information Security Testing and Assessment. In essence, since Penetration Testing is an outsourced task (no company can effectively test their own security status, and even less certify it for compliance), when selecting a Penetration testing outsourcer, organizations must make sure to obtain assurance that the company will follow an industry-accepted penetration testing methodology.

PCI DSS requirement 11.3 allows you to either outsource the Penetration test or do it internally; if you choose an internal security assessment, the penetration tester must be able to prove expertise in this area (e.g., training certification) and must be organizationally separate from the people managing the network that is being assessed (and this can be quite challenging).

As you will continue reading this article you will see that there are more reasons than PCI compliance for planning for a penetration test.

So what is Penetration Testing exactly?

Penetration testing, often called “pentesting”,“pen testing”, or “security testing”, is the practice of attacking your own or your clients’ IT systems in the same way a hacker would, with the objective to identify security holes. Of course, you will do this without actually harming the network. The person carrying out a penetration test is called either a "penetration tester", "pentester", "ethical hacker" or also "white hat hacker". He/she will use tools and techniques agreed upon with the company hiring him/her to carry out the penetration test.

And here is a point to be made crystal clear: Penetration testing requires that you get permission from the company or person who owns the system. Otherwise, you would be hacking the system, which is illegal in most countries. Therefore, the difference between penetration testing and hacking is whether you have the system owner’s permission or not. If you want to do a penetration test on someone else's system, it is highly recommended  to get written permission before starting any of the activities.

Why organizations need Penetration testing?

Reasons are different and generally limited to these broad areas:

  • Compliance: As mentioned before, some regulations, such as PCI DSS, require penetration tests. Make sure you understand how the penetration test should be conducted to ensure that you will pass the audit.

  • Prevent data breaches: Since a penetration test is a benign way to simulate an attack on the network, you can learn whether and how you are exposed. It’s a fire drill to ensure you’re optimally prepared if there’s
    ever a real fire.

  • Check security controls: You probably have a number of security measures (also known as "Controls")  in place in your network already,  such as firewalls, encryption, DLP, and IDS/IPS. Penetration tests enable you to test if your defenses are working—both the systems and your teams. You can frequently discover configuration errors and process gaps after running a penetration test.

  • Ensure the security of new applications: When you roll out a new application or when you make significant changes to it (i.e. affecting the usage of web services), whether hosted by you or a SaaS provider, it makes sense to conduct a security assessment before the roll-out, especially if the applications handle sensitive data and are somehow exposed to the web. Some example applications includes customer relationship management (CRM), marketing automation program (MAP), HR’s applicant tracking system, health insurance providers’ benefits management software, etc.

  • Get a baseline on your security program: New CISOs often conduct a security assessment when they join a new company to obtain a starting point to perform a gap analysis from which a security program will arise. This shows them how effective the organization is in dealing with cyber-attacks. These security assessments are sometimes conducted without the knowledge of the IT security team because it could otherwise influence the results (more on this topic after).

 What are the typical activities conducted within a Penetration testing?

While nowadays there will be more standardization of Penetration testing methodologies, typically these are the activities which will be be carried out by the external company:

1. Reconnaissance: Finding out as much as possible about the target company and the systems being audited.
This occurs both online and offline.
2. Discovery: Port or vulnerability scanning of the IP ranges in question to learn more about the environment.
3. Exploitation: Using the knowledge of vulnerabilities and systems to exploit systems to gain access, either at the operating system or application level. In other words, if you have identified weaknesses (vulnerabilities), now you check what you can actually do by exploiting them. Exploits talk to systems in a way that was never intended by the developers. However, many exploits are perfectly safe to use on a production system. Penetration testing software such as Metasploit Pro from Rapid7 automatically chooses only tested, safe exploits by default to avoid any issues with your production environment.
4. Brute forcing: Testing all systems for weak passwords and gaining access if they do.
5. Social engineering: Exploiting people though phishing emails, malicious USB sticks, phone conversations, and other methods to gain access to information and systems.
6. Taking Control: Accessing data on the machine, such as passwords, password hashes, screenshots, files, installing keyloggers, and taking over the screen control. Often this can open new doors to more exploitation,
brute forcing, and social engineering.
7. Pivoting: Jumping to different network segments, providing the host has multiple network interfaces, such as
some machines in the DMZ.
8. Gathering Evidence: Collecting screenshots, passwords hashes, files as proof that you got in.
9. Reporting: Generating a report about how the penetration tester was able to breach the network and the information they were able to access. This report normally includes general recommendations of what to do to close the identified vulnerabilities. The tested company will start from those to plan for the Remediation project (see below, bullet #11).


In addition to those, some tasks must also be performed as a joint effort with the 2 companies, testing and tested ones:

10. Identify goals: Setting the objective of the security assessment. This includes identifying the scope, which is generally an exclusive task of the tested company. This obviously is the very first task to perform of the whole Penetration test project, possibly preceded only by the selection of the Penetration testing firm.

11.Remediation: Addressing the issues that enabled the penetration tester to enter the network and outlined in the final report (see bullet #9). This is typically a separate project from the Penetration test one and will employ resources in the IT department.


In my personal experience working for organization requiring a Penetration Test, this effort is not a huge one, however large and complex enough to be treated as a Project. So what are the main tasks to carry out internally when asking an external company to test your infrastructure? In my experience in the role of Project Manager, these are the main ones:

1. Define scope - The testing company will want to know what to test. Generally speaking they will need IPs and some test user accounts (sometimes without password): if the tested company processes credit cards, then also test credit cards associated to test accounts should be provided (this will be challenging because test cards will still result into real transactions in result of the testing and some money transfers might actually occur to simulate an unauthorized transaction). At this stage is also essential to identify the type of approach you want the testing company to follow: you may ask their advise but it is better if your company takes an informed decision based on its objective.

  • a) "Black box" approach: the tester will know nothing about the infrastructure to test. Only information will be the targets (IPs) and an account to be used. This account should not be a power one, rather an ordinary user account. Some testers will not even ask the password of this account as part of their test will be to guess or crack it. If they won't be able to get it, good news for you company: you can now release it to them. This approach is more time consuming. Since the cost of a penetration test tends to be be directly proportional to the effort spent by the testing company, if you are either on a tight budget or on a tight schedule this approach should not be your first pick. However this approach provides the best simulation of what a total stranger can do to your environment, so it is the most recommended approach from a pure testing perspective. If you want to know what an insider with malicious intentions can do, then you need the following approach:

  • b) "White box" approach: the tester will be provided with a good (but not exhaustive: not recommended) view of the infrastructure which will be tested. In other words, you will have to collect some architectural diagrams and full accounts. In this case you will be testing what an insider with malicious intentions (i.e. disgruntled employees) can do. One advantage of this test, and more often than not the reason why is chosen, is that it can be much faster than a Black box approach, as the discovery part is unnecessary. So test duration and cost can be reduced to a great extent, but will the objectives of the test be met? What are you trying to test? Are you afraid of what a remote hacker can do or are you afraid of your own personnel? The answer to these questions will giving you the guidance you'll need as to which approach should be your favorite.

  • I have also witnessed some hybrid approach being used.

One more decision to be taken while scoping is whether to run a full Penetration test, inclusive of vulnerabilities exploitation (with the risk that this approach implies), or proceed in what I have seen called "Safe mode", in which the tested company refuses to face the risks that vulnerabilities exploitation implies (i.e. if you are assessed at risk of Denial of Service attacks, the testing company might actually cause an outage as a result of exploiting that vulnerability). This approach essentially does not differ from a Vulnerability scanning effort and should not be considered a Penetration test. It is also to be noted that with PCI DSS 3.0 this approach is no longer an option, as all industry-accepted penetration testing methodologies include vulnerability exploitations. For other needs, the "safe mode" approach can be chosen but an Information Security professional should warn the tested firm that this type of test will be much less thorough in assessing its security posture.

2. Selecting Penetration testing firm: you might need to research some, or perhaps your company has already a list of them, if not an already identified partner for this outsourced activity. Whatever is the case, if your are under a PCI-DSS 3.0 compliance obligation you will have to obtain and retain evidence that the company will follow an "industry-accepted penetration testing methodology," such as  NIST SP 800-115,Technical Guide to Information Security Testing and Assessment.

For penetration testing consultants, you should ask for references and buy services from a reputable firm. As part of their engagement, penetration testers may get access to data that they would ordinarily not be authorized to see, including intellectual property, credit card numbers, and human resources records. This is why trustworthiness is so important. However, this should not put you off from hiring a penetration tester because the alternative is worse: If you do not identify and fix the security issues on your network by hiring someone who is on your side, your most sensitive data will likely be accessed by someone who is not.

Once a firm is selected, provide them the results of your scope definition with clearly stated objectives and ask them to submit you a written proposal complete of a cost: ideallly you will want a fixed cost, however depending on how vague or complex your requirements might be, the testing company may submit you a fixed price for a specific set of tasks and goals and an hourly rate for extra activities which you may elect to add or not to the agreed upon scope quoted at a fixed price. Once you accept the proposal you are accepting the risk involved: if the risk is not clear to you, you have to ask to have it explicitly described (if not quantified, but this could be challenging to obtain) by the testing firm. Remember, if there will be an outage as a result of their activities and it was mentioned as a possible risk of their testing activities, you cannot claim damages from them.

3. Provide scope to the firm (target IPs and test accounts), alert operations team of what the testing company is going to do to avoid opening unneeded incidents which would affect the company SLA and possibly other KPIs. An informative change request should suffice at this purpose. Note that the IT Security team, which is normally the sponsor of the project, may also deliberately decide not to inform anyone in the Ops team to test how effective the intrusion detection capabilities of the firm are; this approach is not so frequent given the resulting incidents which will cost some money, however there are reasons to justify it.

4. Monitor the execution: while the testing company is performing the test, regular touchpoints might be needed on a daily basis if not more. One of the touchpoint could be before the daily testing start, just to explain to the Ops team what will be done. At the end of the day, another touchpoint may be recommendable to have the testing firm describing what they were able to accomplish and to verify with the Ops team of the tested firm how much of this activity was captured by the intrusion detection systems. In between, the Project Manager should act as single point of contact to resolve potential issues, such as providing missing input to the testing firm, or asking to suspend their testing operations if an excessive load is identified as a result of their activities which may result into a major outage or even just a performance slowdown. In all these activities is advisable to have on call one representative from the Ops team of the tested firm and at least one from the testing firm (at least the tester, but would not hurt to have also the coordinator on their side to be available to make quick decisions).

5. Summarize findings for the Operations team: When the testing firm releases the final report, the resolution of the assigned vulnerabilities must be assigned to an owner (team). For example the resolution of a vulnerability found on a web server running on Apache will likely be assigned to the Linux/Unix Operations team. The Project Manager will have to summarize the problems found during the penetration test, explain them to the application or infrastructure owners (Ops teams) and book with each of the identifies teams a planning session for the Penetration test remediation project, in which the company will put the effort needed to close all the gaps. It is my experience that whoever managed the Penetration Test Project is in a better position to lead also the Remediation project.

Questions? Comments?

Please go ahead here below!


  1. Thank you for sharing this article.

    I am also working on penetration testing project and I ave also started new blog related to software testing. And I think the most difficult task in penetration testing is the selection of the pen-testing services.


  2. Project Management
    I am fully satisfied with the information you have posted. Good job. Keep posting
    Building Information Modeling

    I am impressed with your work and skill.Fabulous content.Thank you so much.Good job. Keep posting

  4. Hi, Great.. Tutorial is just awesome..It is really helpful for a newbie like me.. I am a regular follower of your blog. Really very informative post you shared here. Kindly keep blogging.

    security testing services

  5. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing. Need to learn Security Testing Services

  6. Pretty article! I found some useful information in your blog, it was awesome to read, thanks for sharing this great content to my vision, keep sharing. Need to learn
    Security Testing Services
    Test Automation Services
    Software Testing Services
    Compatibility Testing Services
    Regression Testing Services

  7. That's a very detailed article about planning and managing penetration test. I would also like to add here that you should also need to use proper project management tools like this one to manage such projects.