We use cookies to improve your experience and optimize user-friendliness. Read our cookie policy for more information on the cookies we use and how to delete or block them. To continue browsing our site, please click accept.

Case Study: How 16 Penetration Tests Missed a Vulnerability

Case Study: How 16 Penetration Tests Missed a Vulnerability

This Could Have Cost One Company Over $103 Million in PCI Fines

In a recent enhanced red team/advanced penetration test, our team of testers uncovered a major vulnerability in a client’s network. This vulnerability gave them access to data, which had been there since 2012. If our team had been a group of hackers, this breach would have cost the company over $103 million in PCI fines alone.

The interesting fact about this study is that the company had been getting “penetration testing” quarterly every quarter since 2012 by various notable companies. We uncovered the information in the 4th quarter of 2016.

That is a total of 16 penetration tests by 7 different vendors that missed the vulnerability.

How Did 16 “Pen Tests” Miss This Vulnerability?

Because of the way they are being tested. Each penetration test prior to ours had relied heavily on automated tools to identify vulnerabilities. The pen testing teams would run automated scans and then perform manual tests of the results. The problem with that is automated tools only look for publicly known vulnerabilities in systems – leaving vulnerabilities in custom applications or undiscovered “zero day” vulnerabilities unidentified.

Most cyber risks are hidden.

Similar to an iceberg, most vulnerabilities are hidden from automated and compliance-driven vulnerability scanning and penetration testing. Taking an enhanced red teaming approach to advanced penetration testing finds risks “below the surface” by manually emulating the aggressive actions of a hacker. The Withum Cyber approach involves human cyber operations experience, tools, tactics, and procedures at each stage of the test. It has been determined, by comparing test results for organizations that have employed multiple testing methodologies, that applying deep hands-on technical experience towards finding organization-specific vulnerabilities is a truly comprehensive way of identifying and analyzing a network’s level of security.

What is Enhanced Teaming?

An enhanced blue team approach to advanced penetration testing emulates the activities that advanced persistent threat actors (such as nation-state threats or organized crime) would carry out against your organization. Beyond a scan for vulnerabilities, this advanced level of testing takes advantage of the training, experience, and adaptability of our penetration testing specialists in finding, exploiting, and leveraging vulnerabilities to gain access and determine the impact of that access on the organization.


Vulnerability Assessment Traditional Penetration Enhanced Blue Teaming/ Advanced Penetration Testing
Scoping Limited Limited to scan results Comprehensive
Skill Level Required Tutorial Needed Training Required Advanced Degree
Objective Broad scanning for information gathering Utilize broad scanning to manually test a network for compliance driven needs Uncover as many vulnerabilities as possible using the resources leveraged by real attackers
Techniques Fully automated using software which identifies publicly known vulnerabilities Driven by automation with penetration testers manually testing the findings uncovered by automated scanning Human driven with a team of hackers focused on your network identifying vulnerabilities unique to your network
Threat Emulsion None Partial Advanced Persistent Threat Emulation
Reporting Computer generated report with unverified information and no determination of business impact Computer generated report which is verified by penetration tester reducing the amount of false positives Narrative report with actionable remediation steps and verified intelligence determining the business impact of all findings

It is important to understand the difference in the complexity and depth of testing levels, and why Withum Cyber uses an enhanced red team approach to penetration testing.

Key Learnings

  1. There is a vase difference in definitions of “penetration testing.” Make sure you understand the difference in the level of testing you are receiving.
  2. As cybercrime continues to grow and being an increasing threat, you must start to conduct more comprehensive testing in order to truly remain secure and build your cyber resilience.
  3. Becoming a “want to know” organization and proactively looking for threats and vulnerabilities is imperative.
  4. An enhanced blue teaming approach to penetration testing is the only way to uncover organization specific vulnerabilities.

If you have any additional questions or would like more information around penetration testing or Withum’s Cyber & Information Services team, please fill out the form below.

How Can We Help?

Previous Post
Next Post
Article Sidebar Logo Stay Informed with Withum Subscribe

Get news updates and event information from Withum