The Numbers Game

A new way to test as application estates grow

Enterprise applications are now an essential foundation for business success. But do we have the right security measures in place to match their growth in numbers and speed of development?


Organisations are moving faster than ever before to adapt, seize new opportunities, and launch entirely new business models. Development of new applications and websites are at the heart of this change.

Businesses today are desperate to innovate. The big names often quoted as examples of the new “platform economy” include the likes of Amazon, Netflix, and Uber. But in truth, almost all companies across all industries are now heavily reliant on applications to compete and succeed.

Development teams have risen to this challenge as well. DevOps and Agile approaches have revolutionised the speed of application innovation and the frequency and volume of in-life changes.

The result has been an explosion in the number of businesses applications to help organisations adapt. Research by McAfee, for example, suggests companies with 50,000 employees or more now operate 788 custom applications, on average. Even smaller companies (1,000 employees or less) use at least 22 custom applications to do business.

... every new or updated application is a primary target for hackers.”

But in this brave new world comes a severe, unavoidable problem: every new or updated application is a primary target for hackers. Web-based applications are readily available for invisible actors to probe and investigate 24/7 from the comfort of their own home or, as the cybercrime “industry” grows, from the comfort of their offices. With this in mind, it’s no surprise that application vulnerabilities remain a key route used to disrupt businesses or gain access to private (and valuable) data.

That’s why testing for vulnerabilities - finding errors in the applications themselves and the configuration of the underlying infrastructure - is so essential before the bad guys get there first. But with this application revolution comes an important question: has security testing kept apace?

In other words, can the average security testing strategy handle today’s application volumes and speed of development?

Our view is simple: no.

That’s why we have published this two-part series to explore the issues.

In this first instalment, we focus on The Numbers Game - the need for teams to take a different approach to traditional penetration testing as the volume of enterprise applications they face is... well, testing them to the limits.

Also available: Penetration Testing Re-engineered: Part two

At company after company, fundamental tensions arise between the business’s need to digitize and the cybersecurity team’s responsibility to protect the organization.”

— Source: Cybersecurity: Linchpin of the digital enterprise, McKinsey & Company

Section one:

Application Proliferation

Digital transformation may have taken a leap in recent months for many sectors, but it has made steady progress over all industries for decades.

The result, according to McAfee:

“The average enterprise has 464 custom applications deployed today. Across industries, even ones not associated with technology, companies of all sizes are developing applications that help them engage with customers, suppliers, and employees.”

This presents serious challenges for the security team. Some applications may be developed in-house, some by third party developers, and many will be based around off-the-shelf commercial software. But every one offers an attack surface for cyber criminals, especially if there are coding errors, deployment misconfigurations, or issues with the infrastructure they run on.

Be they script kiddies, automated malicious tools such as malware, or persistent skilled adversaries, the volume and range of attacks onto application estates has also been steadily rising. With 43% of breaches focused on web applications, according to Verizon, teams need to change the way they approach penetration testing.

And they need to change fast.


The number of new applications average enterprises will develop in the next 12 months.


The average number of custom applications run by businesses of 1,000 employees or less.


The average number of custom applications run by businesses of 50,000 employees or more.

— Source: Every Company is a Software Company, McAfee


Data breaches analysed this past year that were the result of a web application vulnerability.

More than double the amount in 2019.

— Source: 2020 Data Breach Investigations Report, Verizon

Section two:

Fire Fighting

Against this growing threat and expanding target, security teams have been deploying a number of tools and techniques in a desperate attempt to keep up.

Even now, ‘keep up’ is still an aspiration for many. Any hope of ‘getting ahead’ is a distant dream for all but a lucky few. That’s because the traditional approach to securing an application - and its supporting infrastructure for vulnerabilities - remains the pen test.

A well-executed penetration test has always been an essential part of counter-acting potential cyber threats, using a combination of tooling and human skill that no automated system can replicate in full.

But there is a problem.

A typical application pen test takes 3-15 days (and sometimes more). It requires a skill set that’s in short supply and expensive to hire. For even a moderately sized application estate, testing every application - even on a 12-month cycle - is out of reach for all but a few organisations.

The cybersecurity skills shortfall


The number of additional cybersecurity professionals needed to meet global demand


Organisations struggling to hire cybersecurity professionals


Organisations at moderate or extreme risk due to a cybersecurity skills shortage

— Source: Cybersecurity Workforce Study 2018, ISC

Section three:

Fanning the Fires

The result is that security teams have had to adapt. But are machines the answer?

Today, there are many application and infrastructure scanning tools they can deploy, from open source to expensive vendor products—all lay claims to their effectiveness. Most can offer some benefit, some can offer significant benefit, but few can be integrated and configured without a lot of work. And none are as effective as a skilfully deployed “human” penetration test.

This forces some difficult choices: Which applications do I fully pen test? Which do I only scan, and with what tool?

Then there’s another problem, because all of these approaches are normally structured on a per application-based approach. On the surface, yes, it makes sense to break the problem down like this. But that is not how an attacker sees you.

From the outside, any route in can lead to something interesting. So while you may test your business-critical applications, it could be that the small www site you’ve forgotten about (or were even not aware of) becomes the route your attacker is looking for to infiltrate your entire system.

In other words, tightly securing your most important applications is not enough. If you are a medium-sized business with 22 applications, or an enterprise running 788, you need a considered testing strategy for the estate as a whole.

The application that is not on your radar could be the back door for hackers to enter your entire system.”

Applications: the open source dangers within

Since 2015, Black Duck has published its Open Source Security and Risk Analysis report (OSSRA) looking at the quality and use of open source code and libraries. As they explain, “ source components and libraries are the foundation of almost every application in every industry.”

This is important because the use of open-source elements has been key to development teams, increasing the volume of output. But there can be issues lurking in these components and libraries.

In its 2020 report:


Applications audited


Codebases containing open source components


The percentage of open source components found to be out-of-date


Codebases audited in 2019 containing at least one vulnerability, compared to 60% in 2018


Audited codebases containing high-risk vulnerabilities, compared to 40% in 2018

Whether you are a member of an IT development, operations, or security team, if you don’t have policies in place for identifying and patching known issues with the open source components you’re using, you’re not doing your job.”

— Source: Open Source Security and Risk Analysis Report, Black Duck

Section four:

A need for change

Whatever balance teams get to with their testing approach, there’s no denying we are failing.

Breaches are on the rise, and application vulnerabilities are being discovered at an increasing rate.

As the UK government’s latest Cyber Security Breaches Survey reports, 32% of UK businesses experienced a cybersecurity breach or attack in the last 12 months, and 48% identified at least one breach or attack every month.

As already mentioned, Verizon estimates that 43% of breaches targeted web applications over the last 12 months, more than double the number from the previous year.

... With application estates growing at a faster rate than ever, the answer can’t be to pen test every release. On the other hand, automated scanning isn’t the solution either. Machines don’t “think” like a hacker.”

But the answer can’t be to pen test every release of every application either. There just isn’t the volume of people needed with the right level of skills, even if budgets were unlimited.

On the other hand, automated scanning isn’t the solution either. Whereas it may be tempting to deploy application and infrastructure scanning tools to deal with the coverage challenge, the reality is that there are many types of vulnerabilities that scanning tools still fail to detect. Just as important: machines don’t “think” like a hacker.

That’s why we need to re-engineer the way we test our applications.

One that is led by expert penetration testers working as a team, and also includes automated scanning to address the challenge of testing today’s large application estates.

The road to pen testing re-engineered

So far, the choice for security teams has been to either deploy manual penetration testing, or application scanning using tools.

As already mentioned, manual pen testing is far more thorough but can’t be run over all applications, all the time. Scanning tools have the advantage of being able to be run regularly and over large estates, but they can’t detect every issue and the outputs they provide need careful triage by a skilled resource.

This is where our work with many customers facing these issues has evolved into a new approach to the security testing of large application estates.

Combining skilled manual penetration testers, working as a team on an ongoing basis, with scanners tuned to cover web applications, has proven to offer a continuous testing plan of entire application estates that balances risk and affordability.

This practice has now matured into Continuous Security Testing (CST), a service that also enables security as a service specialists to work in close collaboration with their customers’ development teams.

Rather than simply producing a long list of vulnerabilities once or twice a year, issues are fed in ‘as found’ and testing can be quickly redirected to new releases or changes without having to wait for the next scheduled round of pen-testing.

Continuous Security Testing also allows pen testers to take a broader ‘attackers-eye’ view over the estate - no longer limited by traditional time-boxed assessments - to find exploitable vulnerabilities.

Continuous testing represents a new and more blended way of managing application security. For many organisations, it also enables security teams to work more effectively, and in tandem, with their colleagues responsible for DevOps.

The secret is in the name: Continuous Security Testing. Not an infrequent deep dive into individual applications, or a thin skim over the top of many, but an ongoing combination of specialist human testing and automatic scanning of the entire application estate, with reporting and feedback on an ongoing basis.

In other words, an approach that enables you to think like an attacker… 24/7… 365 days a year.

Allowing you to move fast and stay secure.

A question of coverage

In today’s digital enterprises… the number of assets and processes to protect, and the decreasing practicality and efficacy of one-size-fits-all protections, have dramatically reduced the applicability of traditional cybersecurity decision-making.”

— Source: Cybersecurity: Linchpin of the digital enterprise, McKinsey & Company

Key takeaways:

  • Digital innovation is now an essential platform for business success and market disruption.
  • Which is why business-critical applications have proliferated.
  • But security teams are unable to secure large application estates, and pen-testing approaches have not evolved to keep up.
  • While these new applications provide hackers with multiple entry-points into the entire IT system of their organisations.
  • Traditional (infrequent) pen-testing routines cannot deal with continuous attacks over a large application estate, and tool-based scanning is not comprehensive enough on its own.
  • So, a new routine of application security testing is needed, and has evolved.

Continuous Security Testing in Action

See how Continuous Security Testing can help you adapt to today’s evolving threat landscape.


Web application security review – what you need to know for 2021

Join our expert panel to take a practical look at the latest threats to web applications. Find out how you can identify and mitigate these risks in your own web application estate.

Watch the webinar

Ready to talk?

Call our Security Team on 0800 640 8011, or email, to find out how you can re-engineer the way you test.

See CST in Action - Click here