This Vulnerability Scanning vs Penetration Testing guest blog is authored by Holly Williams, Technical Director of cybersecurity firm Secarma.
I occasionally see the terms vulnerability assessment and penetration test used interchangeably, or even phrases such as ‘automated penetration test’ thrown into the mix. This is something that really pains me because they are all very distinct types of assessment.
In this article, I’d like to clarify the distinction between the different types of assessments. Setting aside any argument of specific terminology, I aim to explain the different approaches that can be taken and the aims of each – regardless of what you choose to call them. I also aim to assist you in engaging with security assessment providers to ensure that the service you’re getting is what you are expecting, and to make you aware of the alternatives.
On the left of the testing spectrum, we have a vulnerability scanner. This is a tool with which you can supply a list of IP addresses or URLs, and potentially a set of administrative credentials.
This tool scans your system for vulnerabilities, then scores these vulnerabilities in order of priority. Scoring could be something bespoke for that scanner or, more likely, it will be something like the Common Vulnerability Scoring System. There are some well-known tools in this space; SAINT, Nexpose, QualysGuard and Nessus, to name but a few. Such systems generally have two ways of determining if a system is vulnerable.
The first way uses signature-based systems, which rely on a database of signatures for known vulnerabilities. For a scanner to recognise a vulnerability, a signature for that specific vulnerability has to be added to its database first.
Then there are exploitation-based systems, where a more generic payload is supplied and a result is observed – a good candidate for detection in this way is command injection, however, generally that is the point in which the line is drawn: prove the vulnerability and move on to something else.
These systems are entirely automated and aim to identify as many vulnerabilities as possible across the breadth of an organisation, presenting them individually in priority order. They’re not there to fix the issue, just to notify you that there is one. As breadth is the aim, it stands to reason that a white-box approach should be taken with contextual information and credentials given to the scanner wherever possible, to ensure as many issues as possible can be discovered. The output is most likely to be a lengthy list of independent configurations issues and missing patches.
Note: White–box testing is a method of software testing that tests internal structures or workings of an application, as opposed to its functionality (i.e. black-box testing)
Penetration tests are human-led engagements that are goal-orientated. Generally the intention is to chain together discovered vulnerabilities in a depth-first assessment of the network. This chaining of vulnerabilities is something that you don’t get with vulnerability scanning. The line is drawn at a full-system take-over and therefore these assessments involve a degree of exploitation and vulnerability chaining.
While pen test assessments may utilise automated tools to ensure efficient testing, the aim isn’t a complete list of all vulnerabilities on the system, it is instead to discover more realistically how far into a system an attacker could go.
With vulnerability scans, a reduction in scope (or “sampling”) could be acceptable to ensure scans can be performed in a timely fashion. With penetration testing, sampling of this nature is rarely useful and can often lead to artificially impeding the assessor, such as by preventing an accurate indication of vulnerabilities (for example, if a database is the target but the active directory authentication service it uses is out-of-scope).
Penetration testing should be conducted with an informed defensive team and set up in such a way that systems can be efficiently attacked, such as supplying an assessor target IP addresses. This may be contrary to some people’s beliefs where they feel the defensive team should not be informed. However, assessments should aim to accurately determine the effectiveness of a single aspect of security at a time.
At the end of the day, you’re assessing the security of your systems, not the responsive capability of the defensive team. Further to this, a penetration test is generally best suited to a company that already performs vulnerability assessments and has managed most of the identified risks presented by automated tools. The output is most likely an example path or small number of paths an attacker could take to fully compromise a specific aspect of a company’s systems.
Testing is a spectrum where there are various time, cost, tactic, and outcome expectation differences with each assessment type. Assessment such as red teaming are only beneficial to companies who have already addressed risks more easily identified. Although these assessment types can be combined effectively e.g. quarterly vulnerability assessments and annual penetration testing.
The main takeaways from this are that these assessments are not interchangeable, and that there are specific and critical differences.
It should also be noted that penetration testing, and certainly red teaming, cannot be an automated assessment.
There is no problem with a tester utilising scanning tools and testing tools to make an assessment as efficient as possible. But someone simply clicking “Go” on Metasploit Pro is not a penetration test.
Penetration tests are technique-driven and not tool-driven, with a requirement to adapt contextually to the systems as presented, while having the ability to compromise bespoke and non-standard deployments. Therefore be wary of any assessor who informs you a penetration test can be driven purely through automated tools like Nessus, or a salesperson trying to sell you “automated penetration testing”.
Identify vulnerabilities before they’re exploited with Penetration Testing from UKFast and Secarma.