Requests for the black-box pentests are the most common inquiries we at BSG get from potential clients. And our semi-automatic answer always is: are you sure? Companies seeking black-box security assessments can often be misled by incorrect marketing information. And we must correct their perception about these services before they make a mistake and spend their time and budgets on achieving the wrong security goals.
The disclaimer
It is a place for a disclaimer here. We are a cybersecurity firm doing pentests, among other things. Of course, we might be biased in our assessment, and our clients may have different opinions based on their security goals. For some, it is compliance, for some, it is a regular check-up, and for some, it is something beyond our imagination. So it is fair to highlight that our perspective is shaped by our own goal: the highest possible value of our services to our clients.
Black Box application pentest is useless
It may sound like a bold statement, but I insist. Building software is complex, and doing it in the dark would be even more complicated. Security testing gives software developers visibility about their product’s attack surface. Disregarding this information breaks the learning feedback loop and leaves developers blind to security threats. Limiting the mode of security testing to a black box is equivalent to turning off the lights.
Black box testing assumes the testers have limited initial knowledge about the scope of assessment, which often amounts to zero. It also usually means that pentesters have zero access permissions in the application at the beginning of the project. Unless the application has users self-registration, security testers have to brute force their way in or get their way around the login mechanism.
In any case, this allows us to test only a limited sector of the attack surface. It has authentication and security configuration in it, and basically, that is it. Is this kind of security testing worth spending time and money on it? Arguably, no. Is it a challenging task for pentesters? Definitely, not. So what is it good for except for fostering the low-quality segment of the pentesting services market? I have no idea.
Black Box infrastructure pentest is useless unless the scope is full
Now let us get to another extreme. We often get requests for the “classic” network pentest of a bunch of IPs. When we start preliminary reconnaissance, it turns out that the client’s infrastructure is much larger than that. Such an approach is another example of inefficient application of time and resources. No cybercriminal will limit their attention to a portion of the target’s attack surface. No ethical hacker should either.
Of course, limited scopes should undergo security assessments too. It is a great idea to pentest a particular system or do a regular vulnerability scan of a certain subnetwork. These are all excellent security exercises. What I imply is that these narrow security practices can and should be performed by internal staff. There is no need to engage high-quality professional services to do it.
The reason a company needs a pentest is simple: to get an independent assessment of the effectiveness of their security effort. The reasons a company thinks they need a pentest may be different. The reasons a company could want a pentest are limited to one’s imagination. While virtually anyone with a little bit of experience can do an unsophisticated vulnerability assessment, my professional concern is that pentesting experts are scarce. And companies need to apply them accurately.
It could be counterintuitive to learn that sometimes service providers would rather refrain from engaging with certain clients than do so on the clients’ terms. However, it is often the case with BSG. We are hesitant to do jobs just for money and we always seek to provide value. To the point where we reject customer requests of narrow scope black box pentesting or something like that.
The bottom line
With all that said, what could be a takeaway lesson? The general rule behind my reasoning is: the narrower the scope – the deeper the visibility; and the shallower the visibility – the broader the scope. You either want depth, or breadth, or both; otherwise, you may not need security testing; you may need something else. And if you do, for a high-quality service provider, it is a clear tell that they shall pass.
Take care.