What is a black-box penetration test, and how come many people get it wrong? What is a white-box pentest, and how is it different from a black-box? What is gray-box testing: a point on a spectrum between black and white or a combination of two? I bet you do not know the correct answers to all of these questions. But do not worry: no one does.
Why do we even have a question?
It is super weird to see how many cyber security professionals misunderstand these terms, let alone the general public. Some unintentionally, caught in the mess created by the cybersecurity firms copywriters. Some deliberately attempt to mislead potential customers and shape the market the way they want it to be. The plethora of opinions and claims about the white-box, black-box, and grey-box pentesting is why we have recorded this webinar and put it out on YouTube.
The wrong way
Well, so what is wrong with the false perception of black-box, white-box, and grey-box pentesting? It is hard to name any particular misconception, as it is usually incorrect. Many people, the laymen and cyber security experts alike, genuinely believe that:
- Black-box testing is timiting the initial access of the pentesters and gives them access to the functions publicly available “to the world”
- White-box testing is removing all the limits of access to functions and information about the systems pentesters assess
- and Grey-box testing is something in between: giving pentesters access credentials to some roles and functions, and so on
In fact, even on the Wikipedia-level of understanding, this kind of thinking is profoundly incorrect and severely flawed. Before we discuss its dire consequences, let us share what these types of testing are.
The right way
Now, let us describe the correct definitions of black-box, white-box, and grey-box penetration testing. But first, we need to fix the major mistake people make when interpreting these terms. You may think of them as the modes of penetration testing that limit pentesters in their initial access to the system, network, or application. And this is inherently incorrect.
The correct interpretation of the “box color” is not the access rights pentesters have but the knowledge about the system. See, black-box means pentesters literally know nothing about how the system works. Dut they still have access to all its external functions and interfaces so that they can figure it out.
White-box, on the contrary, means that pentesters have all access to all available knowledge about the system. Yes, including all the documentation, responsible personnel, such as software developers and system administrators, and the application source code. However, white-box does not mean the testers have access to the system as it runs in production, stage, or testing environment.
And what is the grey box? Unlike many people believe, grey-box pentesting is not a point in the continuum between zero access and full access to the system. Many mistakingly think that the security testers are given some but not full access to application functionality in a grey-box pentest.
As we have just clarified, the box color has nothing to do with access; it is all about the knowledge. So maybe the grey box means that pentesters have some limited knowledge about the system? Again, wrong. Contrary to popular belief, and I bet you may be surprised as well, grey-box pentesting is not a balance between black-box and white-box. It is a combination of the two.
Yes, you have read it correctly. In grey-box pentesting, we do not have limited knowledge, limited access, or any limitation at all. The Grey-box mode of pentesting means that pentesters have full access to all application functions and information of its internal workings, including the source code.
OK, people put their meaning into obscure cybersecurity terminology only a few hundred thousand people on Earth use anyway. Why is it a big deal? There are three areas where things can go wrong because of this poor understanding of basic definitions.
Well, the first risk is that security standards, regulations, and best practices have a specific meaning behind these terms. And If a standard or guideline does not redefine the terminology, its requirements and recommendations mean specific things. And this meaning is much closer to the standard definition than what people imagine it to be.
In other words, if a standard, such as ISO 27001 or PCI DSS, requires a company to have an annual third-party black-box penetration test of its infrastructure and business applications, it means that the pentesters have to be given access to all appropriate systems and functions. In reality, the misinterpretation of the term results in pentesters having just a general idea of what and where the scope is. They often still can penetrate the system or network, which says much about the level of its cybersecurity, but they rarely are as productive during the pentest as the creators of the standard meant them to be.
Then, the second risk is that the wrong understanding of terminology shifts the organizations’ security objectives. Instead of making their infrastructure elements harder to hack, companies focus on bringing more secrecy and obscurity into the equation.
The problem with obscurity is that it can protect you only for this long. And when the system’s inner workings become known to the attacker (and it is a question of when not if), the system will be easily hacked. Simply because its creators relied too much on the secrecy of its design and implementation and did not leverage white-box security testing in the SDLC.
And the third problem that impacts BSG directly is the wrong perception of a fair penetration test price in the market. The wrong understanding of a black-box pentest creates a false sense of its brevity and straightforwardness in the clients’ heads.
Uncomfortable for many, black-box pentesting is not just brute-forcing the login form and scanning the network for known vulnerabilities with optional exploitation. It is a comprehensive analysis of all possible ways to provide input to the system and then use those entry points to find, diagnose, validate, and contain the security flaws. Apparently, this kind of work needs much more skills, time, and energy, than riding a commercial penetration testing tool.
On the other hand, white-box and grey-box pentesting have their aura of something large, long, and cumbersome. They obviously require the pentesters to go through all the available functionality and the application source code to complete the testing. In fact, all available functionality must be tested in a black-box pentest as well, and access to the source code does not extend but contracts the time needed to complete the job.
You see, given access to the source code and other information about the system’s internal workings, pentesters can be much more efficient as they do not need to spend a lot of time to diagnose each vulnerability. Instead, they can open the source code and directly see how software developers implement the corresponding functionality. It saves us tons of effort and a lot of time, allowing us to complete a day’s work within an hour.
Thus, the webinar
So this is why we have decided to record a video where we go through the significant misconceptions about the modes of pentesting and debunk them one by one. We hope you enjoy this video and learn from it. And most importantly, we hope you will share it and pass the word on.
Thank you to all of you who have attended our webinar! It was an excellent experience for the hosts and a fruitful discussion for everybody. We have covered a lot of ground and cleared out our professional opinion. And most importantly: we have put it on record, so now we can direct the interested audience to the YouTube video below. The slides are also available at this link.
The penetration testing efficiency debate is far from being over. But we believe we have made a good contribution, dissolved a few misconceptions, and clarified a few uncertainties. We will continue to learn from our professional experience. If our views change, we will update you in a future webinar. So stay tuned, subscribe to this blog, our social media accounts, and our YouTube channel.
Also, at the end of the webinar, we had some time for a mini-lottery. We have identified the winner of a YubiKey that we are eager to send out; however, we do not know where. So Andrii, please check your email spam folder and help us with the logistics if you read this. Thanks!