It’s pretty understandable that a tech person likes hands-on work and doesn’t like any related overhead, including documentation. Similarly, penetration testers love finding vulnerabilities and much less like reporting them. However, the business value comes not from the finding itself, but from its proper communication to the client and actionable remediation measures that may help fix it. So, the report is as important as the finding, not saying that it’s, in fact, the only tangible deliverable of an appsec assessment.
We’d like to give you an overview of the path you have to take on your way to a mature report. We’re not yet on the peak of this mountain ourselves; however, we can see the route from where we are now. So, let’s take a look at it.
The basecamp
Even if you don’t issue a formal report on your work, you still communicate your findings. Probably in email or a messenger, hopefully with end-to-end encryption. When tired of the counterpart not getting your point, you start structuring the description, providing steps to reproduce the bug. You begin to add remediation recommendations, sometimes even tailored to the customer. And when the email or the message becomes way too large to cover a single finding (and hopefully you usually have more than one), you conclude you need a report.
Khumbu icefall
Now when the customer starts treating you as a vendor rather than a freelance guy, you realize your report structure needs some improvements. Creating an executive summary will allow a higher-level manager to read at least one page in your report, hence having more confidence to approve the budget for your next project. Also, you start getting questions about the methodology you use. Of course, at first, you think your “own” methodology is much more flexible and comprehensive.
However, after having to explain your terms and risk metrics several times, you realize it’s better to learn how to map your “bright” approach to a “faded” industry standard. And chances are your choice will be the OWASP TOP 10 in the beginning, allowing you to put anything uncertain into the A6: Security Misconfiguration.
Camp 1
At some point in time, you realize the beauty of the OWASP approach and start delivering some developer awareness training. You still feel you may need more detailed guidance on the findings, especially risk descriptions and mitigations. One day you decide to extend your findings taxonomy to CWE and SANS TOP 25. Sometime later, you start using a great thing called the OWASP Security Testing Guide. (Surely you start reading guides only once you think you are already a professional, not before that). Then you realize that everything you wanted to tell your junior colleague is already written.
Of course, as you previously, your juniors don’t want to use a guide before becoming professionals. Then you implement a magical tool called the checklist so that they can’t escape it.
Camp 2
After another cold customer feedback on your report having too few findings, you decide that sharing the checklist with the customer would make it easier to show you’ve done hell lot of work. And that the customer should be glad, not sad, that still there are so few findings. Of course, a curious customer starts asking why specific findings don’t match the checklist and vice versa. Also, you realize you need more risk levels than high, medium, and low since some of the checklist’s findings are even lower than low. You cross-map findings to the checklist and checklist to the findings, and your report becomes consistent.
Camp 3
Being tired of arguing about the issue risk level, you start using the NVD CVSS calculator. (Which opens a whole new argument about which of its versions to use). You add a description of the overall attack narrative to the report, and finally, your customer appreciates you reporting all those info-level vulnerabilities seeing how they enable the attack. (In case they don’t, but you feel that they need to be addressed anyway, you just make them shine a little brighter.) Since you have a lot of experience at this point, you finally implement a pentest reporting tool beyond MS Word.
Camp 4
Time to automate. Your experience converts into templates; now, your report is half ready just after the reconnaissance is over. And it appears that it’s faster to remove irrelevant things from your templates than to create the parts of the report from scratch. You feel like you are ready to open your reporting tool to the customer, not just deliver the report in a PDF. You hesitate for a while but still do it. Now your deliverable is not just a file, and it makes a difference to your customer.
The Summit
As usual, the last mile is the most difficult and least predictable until you’re there. However, you already see the vision of an automated solution: instead of providing a report you provide an API, the customer has the remediation action plan automatically created in Jira, resolved Jira issues trigger a retest, and some of the vulnerabilities are even retested automatically. Your customer literally has a dashboard of their security level how it changes, with breakdowns to particular applications and business functions.
Do you like the vision of the peak? We hope you do. Where there is a goal, there is a way. And we do wish you luck and patience on your way and hope to see you somewhere at the summit.