Miloslav Homer

I've written this article for the code.kiwi.com blog.

I got a message from my friend in the lines of: “Hi! I remember you did penetration tests, we want it. Where can we order some penetration tests?” Dear friend, I sincerely hope to answer your question in this post.

Why ?

Let’s quickly summarize why you would even do such a thing. There are plenty good reasons usually along the lines of:

• We have a critical piece that is handling ultra sensitive data and we need to know the security properties
• We have a requirement for an independent penetration test from a 3rd party such as: The Government, An Insurance company or a key business client.

For a more in depth-answer, please take a look at Part 1.

There will be no direct names and recommendations mentioned in this post and that’s largely because nobody is paying me to promote them. But here are some general guidelines to look for.

Certifications

Pentesting engagements are surprisingly personal. Unless you are assessing a large infrastructure, doing red team engagements (multiple vectors of attack including social engineering), or testing a really big application (I’d use SAP size as an example), chances are that you’ll be tested by a single person.

I don’t think that certifications are the be-all and end-all. That being said, it’s still the easiest way to demonstrate some level of competence. I’d be suspicious if nobody from the testing team, not even some kind of tech lead, has any relevant certifications. These certificates can be expensive, but a good company should treat it as a cost of doing business. I can understand if several members of a team are not certified. Maybe they are new. Maybe they are experienced in general IT matters but new in a pentesting role. Or maybe they are studying hard right now for one such certificate and their exam is scheduled in the near future.

Loads of posts are written (and drawn) about what certifications to have and different skills each one offers. For example, one certification might focus more on hard technical skills whereas another might focus more on management roles.

As an OSCP holder, I know what is the minimum I can expect from my fellow holders, and it’s my personal favorite. There, let’s conclude the certification discussion on this.

Costs and Estimates

Apps of different sizes need different amounts of work, and this is measured in “man-days” or MDs. What one person can do in one workday, or 8 hours, is equivalent to 1 MD.

Beware of the salesman who can offer you a price without seeing your apps and APIs first. That’s a major red flag, and you’d probably only get a vulnerability scan. Using an automated tool that does everything for you will speed up the process a lot for the testers, and slow it down for you, the customer. You’d be dealing with hundreds of false positives that you might not be able to interpret. At best, it would slow you down. At worst, it would produce incorrect roadmaps for your developers, wasting a lot of time and resources.

MDs usually roughly correspond to real-time. This is not always the case — the testers can go on vacation or have a sick leave. In rare cases, the real-time of the test can be slightly shorter than the expected count of MDs. Of course, if a 20 MD test is done in 5 MDs, verify very carefully what was actually going on. An acceptable explanation could be that 4 testers shared the workload.

Methodology

Different assessments require different approaches and methodologies.

Here I’d like to highlight the excellent sources of the OWASP foundation. The “gold standard” of web testing is their flagship project OWASP Web Security Testing Guide. Most of the chapters are also applicable to APIs, so if you are serious about your web app/API then this methodology is the way to go. There is another methodology for mobile apps, the OWASP Mobile Application Security Testing Guide that covers apps in iOS and Android.

You might come across testing according to OWASP Top 10. Please note, this is not a testing methodology, this is a consensus about the most common vulnerabilities and issues. You could use a testing methodology and focus only on these vulnerabilities, but I’d recommend “not stopping at 10” and trying to discover all of the issues. Still staying in OWASP territory, there is OWASP Application Security Verification Standard. This is a more compliance-based approach and it’s a great free resource, unlike, say, ISO27k. I’ve never seen an application compliant to the highest level of all categories, so don’t be scared when reading this.

Moving away from OWASP, there are plenty of different standards and certifications, this time for the app/company. These are not really in the scope of this section, although I’d like to specifically point out one — the PCI-DSS (Payment Card Industry Data Security Standard). You might use pentesting to verify your compliance with these requirements, although they are practically speaking, inapplicable for anything other than payment cards. But still, payment card data of customers is quite often the most sensitive information you’ll be processing.

Example Report

The fortunate fact of B2B relationships is that you are in a position to request additional goodies. I’d recommend requesting an example report. You shouldn’t get a real report — if you do, that’s a major red flag. You should have an NDA as this data is sensitive to your company — “Hey, everybody, look here, this is how to exploit our apps!!” So expect a non-descript example application that is vulnerable to pretty much everything.

Look at the management summary — would this information be of benefit to the management team? Would you, as a security professional, be able to use this directly to get support from management?

Look at test details — is the timeline included? Is there a simple overview of the target, used accounts and others?

Look at the vulnerability details — is the proof-of-concept fully included? Would you be able to reproduce the vulnerability from these descriptions? Would the included data help the development teams in bug specification? Is there a remediation recommendation? Does it make sense?

The example report might not have all of these present. However, that doesn’t mean the team won’t provide additional custom entries. A good company should be able to customize their report within reason. You are now better equipped to request the entries in the report that deliver value to you.

And last but not least — the example report might be your only shot at differentiating companies that do penetration tests from companies that only scan for vulnerabilities.

Concluding — TL;DR

When choosing a vendor for pentests, take a good look at certifications of their team members, ask about their methodologies and request an example report (don’t be afraid to ask for customizations in your final report).