How to Choose a Penetration Testing Provider: Red Flags, Green Flags, and the Right Questions

TLDR: The penetration testing market has a dirty secret: a significant proportion of what is sold as a “pen test” is an automated scanner report with a consultant’s name on the cover. Cyber insurance underwriters have noticed. Enterprise procurement teams are catching on. This post gives you the eight questions to ask any provider before engaging — and the exact answers that separate genuine expertise from expensive automation.

The Market Problem Nobody Talks About

There is a version of “penetration testing” that looks like this: a vendor runs Nessus or a similar scanner against your application, exports the report, reformats it into a branded PDF, writes a one-page executive summary, and sends it to you as a deliverable. Somewhere between £500 and £2,000. Quick turnaround. Clean layout. Looks professional.

It is not a penetration test. It is a scan.

The distinction matters enormously because the vulnerabilities that cause real breaches — business logic flaws, chained attack paths, broken access control, authentication bypasses — don’t appear in scan reports. They require a human tester who understands your application and thinks adversarially about how to break it.

If you’ve never bought a pen test before, you have no easy way to tell the difference from the outside. So here’s the inside view.

8 Questions to Ask Before You Sign Anything

1. “What tools do you use — and what manual testing do you do beyond them?”

Every legitimate pen tester uses automated tools. That’s not the issue. The issue is what happens after the tools run.

A good answer describes the tool layer as a starting point — Nmap for network mapping, Burp Suite as a proxy for web testing, automated scanners for quick coverage — and then explains the manual phase: the authentication testing, the access control probing, the business logic mapping, the attack chain analysis. They should be able to articulate the difference between what the tools find and what the tester finds.

A bad answer describes only the tools. If the tools are the whole answer, the tools are the whole test.

2. “Can you describe a business logic vulnerability you found on a recent engagement?”

This is the single most reliable question on this list. Business logic flaws require genuine manual testing and genuine understanding of how an application works. A tester who has done real work can answer this immediately and specifically — a race condition in a payment flow, a workflow step that could be bypassed, a discount that could be applied indefinitely.

A vendor selling scan reports cannot answer this question. They’ll give you a generic description, redirect to a case study, or describe a vulnerability that sounds more like a CVE than a logic flaw. The specificity of the answer tells you everything.

3. “What does your scoping process look like before testing begins?”

Real engagements start with a scoping call. The tester needs to understand what they’re testing: the application’s purpose, its user roles, its technology stack, its highest-risk features, what’s in scope and what isn’t, the rules of engagement. This conversation takes time and it shapes the entire engagement.

If a vendor can quote you a price and start testing without a detailed scoping conversation, they’re not running a bespoke test. They’re running a template. Understanding what a real pen test involves from the start is the foundation of getting value from one.

4. “Does your report include attack path mapping, or just individual findings?”

As we covered in the post on vulnerability chaining, individual severity ratings are incomplete. A real engagement identifies not just vulnerabilities but how those vulnerabilities combine to produce real-world attack scenarios.

Ask to see a sample report. If the report is a list of findings with CVSS scores and remediation snippets — and nothing that shows how a real attacker would move through the application — you’re looking at a scan export, not a manual assessment.

5. “Do you include a re-test after remediation?”

A pen test without a re-test is incomplete. After you’ve fixed the findings, someone needs to verify the fixes were applied correctly and didn’t introduce new issues. Good providers include at least one remediation re-test as part of the engagement. Some include it automatically; others offer it as an add-on.

If a vendor doesn’t mention it at all, ask directly. The answer tells you whether they think in terms of security outcomes or just deliverable completion.

6. “What methodology do you follow — OWASP, PTES, NIST?”

Reputable providers structure their testing against recognised methodologies: the OWASP Testing Guide, the Penetration Testing Execution Standard (PTES), or NIST SP 800-115. These aren’t just credential-signalling — they ensure that the engagement covers the right areas systematically rather than relying on whatever the tester happened to think of.

A tester who can’t name the methodology they follow, or who says “we have our own proprietary approach” without being able to describe what that covers, should be pressed further.

7. “Have you tested applications similar to mine in complexity and stack?”

A tester with experience in SaaS platforms understands multi-tenancy, subscription logic, API-heavy architectures, and the specific failure patterns those produce. A tester whose background is entirely in network infrastructure may miss application-layer nuances.

You don’t need the vendor to have tested your exact product. You need evidence that their experience map overlaps meaningfully with your application’s risk profile.

8. “Can I see a sample report — redacted if necessary?”

This is the clearest signal of all. Ask to see a real deliverable. A quality report includes an executive summary written for non-technical leadership, a technical findings section with evidence and reproduction steps, clear severity ratings in context (not just raw CVSS), attack path narratives, developer-facing remediation guidance with code-level specificity, and a re-test section.

If the vendor refuses, or if the sample looks like a formatted export from a scanning tool, you have your answer.

Red Flags: What to Walk Away From

Extremely fast turnaround with no scoping call. Real testing takes time. A comprehensive web application assessment typically requires several days of testing time, minimum. A 24-hour turnaround is almost certainly automated.
Price so low it couldn’t cover meaningful manual time. Skilled security professionals are not cheap. A test priced below what a day of a senior consultant’s time costs cannot include significant manual effort. You get what you pay for — and in security, “not quite enough” is worse than nothing because it creates false confidence.
Report is a raw scanner export. Generic CVE descriptions, no application-specific context, no attack paths, remediation advice that’s clearly copy-pasted from a vulnerability database.
No NDA or data handling discussion. You are granting access to your live systems. Any reputable provider will initiate a conversation about confidentiality, data handling, and scope boundaries before testing begins.
Can’t answer the business logic question. Described above. This is the single clearest differentiator between a real tester and a scan vendor.

Green Flags: What Genuine Expertise Looks Like

They push back on your scope. A tester who asks probing questions — “Do you want us to test authenticated flows? What are the highest-risk user journeys?” — is thinking about your application, not their template.
They describe findings in your application’s context. “In an application like yours, the risk areas we’d focus on are…” shows they’re mapping their experience to your specific situation.
The sample report has attack narratives. Not just “here is a finding” but “here is how an attacker would move from this finding to this outcome in your application.”
They mention the re-test proactively. It’s part of their standard process, not an upsell.
They’re honest about what they won’t find. A provider who says “our engagement covers X and Y; for Z you’d need a separate assessment” is more trustworthy than one who promises comprehensive coverage of everything.

What a Quality Report Must Include

The complete pen test checklist post covers this in full depth. At minimum, before accepting a deliverable, verify:

An executive summary that a non-technical CEO can read and understand — risk in business terms, not vulnerability jargon. Findings with reproduction steps specific to your application — not generic descriptions. Severity in context, not just CVSS scores in isolation. At least one documented attack chain. Developer-facing remediation guidance. A re-test confirmation.

The Kuboid Approach

At Kuboid, we start every engagement with a scoping call — no exceptions. We use tools as starting points and manual testing as the core of the work. Our reports include attack path mapping, executive summaries written for leadership, and developer-specific remediation guidance. Re-testing is included.

We’re happy to answer every question on this list. If the answers don’t satisfy you, we’re genuinely not the right fit — and we’d rather tell you that than deliver something that doesn’t serve you.

Book a free 30-minute scoping call and ask us anything on this list. That conversation costs nothing and tells you everything you need to know about whether we’re the right provider for your application.

Have you ever received a pen test report that turned out to be a scan export in disguise? Or found a green flag with a provider that changed how you think about security assessments? Drop a comment — the more transparent this industry is about what good looks like, the better it gets for everyone.

Kuboid provides web application penetration testing and manual security assessments. Explore our services or get in touch.