How to Read a Pentest Report and Judge What You Paid For

Two teams can pay the same money, receive similar-looking documents, and get completely different things. One gets a list they can hand to engineers on Monday morning. The other gets a PDF that triggers four weeks of meetings about whether a finding is real. The difference is almost never tester skill. It is whether the reader knew what to ask for, before and after.

Core assessment: a penetration test report has exactly one job — to let someone who was not there reproduce, judge and fix each finding without contacting the author. Every section that does not serve that job is compliance decoration, useful for auditors and useless for remediation. Both belong in the document; only one belongs in your backlog. Read the report by checking evidence quality first and severity second, because a well-proved medium will be fixed and acted on, and a badly-proved critical will be argued about until it quietly closes.

The One Question Every Finding Must Answer

"Could an engineer who has never met the tester act on this?" If the answer is yes, the finding is evidence-grade. If the answer is "they'll ask a question first", it is an allegation. Most reports are a mix of both and most readers cannot tell which is which, because both are formatted identically.

That distinction is worth more than the entire executive summary, because it determines whether your remediation loop runs at all.

Anatomy of an Evidence-Grade Finding

Note the recommendation row. Generic advice is the clearest tell that the tester did not know your architecture, and the clearest reason your engineers will distrust the rest of the document. A recommendation that names a component your codebase contains is a recommendation someone will implement.

Sections That Are Decoration — and Why You Still Want Them

Methodology narrative, risk matrix graphic, executive summary, scope table, tester CVs, tools appendix. None of these produce a single line of remediation. They are what your regulator, your insurer, your customer's procurement team and your board ask for. Keep them, budget for them, and do not confuse them with the assessment.

The honest version of this: when a vendor leads a sales conversation with the template and the dashboard, they are selling the decoration. When they lead with an anonymised finding, they are selling the test. Ask which one you're buying.

Nine Questions That Separate a Test From a Scan

Ask these of any report, including ones you have already paid for.

1. How many findings require an authenticated session to reproduce? Zero is a strong signal that nobody understood your application. Scanner output does not need to log in.
2. How many describe a business-logic flaw? Price manipulation, workflow skipping, state-machine violations, cross-tenant reads. A scanner has no model of your intended behaviour, so it cannot find any of these. Zero of these and you bought a scan with a nicer cover.
3. How many are unique? Deduplicate by root cause. "Missing security headers on 40 endpoints" is one finding with a wide footprint, not forty. Inflation is the most common way to make a scan look like work.
4. Does any finding reference something that exists only in your application? A custom endpoint, an internal role name, a workflow only your product has. If nothing in the report could only have been learned by reading your system, the system was not read.
5. Is there at least one finding that a tool would have missed and a human would not have hesitated about? Low-and-slow: an IDOR behind a pagination parameter, a password reset that accepts an old email, an internal endpoint reachable because of a rewrite rule.
6. What is the ratio of findings to tester-days? Twenty findings in five days on a medium web app is a scan. Two findings in five days that each take a week to fix can be the better engagement. Volume is not the deliverable; consequence is.
7. Do the recommendations reference your stack? Frameworks, libraries, cloud services, deployment model. Generic text means the tester stayed outside your architecture.
8. Is negative coverage documented? Which endpoints, roles and flows were not tested, and why. A report that claims to have tested everything tested nothing carefully. Absence of this section means you do not know what you bought.
9. How long from last test day to report? Fast is good; implausibly fast with many findings suggests the document was generated and lightly edited rather than written from a lab notebook.

Severity Is the Wrong Column to Sort By

CVSS describes the flaw. It does not describe your situation, and your backlog should be ordered by the second. Five axes, and you need all of them:

Ordering by severity alone is how organisations spend a quarter fixing something nobody could reach while leaving a genuine exposure open because the tester wrote 5.3. The reordering conversation is legitimate and it belongs at the report review meeting, not in the tester's scoring choice — the tester's job is to describe the flaw accurately, yours is to decide what to do first with it.

One caveat against over-correcting: "our exploit requires a paid enterprise account so we will not fix it" is usually wrong, because that account is a target, not a protection. Reachability arguments are about whether a real attacker has a path, not whether the configuration is common.

What to Demand Before You Pay the Invoice

Get these into the order form. Retroactive demands have no leverage.

• A reproduction walkthrough session with the tester and your engineers, before the report is treated as closed. Thirty minutes on a call resolves half the arguments a report generates in a month.
• The raw evidence export: requests, responses, screenshots, callback logs, scripts — in a form your team can re-run. Evidence that lives only inside a PDF is evidence you cannot verify after the tester leaves.
• A coverage matrix naming what was tested and, explicitly, what was not.
• A written explanation for zero findings in any major area. A clean report on a real application is either excellent work or shallow work, and only one of those is common.
• A defined retest window with scope, size and price fixed in advance, so remediation verification is not renegotiated at the moment you need it.
• A finding-quality bar agreed before testing: the field list above, not "industry standard".

The last one matters most. Every ambiguity you fail to define beforehand becomes a dispute afterwards, and disputes are where engagement value goes to die.

What "Fixed" Means, in Four Levels

• Level 0 — Not fixed. Reported endpoint patched, same class reachable elsewhere. The most common outcome, because reports are organised by endpoint and flaws are organised by pattern.
• Level 1 — Mitigated. A WAF rule, a rate limit, a validation on one input. Real reduction, unchanged root cause, and a false sense of closure.
• Level 2 — Fixed as described. The reported instance is genuinely corrected.
• Level 3 — Fixed and generalised. The class was checked across the codebase, the fix is in a shared layer, and there is a regression test that would have failed before.

A retest that only re-checks the reported instance tells you about Level 2. Ask for Level 3 verification: the tester searching for the pattern, not the line. Where the same root cause has other instances, that difference is the entire value of the retest.

How to Compare Two Vendors

Ask each for one anonymised report from a client in your stack, and score them on four counts.

• Proportion of findings with literal request/response evidence attached.
• Number of findings that required an authenticated session.
• Number of findings that are not scanner-shaped — i.e. that a scanner could not have produced.
• Time from end of testing to report, and whether the document reads like it was written by someone who used it.

Those four counts predict your remediation experience better than methodology names, certifications, tooling lists or platform demos.

Reading a Report When You Are the Receiver

A short process for whoever inherits the document, because a bad process can waste a good report.

1. Triage by the five axes above before you touch severity.
2. Reproduce the first three findings yourself, on a call, before distributing. If one does not reproduce, the report's credibility is now open and you need to know that before an executive reads the word "critical".
3. Ask for the pattern grouping in writing — which findings share a root cause. Then fix groups, not items.
4. Assign every finding an owner with a date, in your tracker, not in an attachment to an email.
5. Write the retest questions now, while the tester is still engaged and the context is fresh.

Frequently Asked Questions

Is a report with only a few findings a bad report?

Not necessarily, and the number alone cannot tell you. A short report where each finding has literal evidence, a described preconditions chain and a stack-specific recommendation can be excellent work. The diagnostic is not the count — it is whether any finding in it required understanding your application to find, and whether the report says what was not tested. Two findings plus an explicit coverage gap list is more useful than twenty findings plus silence.

Should we pay extra for a retest?

Yes, in almost every case where the original test was real work. The retest verifies whether the money you spent on remediation bought what you think it bought, and it checks the pattern rather than the line. Build it into the original engagement so it is not a negotiation later. What is worth questioning is a retest scoped to re-running the same requests — that verifies Level 2 at best.

Can we just forward the report to engineering as-is?

You can, and you will get the meeting about whether finding four is real. Forward it after you have (a) reproduced a sample, (b) grouped findings by root cause, (c) re-ordered against reachability and data exposure, and (d) attached an owner and a date to each. That conversion is a fifteen-minute task when the report is evidence-grade and a two-day task when it is not, which is itself a useful signal about what you bought.

How do we handle a finding we disagree with?

Ask for the reproduction, on a call, and record what happens. If the tester demonstrates the effect in your own session with your own data, the disagreement is about risk acceptance, which is your decision to make and to document. If the tester cannot demonstrate it, the finding should be marked as unverified and treated accordingly — not deleted, and not fixed on the basis of a name. Disagreements are information about report quality; keep them.

Who should sign off on the report?

Someone accountable for the risk, not someone accountable for the project. A project manager closing a pentest because the deliverable arrived is how organisations accumulate twenty-eight days of clean audit history and one public incident. The sign-off should state, in writing, which findings are accepted as residual risk and by whom — that sentence does more for your future self than the executive summary.

Next Step

If your last report needed interpretation rather than action, the gap is on the specification side and it is fixable in the next order form. Eresus Security works to a written evidence standard — every finding with a reproduction, an impact chain and a stack-specific recommendation — across web application penetration testing, API security testing and red team operations. The same standard is described on Eresus Guard. Contact us to scope an engagement, and ask for a sample report first.