Five Minute Argument or the Full Half Hour?

It is nearly fifty years since John Cleese first asked Michael Palin whether he wanted just a five-minute argument or “the full half-hour.”  Although the question is absurd it does highlight an important reality in the service industry.  Clients’ requirements differ as do their budgets.

Penetration Testing

We would not characterise our services as “argumentative” but there is a degree of controlled confrontation in one of our most popular offerings – Penetration Testing, sometimes known as Ethical Hacking or Red Teaming.

The purpose of a penetration test is to carry out carefully controlled attempts to breach the security of a client web application, office or data centre to identify security weaknesses so they can be fixed before a true “black hat” hacker finds them.  The penetration test is intended to be a simulation of a hacker attack by employing many of the same techniques a black hat would use.

Fidelity – Five Minute Argument or the Full Half Hour?

Nevertheless, it remains a professional engagement that must be scoped, priced and project managed like any consulting job.

There is an inherent tension here because hackers and consultants behave in totally different ways.  A black hat does not keep office hours, bill their time, write formal reports or have a deadline.  They might keep attacking a target for months until they succeed.  They may lie dormant until the second Tuesday of the month when details of the latest security flaws in Microsoft Windows are released and move quickly to exploit them before security updates are applied on target systems.

The question of fidelity arises – just how close to the real thing should a penetration test engagement be?  Should it be the five-minute version or the full half-hour?

The Fidelity vs Cost Trade-off

A “low fidelity” simulation of a black hat attack could be delivered using several automated penetration testing tools supplemented with a modest amount of manual testing.  Such an engagement would be relatively quick to complete and therefore comparatively cost-effective.

Because a low fidelity penetration test is not a realistic simulation of how a black hat hacker would attack the client network it would be inappropriate to characterise the engagement as a realistic simulation or infer in a report that a lack of serious adverse findings means the network is not vulnerable.

Low fidelity penetration tests can give negative assurance – if a flaw is found then the network has a flaw.  They should not be relied upon for positive assurance – i.e. the absence of flaws in the report does not mean none exist.  It might be that the scope of the engagement, and time spent, was insufficient for flaws to be identified.

A high(er) fidelity penetration test will take more time and therefore be more costly to deliver.  Such a test will be more likely to find security weaknesses and therefore will be more likely to give negative assurance.  However, there is a stronger basis for a high(er) fidelity test giving positive assurance.  The more testing work completed and the closer the engagement resembles a genuine black hat attack the stronger the basis for drawing a positive inference from an absence of identified security problems.

False Assurance

We have observed a worrying trend in the penetration testing industry where the conclusions reached in written reporting cannot be supported by the technical testing work undertaken by the consultant.

The classic example is a report that advances a conclusion that the client network or application might not be vulnerable (positive assurance) based on a small amount of technical testing, often by automated tools, that does not in any way represent the techniques of a real black hat.

Another issue is the scope of testing being inadequate to support the conclusions reached.  It is very common to see penetration test reports that reach a conclusion as to the security of a web application based solely on penetration attempts against the web application itself.

We know from experience that web applications are often compromised via email-borne malware threats and phishing/whaling attacks directed at company staff.  This is corroborated by the latest UK Government cybersecurity survey results.  However many web application tests completely ignore this, significant, attack vector.

It may be that this false assurance problem is caused by consultant inexperience.  It is common for penetration testers, even those with the usual certifications, to have only a few years experience.  Those of us with grey, or no, hair are more likely to understand the relationship between work undertaken and conclusions reached.

Commercial and competitive pressures may also prompt some in the industry to over-promise and under-deliver.

Our Approach

Our approach to penetration testing engagements is driven by deep experience and a desire to be flexible to meet client requirements.


Our penetration testing consultants have in excess of 25 years of professional experience – that predates the first network security scan tool (SATAN) and all IT security certifications.

That experience includes teaching security certifications, looking after national elections, consulting to several governments around the world and carrying out thousands of penetration tests for national government, local government, emergency services, banks, airlines, mobile telephone providers, financial markets, television networks, IT providers and many other clients.

Our 20 years of forensic experience includes working with law enforcement to investigate black hat activity and giving expert evidence in Court, including as a Crown expert in criminal prosecutions.  This hands-on digital forensic experience is invaluable when carrying out penetration testing engagements or designing & deploying protective measures in client environments.


We can adapt our approach to suit your requirements and budget.

Obviously the strength and type of assurance we can give will depend on the scope and level of effort involved in the engagement.  For example, we could carry out:

  • vulnerability assessment – an automated scan looking for known security vulnerabilities. Vulnerability assessments of Internet-facing systems can be carried out and it is also possible to vulnerability scan cloud servers, a data centre or an office or site.
  • compliance scan – similar to a vulnerability assessment, a compliance scan is usually only carried out internally and requires privileged login credentials for the scanned systems. Systems can be tested for compliance against security standards such as PCI-DSS, HIPAA, CIS, etc.
  • penetration test – as described above, this is a vulnerability assessment followed by semi-automated or manual intrusion attempts. Penetration tests of Internet-facing systems can be carried out, and we recommend also carrying out internal tests of cloud environments, data centres and offices/sites.  We recommend including social engineering, phishing and malware tests as these are favourite black hat techniques.

We can also work with you proactively to secure your environment, this is known as Blue Teaming.


There is no such thing as a “one size fits all” penetration test. Because of our decades of experience and strong technical capability we can work with you to devise a penetration test that best suits your needs.

Call us on 0203 858 7372 and we will be pleased help you – whether it takes five minutes, a half hour, or even longer.