Testing Modes

The three modes compared

The distinction that matters is access. Everything else follows from it.

Black box

Black box testing simulates an attacker who has what an attacker actually has: the running application and the accounts anyone can obtain. The testers are given no source, no architecture documents and no help beyond credentials for each role that exists.

Its strength is that everything it finds is reachable. There is no argument about whether a defect is exploitable in practice, because it was exploited from the position an attacker occupies. Its limit is the reverse: a defect sitting behind a code path the interface never reaches will not be found, and the test cannot tell you whether it was absent or merely unreachable during the engagement.

It is the mode most often required by a customer, a procurement process or an auditor, because it is the one whose result needs no trust in the supplier's own account of their code.

Grey box

Grey box testing gives the testers what a knowledgeable insider would know — the architecture, the data model, the roles, and usually some source — while still attacking the running system from outside. It is not a compromise between the other two modes so much as a different trade: the same attacks, aimed better.

The effect is on efficiency rather than on the kind of defect found. Knowing that a field is used to build a database query, or that two services trust each other, removes the days a black box test spends inferring it. That time goes into testing instead of into reconnaissance, which is why this mode usually produces the most findings per day of effort.

Its limit is that it inherits whatever the shared material gets wrong. Where documentation and implementation disagree, the test follows the documentation unless something contradicts it, so we treat shared material as a hypothesis and confirm it against the running system.

White box

White box testing reads the code as well as attacking the system. It reaches the paths an external test cannot: error handlers, administrative routines, code behind feature flags, and the branches that only execute under conditions the interface will not produce.

It also changes what a finding is. A black box test reports the instance it reached; reading the source shows whether that instance is one occurrence of a pattern repeated across the codebase, which is usually the more useful thing to fix. Backdoors, deliberate or accidental, are realistically only found this way.

Its limit is that source is not deployment. Code can be correct and the environment it runs in insecure, so a white box engagement is run against the running application as well as the repository rather than instead of it.

A security code review works through the areas below, in addition to whatever the application's own design makes significant:

• Authentication and the mechanisms around it
• Authorisation and the enforcement of least privilege
• Input validation and output encoding
• Protection of configuration data and secrets
• Confidentiality and integrity of stored and transmitted data
• Cryptography and key management
• Password policy and credential handling
• Session management
• Protection of administrative interfaces
• Database access and query construction
• Endpoint protection for sensitive data
• Runtime error handling
• Auditing and logging

Choosing between them

• If someone else has to trust the result — a customer, an auditor, a procurement process — black box is usually what they mean, because its findings do not depend on the supplier's own account of the code.
• If you want the most from a fixed budget, grey box almost always returns more, because the reconnaissance a black box test pays for in days is handed over on the first morning.
• If being wrong is expensive — payments, health data, safety, anything with a regulator attached — white box is the only mode that reaches code the interface never exercises.
• Modes can be mixed per component. A common shape is white box on the authorisation layer and the payment path, grey box on everything else.

Whichever is chosen, the depth stated against each test case does not change: a check a scanner performs is described as automated in every mode, and a test requiring judgement is described as manual in every mode.

What this means for coverage

Not sure which mode fits? Tell us what the application does and we will recommend one, including saying so when a lighter mode would answer your question.