Six Years on the Certifier's Side of the Table: What Buyers Should Know

For six years I was the Technical Manager of Norway’s Common Criteria certification scheme. The things I saw from that seat are not what most procurement teams imagine they are buying.

I spent 18 years inside the Norwegian national security authority before that, joining in 2000 after a defence-side career in the Army material command and TADKOM communications. The last six of those years, from 2012 to 2018, I held the Technical Manager role at SERTIT, the Norwegian scheme. I sat through more evaluation reviews than I can usefully count, across smart cards, secure ICs, network and telecom infrastructure, mobile and payment products, java card applications, Trusted Execution Environments, and eIDAS Qualified Signature Creation Devices. I represented Norway in the Common Criteria Development Board, in the SOG-IS Joint Interpretation Working Group, and in ISO SC 27/WG3, the working group that maintains the standard itself. Today I run Scandicert AS, performing evaluation and certification work inside the Dutch government scheme for SESIP IoT and eIDAS QSCD evaluations.

This post is about what that vantage point shows you, what a buyer does not see, and what I think buyers should do about it.

The job the title does not describe

“Technical Manager of a national CC scheme” does not, on its own, say much. The work itself is straightforward to summarise. You manage the relationship between the scheme and the accredited evaluation laboratories that perform the technical work. You monitor evaluations while they are running, attend technical meetings between the lab and the vendor when the questions get hard, and review the Evaluation Technical Report (ETR) before the certificate is signed. You host international meetings on behalf of your scheme, write your country’s position on interpretation questions that come up in the international bodies, and negotiate the result in committee.

What the role is not is what most outsiders imagine. The Technical Manager rarely runs tests. The lab does that. The Technical Manager reads the ETR: the lab’s accumulated record of what was actually evaluated, how scope shifted during the work, what questions were raised, and how the vendor answered. The ETR is confidential. It is never a document the buyer will see.

Three documents, three audiences

Pull the certification pipeline apart and you get three documents pointed at three audiences.

The ETR is written by the lab and read by the certifier. It is the working record of the evaluation: every iteration, every scope adjustment, every question the lab raised, every answer the vendor returned. The ETR is confidential to the scheme and the vendor.

The Security Target (ST) is written by the vendor and read by the lab and the certifier. It is authoritative. The ST defines the Target of Evaluation, declares the conformance claim, lists the assumptions about the deployment environment, and enumerates the security functional requirements. It is published with the certificate, but it is written for evaluators, not for buyers.

The certificate page is what the buyer reads. It is a summary: identifier, product name, assurance level, dates, signatures. The certificate points at the ST and the certification report; it does not reproduce them.

Three documents, three audiences, very different signals. The certifier’s view is the closest thing the process has to a single source of truth. None of it is exportable.

What I saw repeatedly from that seat

A handful of patterns came up often enough across schemes, sectors, and product categories that they are worth calling out as patterns rather than incidents.

Augmentations dropped from the procurement spec

A buyer writes “EAL4 minimum” into a specification. Two vendors respond. One holds an EAL4 certificate with augmentations such as AVA_VAN.5 and ALC_DVS.2 because the product is a smart card and the high-attack-potential vulnerability assessment is part of the certified scope. The other holds an EAL4 certificate with no augmentations because the product is general-purpose software at the EAL4 baseline. Both certificates say EAL4 on the front page. The work behind them is not comparable.

When the buyer’s evaluation matrix flattens “EAL4” into a single column, the augmented certificate gets no credit for the depth it actually carries, and the unaugmented certificate gets too much. The right specification names the augmentation, not just the level. From the certifier’s seat I watched this asymmetry repeat across product categories for years; from the buyer’s seat it usually goes unnoticed.

Composite chains that quietly invalidate

Many of the products I signed off on were composite evaluations. Smart card products are typically evaluated as an applet on top of a certified java card platform on top of a certified integrated circuit, with each layer holding its own certificate. SESIP IoT platform claims sit on top of certified roots of trust. eIDAS QSCDs compose against certified hardware and certified cryptographic libraries.

The composite certificate is only as strong as the weakest in-date underlying component. From the certifier’s chair you watch these chains carefully because a lapse anywhere quietly invalidates the composite claim. From the buyer’s chair the top-level certificate looks active and the chain is invisible. The vendor is rarely the party that volunteers a downstream impact when an upstream platform certificate is suspended or archived. Most buyers I observed never checked the chain.

Scope narrowing during evaluation, frozen at publication

A vendor enters an evaluation with one description of what the Target of Evaluation will cover. By the time the certificate is signed, the TOE is often substantially smaller. Features that turn out to be hard to evaluate get carved out. Optional functionality gets declared out of scope. Configurations the lab could not exercise to its satisfaction become “the evaluated configuration excludes …”. All of this lands in the Security Target as published, but the published ST is a snapshot at the end of the work, not a record of the path.

A buyer who reads only the certificate page never sees the journey. A buyer who reads the ST sees the destination but not the gradient. The certifier sees both, because the ETR carries the iterations explicitly and a Technical Manager has been reading that genre of document long enough to spot the shape of a late-stage carve-out at a glance.

Vendor maturity that shows up in how questions get answered

During an evaluation, the lab raises issues, the certifier may raise issues on top of those, and the vendor responds. Some vendors come back with structured analyses and a clear sense of what their own product does and does not promise. Others come back with marketing-shaped responses that do not engage with the technical question. By the time a certificate is issued, both hold the same certificate. The certificate cannot tell the buyer which is which.

The buyer is left to infer it from how the vendor responds at procurement time, which is often the first real chance they get. The signal is genuine but late. The certifier saw it months earlier.

Why this matters for buyers

Pull these threads together and the asymmetry is structural, not anyone’s fault. The lab reads the product, the vendor, and the test results. The certifier reads the ETR and pressure-tests it against the standard. The buyer reads the certificate and, if diligent, the ST. None of those three audiences sees the same picture, and the audience furthest down the chain has the least access to what was actually done.

A certificate is real evidence. It is also a pointer to a body of evidence the buyer cannot fully reconstruct. Reading the certificate and the ST gets you closer than reading the brochure. It does not get you the rest. Closing the rest of the gap requires reading the published documents with “what does this not say” front of mind, walking the composite chain, checking the lifecycle state at procurement time rather than at evaluation time, and translating the ST’s environmental assumptions into questions the vendor must answer in writing before signature.

That habit is hard to acquire inside a procurement department whose primary work is contracting, not evaluation. The patterns that matter are patterns of attention rather than patterns of fact. They are what a certifier learned to look for after the third or fourth time the same gap showed up in a different product category.

Where an independent advisor changes the calculation

There is a real role for an independent procurement advisor here, and it is not “we have read the standard.” Anyone can read the standard. The role is to read a published Security Target with the same skepticism a certifier brings to the ETR, to spot where the published scope is narrower than the marketing claim, to check the lifecycle state at procurement time rather than at evaluation time, to walk the composite certificate chain, and to translate the certificate’s assumptions into questions the buyer can put to the vendor in writing.

In short: reading certificates with “what does this not say” front of mind, rather than “what does this say.” That orientation is not exotic, and it does not require access to confidential documents. It does require having sat in the room across enough evaluations to know which carve-outs are routine and which ones are flags.

How Nenkin uses this

That experience is what Nenkin Technologies AS sells, buyer-side, through our procurement advisory practice. We sit on the buyer’s side of the table. We do not represent both sides of a single transaction. We do not take vendor referral fees. The recommendation we give a buyer is the recommendation we would give if we were spending the budget ourselves. The Common Criteria procurement guide and the twelve procurement red flags post collect the same patterns as free reference material, for procurement teams who would rather build the muscle in-house. NenkinTracker is where we surface the lifecycle states, composite chains, and CVE links the published portals leave disconnected.

A certificate is real evidence. It is also less evidence than buyers usually assume. Six years on the certifier’s side of the table taught me that the gap is structural, predictable, and worth closing before the contract gets signed.