Specification and result are two columns, and only one of them is a measurement

“Conforms” is the most consequential word in this trade and it is almost always unaccompanied. Conforms to what? Measured how? Against which limit, applied under what rule about measurement uncertainty? The accreditation framework governing testing laboratories is explicit that a statement of conformity requires a documented decision rule — the policy determining whether a result sitting close to a limit is reported as a pass — precisely because the word otherwise conceals a judgement rather than reporting a measurement.

The test table: four columns, and most certificates carry two

The body of a certificate is a table, and a complete one has four columns: the test performed, the method used, the specification applied, and the result obtained. Four columns, one row per test. That structure is not a convention peculiar to pharmaceuticals; it is what a record of controlled testing looks like in any field, because each column answers a question the other three cannot.

The method column is where the detail belongs — not the word HPLC but a method identifier, a gradient, a wavelength, a column chemistry. The specification column states what the batch had to achieve. The result column states what it did. A certificate carrying only test and result has dropped the two columns that make the result interpretable, and this is by far the commonest structural deficiency the Journal encounters.

The compendial framework for validating an analytical procedure exists precisely to establish that a stated method can discriminate what it claims to discriminate, which is why a method reference is not bureaucratic ornament but the hook on which everything else hangs.¹² A named method can be looked up, compared, criticised and repeated. An unnamed one cannot be, and a result generated by one is a number whose provenance stops at the page.

The two columns, and why both are needed

Consider a certificate reporting a purity of 98.6%. Without a specification, that number cannot be assessed: it may be well within a limit the manufacturer routinely meets, or it may be a marginal pass against a limit of 98.5% that the batch scraped through. Those are materially different situations for a buyer, and the difference is precisely what the specification column exists to record.

Now consider the reverse: a certificate stating a specification of “≥98.0%” with the result given only as “conforms”. Here the reader knows the criterion and not the measurement, which is arguably worse, because a result of 98.1% and a result of 99.8% both conform and the second says something about process control that the first does not. Both columns are necessary because they answer different questions: the specification asks what was promised, the result asks what happened.

Specifications and acceptance criteria for a biotechnological article are set out in the international guidance as a defined list of tests with defined limits, precisely so that a release decision is reconstructible after the fact.³ The research-grade certificate has inherited the vocabulary of that framework — the word specification appears, the word conforms appears — without the substance, and the vocabulary is doing rhetorical work the underlying documentation does not support.

What “conforms” conceals

A statement of conformity is a judgement, not a measurement, and the judgement depends on how the reporting party handles results that sit near a limit. If a specification requires not less than 98.0% and a measurement returns 97.9% with a method uncertainty of ±0.4%, does the batch conform? The answer depends entirely on a policy — whether the limit is applied to the measured value, or to the measured value adjusted for uncertainty, and in which direction. Metrological reporting practice has a settled answer here, which is that a result quoted without its uncertainty cannot be compared to a limit at all.⁴

The accreditation standard for testing laboratories requires that where a statement of conformity is given, the decision rule employed is documented and the report identifies it.⁵ This is not a technicality. Without a decision rule, “conforms” means whatever the person writing it decided it meant on the day, and two laboratories applying different rules to the same measurement will report different verdicts without either being wrong.

In this market decision rules are essentially unknown, and the practical consequence is that the word carries no information about marginal cases. The Journal therefore reports numbers rather than verdicts wherever a number is available, and where a source supplies only a conformance statement we say so explicitly rather than paraphrasing it as a pass. Readers occasionally find this pedantic. It is the difference between reporting a measurement and repeating an opinion.

The specification that cannot be failed

There is a class of acceptance criterion which is technically present and practically inert, and once you have learned to see it you will find it everywhere. A purity limit of ≥95% on material the supplier consistently reports above 99%. A mass tolerance of ±1 dalton on a peptide of four thousand daltons, which corresponds to two hundred and forty parts per million and would be met by essentially any instrument. An appearance specification of “powder”. A water specification of “≤10%” on a material that typically carries three.

None of these is a false statement and none is evidence of bad faith. They are the residue of forms designed for a broad catalogue, where a single specification has to cover hundreds of products with different behaviours, and where the limit is set at the point below which the supplier would definitely intervene rather than at the point representing normal performance. Distinguishing between the two is the whole art of reading the specification column.

The useful question to ask of any limit is what the supplier’s typical result is. A specification of ≥98% with results clustering at 98.2% is a real control operating close to its edge. A specification of ≥95% with results at 99.4% is a wide net around a well-behaved process. Both are defensible; they mean different things; and a buyer who reads only the result learns neither. The pharmacopoeial convention is worth holding on to here: a specification states the quality a substance must possess, not a complete description of what it is.⁶

How this publication handles certificates

Four rules, arrived at over two years and revised twice. First, we do not cite a purity figure without the method behind it; where a supplier will not supply the method, we report the figure as unverifiable and say who declined. Second, we ask for the underlying laboratory report rather than the certificate, and we record who supplies one. Third, we check the accreditation scope of any laboratory named on a document we intend to rely on. Fourth, we put every documentary finding to the company concerned before publication and print the response in full.

The fourth rule is the one that has changed our coverage most. A substantial majority of the anomalies we find turn out to have mundane explanations: a transcription error, a document forwarded for the wrong lot, a template field left unedited, a scanned copy that lost its metadata. Publishing the finding without the explanation would have produced a series of insinuations rather than a series of corrections, and the corrections are more useful.

The rules also mean we publish less than we could. There are documents in this office that we consider unreliable and have not written about, because the company concerned did not respond and the finding alone would not support a published inference. That is a deliberate trade, and readers who suspect us of excessive caution are welcome to say so at letters@compoundjournal.com, where several already have.

A necessary distinction between a bad document and a bad product

Almost nothing in this article supports an inference about the contents of a vial. A certificate missing a specification column, an unsigned footer, a stale date of analysis and a batch number that appears only on the carton is a poor document. The material it accompanies may be excellent, and in the Journal’s experience frequently is: the analytical work behind these products is often better than the paperwork that reports it, because the paperwork is produced by a commercial function and the analysis by a laboratory.

The reverse also holds. A beautifully constructed certificate with four columns, two signatures and a named method is evidence of a functioning documentary process and is not evidence about the vial either, since a document cannot testify to material it does not accompany. This is why the Journal reports documents as documents and material as material, and declines to convert one into a claim about the other.

We labour the point because the alternative is a genre of coverage that treats documentary weakness as proof of dishonesty, and that genre is both unfair and useless. Unfair because most documentary weakness in this trade is inherited convention rather than intent. Useless because it gives a reader nothing to do. Reading the document properly gives a reader something to do, which is the entire purpose of this piece.

The Journal will keep asking the twenty companies in its dossier programme for the same seven lines each quarter, and will keep publishing who supplies them. Two years of doing this has produced measurable movement: the number of programme certificates carrying a specification column for every test has risen from four to nine, which is slow and is not nothing.