Qualification to regulators is what validation means to scientists

Definition 

Biomarker qualification is about thinking through the full chain of evidence to prove that a biomarker can be used for a particular clinical decision.

HDL serum cholesterol, for instance, is great for evaluating risk of heart disease but not for evaluating effectiveness of treatments to improve cardiovascular health. There is no such thing as a “good biomarker” in a vacuum. Decisions to use biomarkers are always dependent on the intended applications.¹ Sadly, this is not something that most biomedical researchers think about when they do biomarker discovery.

Biomarker guided therapeutic decisions require developing and validating biomarkers. Specifying what these criteria are requires constant meta-scientific innovation. It is easy to conflate the enterprise of biomarker validation with analytical validation² , followed by reproducible clinical studies. But what constitutes clinical validation?

Analytical validation is all about evaluating the measurement process or assay. Many biomarker discovery studies will demonstrate test-retest reliability that looks for whether people can be reliably differentiated based on their biomarker measurements. One has to evaluate a far more thorough checklist of measurement issues that go well beyond test-retest reliability for analytical validation. We also need repeatability of measurements for any individual with good tolerance intervals, comparability of quantitative measurements in a wide variety of circumstances and many others. Yet, analytical validation is the easier part of the biomarker evaluation process with systematic criteria. Qualification on the other hand encompasses the full spectrum of validity problems across all the life-medical-health sciences — it includes all the possible “does it mean what you think it means” problems. Biomarker qualification includes assessing the clinical validity³ of the biomarker as well as other validation criteria specific to a therapeutic decision.⁴

Importantly, there is no easy way to preemptively specify what all the threats to validity are — construct validity, causal validities including internal validity and external validity, all the modern validities beyond reliability of the measurement that link it to disease and/or therapeutic outcomes.

A proposal for an end-to-end evidential roadmap from Altar et. al. (2008)

Here is a concrete example of what a comprehensive understanding of biological and clinical validation looks like.

Credit: Altar, C.A. et al. (2008) ‘A prototypical process for creating evidentiary standards for biomarkers and diagnostics’, Clinical pharmacology and therapeutics, 83(2), pp. 368–371. https://doi.org/10.1038/sj.clpt.6100451.

However, this table reflects 20th century understanding. It could use significant updating given how far scientific and statistical methodology has come in 20 years.

Checklist for a biomedical stakeholder

Analytical validation and biomarker qualification are terms of art when biomarkers are proposed for drug development decisions in clinical trials. Unfortunately, these terms are not widely used within mainstream biomedical research. It is easy to think these are regulatory concerns don’t matter until one wants to bring biomarkers to the clinic, as opposed to scientific concerns that need to be addressed. Every research community has its own epistemic norms around “validation”. When you visit premier conferences in different niches of life science where biomarker research occurs, these differences become apparent. No one actually owns the problem of understanding the full scope of scientific R&D that needs to occur.

If you read an article that calls for a large-scale validation for new biomarkers, here is what you should ask yourself —

1. Is it clear that the biologist or scientist’s notion of validation is distinct from analytical validation? Has it at least considered all the problems in Altar 2008, for instance?
2. Does the roadmap for biological plausibility and clinical validation cover the full spectrum of research designs and grades of evidence that need to be generated?
3. Does it address all threats to scientific validity known to methodologists for a particular decision or context of use?

If not, then the field might need a better specification of the validation roadmap. It is far too late to do the necessary R&D if you wait until someone is ready to initiate conversations with the FDA.

References

1. Institute of Medicine. 2010. Evaluation of Biomarkers and Surrogate Endpoints in Chronic Disease. Washington, DC: The National Academies Press. https://doi.org/10.17226/12869.

2. FDA on analytical validation, ICH on analytical validation

3. Ransohoff, D. Bias as a threat to the validity of cancer molecular-marker research. Nat Rev Cancer 5, 142–149 (2005). https://doi.org/10.1038/nrc1550

4. Fleming, T.R. and Powers, J.H. (2012) ‘Biomarkers and surrogate endpoints in clinical trials’, Statistics in medicine, 31(25), pp. 2973–2984. https://doi.org/10.1002/sim.5403.