Consensus guidelines for the validation of qRT-PCR assays in clinical research by the CardioRNA consortium

Abstract

Despite promising findings, quantitative PCR (qPCR)-based tests for RNA quantification have experienced serious limitations in their clinical application. The noticeable lack of technical standardization remains a huge obstacle in the translation of qPCR-based tests. The incorporation of qPCR-based tests into the clinic will benefit from guidelines for clinical research assay validation. This will ultimately impact the clinical management of the patient, including diagnosis, prognosis, prediction, monitoring of the therapeutic response, and evaluation of toxicity. However, clear assay validation protocols for biomarker investigation in clinical trials using molecular assays are currently lacking. Here, we will focus on the necessary steps, including sample acquisition, processing and storage, RNA purification, target selection, assay design, and experimental design, that need to be taken toward the appropriate validation of qRT-PCR assays in clinical research. These recommendations can fill the gap between research use only (RUO) and in vitro diagnostics (IVD). Our contribution provides a tool for basic and clinical research for the development of validated assays in the intermediate steps of biomarker research. These guidelines are based on the current understanding and consensus within the EU-CardioRNA COST Action consortium (www.cardiorna.eu). Their applicability encompasses all clinical areas. Despite promising findings, quantitative PCR (qPCR)-based tests for RNA quantification have experienced serious limitations in their clinical application. The noticeable lack of technical standardization remains a huge obstacle in the translation of qPCR-based tests. The incorporation of qPCR-based tests into the clinic will benefit from guidelines for clinical research assay validation. This will ultimately impact the clinical management of the patient, including diagnosis, prognosis, prediction, monitoring of the therapeutic response, and evaluation of toxicity. However, clear assay validation protocols for biomarker investigation in clinical trials using molecular assays are currently lacking. Here, we will focus on the necessary steps, including sample acquisition, processing and storage, RNA purification, target selection, assay design, and experimental design, that need to be taken toward the appropriate validation of qRT-PCR assays in clinical research. These recommendations can fill the gap between research use only (RUO) and in vitro diagnostics (IVD). Our contribution provides a tool for basic and clinical research for the development of validated assays in the intermediate steps of biomarker research. These guidelines are based on the current understanding and consensus within the EU-CardioRNA COST Action consortium (www.cardiorna.eu). Their applicability encompasses all clinical areas. IntroductionA literature search on biomarkers and cardiovascular diseases (CVDs) highlights the poor correlation between the efforts performed in the initial steps of the development of quantitative PCR (qPCR) assay-based biomarkers, i.e., discovery and preclinical stages and their incorporation into clinical practice. There are a number of barriers that contribute to this poor implementation. The lack of technical standardization constitutes a key limitation in the incorporation of qPCR-assay-based biomarkers into the clinic. Limitations are also linked to the absence of consensus reference values, poor harmonization of the study populations, and the barriers in collaboration between academia, physicians, and industry. For instance, despite the thousands of noncoding RNA (ncRNA)-based biomarker studies published to date, there is a paucity of potential indicators that have been successfully translated into clinical practice, mainly due to the lack of reproducibility of research findings. Kok et al.1Kok M.G.M. de Ronde M.W.J. Moerland P.D. Ruijter J.M. Creemers E.E. Pinto-Sietsma S.J. Small sample sizes in high-throughput miRNA screens: a common pitfall for the identification of miRNA biomarkers.Biomol. Detect. Quant. 2018; 15: 1-5Google Scholar nicely illustrate the situation for coronary artery disease (CAD)-associated circulating microRNA (miRNA) biomarkers based on a literature review yielding 13 miRNAs found to be up- or downregulated in more than one study, of which more than half (7 out of 13) showed a contradictory result between studies (e.g., for miR-21, two studies showed upregulation and one study showed downregulation). This lack of reproducibility has also been addressed in several publications,2Faraldi M. Gomarasca M. Sansoni V. Perego S. Banfi G. Lombardi G. Normalization strategies differently affect circulating miRNA profile associated with the training status.Sci. Rep. 2019; 9: 1584Google Scholar, 3Godoy P.M. Barczak A.J. DeHoff P. Srinivasan S. Etheridge A. Galas D. Das S. Erle D.J. Laurent L.C. Comparison of reproducibility, accuracy, sensitivity, and specificity of mirna quantification platforms.Cell Rep. 2019; 29: 4212-4222.e5Google Scholar, 4Leidner R.S. Li L. Thompson C.L. Dampening enthusiasm for circulating microRNA in breast cancer.PLoS One. 2013; 8: e57841Google Scholar with reported causes ranging from technical analytical aspects to variable patient inclusion criteria and underpowered studies to sample quality. As such, the field of in vitro diagnostics (IVD)-grade quantitative reverse transcription PCR (qRT-PCR) assays for clinical use, initially developed in research laboratories, is still in its infancy. The incorporation of novel molecular biomarkers for clinical decision-making and patient management, i.e., diagnosis, prognosis, prediction, and monitoring of the therapeutic response or toxicity, need clear assay validation guidelines to be followed in the context of clinical research.In this context, basic and clinical researchers often resort to the use of laboratory-developed assays with variable and undefined quality, commercial research use only (RUO) assays or, in the best-case scenarios, laboratory-developed assays validated in accordance with guidance such as minimum information for the publication of quantitative real-time PCR experiments (MIQE) guidelines.5Bustin S.A. Benes V. Garson J.A. Hellemans J. Huggett J. Kubista M. Mueller R. Nolan T. Pfaffl M.W. Shipley G.L. et al.The MIQE guidelines: minimum information for publication of quantitative real-time PCR experiments.Clin. Chem. 2009; 55: 611-622Google Scholar The difference between such assays and certified IVD assays is significant. Laboratory-developed assays for clinical research are typically less controlled and standardized and do not need to comply with regulations, such as the European In Vitro Diagnostic Regulation (IVDR 2017/746). The European regulatory framework, based on the aforementioned IVDR and the Clinical Trials Regulation 2014/536, leaves a gray area relative to the status of laboratory assays that are used in the context of clinical trials. Poorly validated assays are not appropriate for large-scale clinical biomarker studies. Therefore, researchers would benefit from guidelines on the validation of what we refer to as clinical research (CR) assays, an assay type filling the gap between RUO and IVD that addresses the specific needs of researchers in the development of biomarkers. To some degree, such CR assays are similar to laboratory-developed test (LDT) assays in that they have undergone more thorough validation without reaching the status of a certified IVD assay.GlossaryAnalytical precision (or precision): closeness of two or more measurements to each otherAnalytical sensitivity: the ability of a test to detect the analyteAnalytical specificity: the ability of a test to distinguish target from nontarget analytesAnalytical trueness / analytical accuracy: closeness of a measured value to the true valueClinical research: in this article, “clinical encompasses clinical studies and their of which the are to therapeutic or understanding of disease Clinical typically to laboratory in clinical of that the appropriate use of a or a that the of validation associated with a development tool is to its context of the ability of a test to the absence of the disease in with test the ability of a test to disease in with / of that are / of that are a CR researchers can more out RUO assays that are and to in the stages of biomarker research to test or clinical laboratory This is in Here, we will focus on the necessary steps that need to be taken toward the appropriate validation of qRT-PCR for CR and clinical the of this review is not to regulatory guidance for with The is to and technical in the specific context of qRT-PCR for which are not to or are to researchers of the for biomarker and clinical biomarker is a a of or the to an or therapeutic or the of a or to their use, biomarkers can be into several and with the of biomarkers, can be addressed such as a will will benefit from a will a be or and will be or and the of a patient would the validation of a biomarker an evaluation of the analytical and analytical and and the clinical sensitivity, and trueness (or analytical to the closeness of a measured value to the true analytical precision to the closeness of two or more measurements to each and the and reproducibility of the is the ability of a test to detect the the minimum or and analytical specificity is the ability of a test to distinguish the target from nontarget assays, the of a target than is the ability of a test to between the or absence of and is using of of accuracy: basic 2009; Scholar The of a test is in the true the identification of with the the specificity of a test is measured as the true the identification of without value is the ability of a test to disease in with value is the ability of a test to the absence of disease in with test are on the of of on the context of use and to the and be to the is that the (or of validation associated with a development tool is to its L. and Scholar validation is the of an assay including the of the of the the standardization of the experimental the accuracy, the and the V. R. J. S. S. M. L. et development and validation for biomarker are out in the and and European guidelines and an and for a of for to of Scholar what of the biomarker is measured and in what what is the clinical of the and what is the and based on the that are to clinical decision-making have to be validated to a which that a biomarker a specific and an to its in clinical of the use of a therapeutic is in the to specific can the of the of a in a clinical can a of in a patient clinical selection, and to clinical trials for of and can This can be to all of discovery research and development with the the use of a biomarker the of analytical as as the of the criteria of the biomarker its validation of with to the validation be necessary and the of validation with followed sample sizes and tests of the biomarker is for the of the and in more biomarkers which are the of the validation do not regulatory RUO biomarkers can be used as biomarkers or for However, validation will be in the biomarker will be developed toward clinical use, as a regulatory a biomarker needs to and comply with the of a that a biomarker will be used in the clinical The will be a biomarker within its one that is measured in an analytical test with and for which there is an or of that the or clinical of the test of for to Scholar, of Scholar, experimental which experimental with analytical and are for and of for validation of on validation ranging from guidelines and such as and to regulatory such as the and IVDR V. R. J. S. S. M. L. et development and validation for biomarker S. assay for clinical 2018; 9: J. A. M. validation in J. Scholar for the use of of is the that they have not been to the of tests. for the of quantification for target and PCR which addresses specific we guidelines for the analytical validation of assays in clinical is that the of a researchers to from as a of the specific needs and of their tests. a the validation of assays the the specificity of an assay is typically a assay reference this be their measurements are to the of not the of a specific the of is an for assays or for tests to with tests the is the addresses specific in and regulatory without to the of the or the of an assay to a thorough validation used for biomarkers that on such the less validation experiments be for tests that on the of development for for a of analytical assay is to with validation. validation is and and for the of to the more CR (or ultimately to the of the analytical the and the and be and are on used within the one needs to be that the validation only to that As there are between with to assay and the is validated only to the that is the analytical validation of and reference the on a sample lack of on a of and and specificity of an experimental for validation be performed as and and a than is as a The RNA be from a sample or in vitro transcription from a or from an RNA sample that has undergone the as clinical and that is to the of this also be a commercial sample in and with and similar RNA to the test Clinical be a of commercial are and the of the of and for and (e.g., or for without reverse will also test for of the RNA to for the from to test for in preclinical studies are used in a from a specific RNA sample not to the or of is the of its similar to a to the PCR and analytical of the assay type the of is or of the and of the This can be also for with or for assays, the also the RNA the from patient reference for sample for is to (or the to the for this are of the use of reference or of of the also be used for and for all be performed or a of For with a between and for in assays (e.g., a the of the (or for some with as a reference a is In the of is a of be the in a from be the of the of due to or or of due to the use of The is to the of the in the of or are for some assays, the is and such technical do not with their CR assay of the from RNA to The standardized of each is to biomarker reverse the is this a to the RNA this is on the type and of RNA a reverse transcription be The use a of of the aforementioned or target to used in the experiments the test of an RNA which is performed as with of including an RNA followed for the Here, there be between the of the and the of RNA within a to of the RNA of target RNA can be to reverse transcription of one specific RNA to test for we also a RNA reverse transcription that is from the one used for RNA The of RNA be in the of the which would often be in the of This is to for in the to the identification of target is the of guidelines on the is one in the with all its or in a of a the assay have to detect a specific or there to be for assay of with that would with of tests on reference for validation is not to the assays the target of assays for reference as the of the test on the of of the reference is to that and reference are study reference in and a tool such as J. de de A. of real-time quantitative of Scholar or C.L. Normalization of real-time quantitative reverse a to for to and Scholar can the that the reference are the experimental and are in the and sample type of for in a of reference has been A. L. de D. G. A. R. M. et and validation of miRNAs in for of for Scholar a evaluation of reference is for each experimental and study the of analytical assay the of a such as A. T. J. M. and Scholar be with for the and absence of and common in or be in assay such as S. S. Hellemans J. J. PCR assay for using Scholar performed with such as A. T. and search tool for PCR on Scholar and M. for and Scholar or performed in and the benefit of such in they a of in in the Therefore, can be into analytical for the of the assay and the of one assay all for CR assay validation of qPCR-based an of a that be used for tests This have to be to the of specific analytical with or assays, in a context, or for The number of also be for biomarkers with a research assay also needs to the assay These to of the instance, the criteria for the from the to the The are criteria such as accuracy, sensitivity, and of quantification The of this validation have to be to the lack of reference the of is to the experiments in the studies to the of for we an on of to the of qRT-PCR tests. To this of the reference for the of and for the reference the of the test for RNA we the evaluation of a of RNA with of This can be or with D. the of RNA R. M. study of RNA Scholar are to and its is one this of the of RNA can be and based on the RNA number or an is more to the of such as this can be the of (e.g., or that have been in specific J. P. L. P. S. L. D. et RNA an assay to detect 2018; S. of and 2019; experimental for a assay is and relative The criteria for precision and on the of the a and two (or a for in vitro transcription (e.g., or followed a of To on the each such also the of a assay with a for the a number of assays for are to be one with each a of for the reference the number of the of using the is not one has to on the the to the a for and the to the of the the the of can be is the of the can also be more than (or difference that is for the a be an RNA an RNA sample for for the RNA sample and be such RNA is to be used as a for the RNA or the is for one have to on commercial such as the RNA of The of RNA and have to be have been found to the RNA for the RNA an test can be an vitro transcription of the two the and two of and RNA to RNA with of and the of RNA as sample only of RNA with variable of with variable RNA with RNA The RNA provides for a more RNA RNA sample of be RNA from is often a RNA for all RNA and for RNA an not in vitro transcription and can be and clinical with be for for is one and of to with to the of of the for the two of RNA measurements to RNA and specificity on in for all assays on of with from RNA from clinical specificity on from the using for assays using of and from for from clinical and for the of the The typically as the of the absence or of on the the of the assay RNA assay would relative for RNA as for for would an impact of of RNA measurements to the trueness and of the assay an the from the and from the and reverse assays for the reference and a of (or number as will be used in the of the within the the of one and reverse transcription and the of and reverse for the their relative with the of the relative of the reference J. G. de A. J. relative quantification and for management and of real-time quantitative PCR 8: trueness the the relative in a the for the two within the measurements to the precision of the measurements of a a on a precision of the assay the for the two between the The for only the of the including The for the also the of of the and of a steps to relative quantification assay and or a the from a reference RNA sample reverse transcription sample or than the value of the test used in the sample or than the value of the test used in the are they within their or reference are used to their within their value J. de de A. of real-time quantitative of J. G. de A. J. relative quantification and for management and of real-time quantitative PCR 8: steps to quantification assay and or a the sample an result or a result the of the the the is between and or there are in the the for the guidelines which are in are not and are to researchers in their assays in CR and clinical trials. are not to validation to the consensus within the European consortium COST Action on a of validation These guidelines are on the analytical validation of aspects are not assays, or or These assays that are guidelines or are for reference guidelines in based on the from the CR the in using the consensus in the and and and in research in the has to in the and specific of the and of to the identification of biomarkers that are in clinical practice. Despite the promising findings, qPCR-based tests for RNA quantification have experienced serious limitations in their clinical application. The development and of novel is a and assay validation the and of recommendations for CR assays the and of assays from the research laboratory to clinical and the recommendations can fill the gap between RUO and are the of a of the EU-CardioRNA consortium with collaboration and the European for and the and with of the of is to the of biomarkers in their development to and to and of and are that of guidelines will result in more biomarkers development for diseases that in clinical practice. IntroductionA literature search on biomarkers and cardiovascular diseases (CVDs) highlights the poor correlation between the efforts performed in the initial steps of the development of quantitative PCR (qPCR) assay-based biomarkers, i.e., discovery and preclinical stages and their incorporation into clinical practice. There are a number of barriers that contribute to this poor implementation. The lack of technical standardization constitutes a key limitation in the incorporation of qPCR-assay-based biomarkers into the clinic. Limitations are also linked to the absence of consensus reference values, poor harmonization of the study populations, and the barriers in collaboration between academia, physicians, and industry. For instance, despite the thousands of noncoding RNA (ncRNA)-based biomarker studies published to date, there is a paucity of potential indicators that have been successfully translated into clinical practice, mainly due to the lack of reproducibility of research findings. Kok et al.1Kok M.G.M. de Ronde M.W.J. Moerland P.D. Ruijter J.M. Creemers E.E. Pinto-Sietsma S.J. Small sample sizes in high-throughput miRNA screens: a common pitfall for the identification of miRNA biomarkers.Biomol. Detect. Quant. 2018; 15: 1-5Google Scholar nicely illustrate the situation for coronary artery disease (CAD)-associated circulating microRNA (miRNA) biomarkers based on a literature review yielding 13 miRNAs found to be up- or downregulated in more than one study, of which more than half (7 out of 13) showed a contradictory result between studies (e.g., for miR-21, two studies showed upregulation and one study showed downregulation). This lack of reproducibility has also been addressed in several publications,2Faraldi M. Gomarasca M. Sansoni V. Perego S. Banfi G. Lombardi G. Normalization strategies differently affect circulating miRNA profile associated with the training status.Sci. Rep. 2019; 9: 1584Google Scholar, 3Godoy P.M. Barczak A.J. DeHoff P. Srinivasan S. Etheridge A. Galas D. Das S. Erle D.J. Laurent L.C. Comparison of reproducibility, accuracy, sensitivity, and specificity of mirna quantification platforms.Cell Rep. 2019; 29: 4212-4222.e5Google Scholar, 4Leidner R.S. Li L. Thompson C.L. Dampening enthusiasm for circulating microRNA in breast cancer.PLoS One. 2013; 8: e57841Google Scholar with reported causes ranging from technical analytical aspects to variable patient inclusion criteria and underpowered studies to sample quality. As such, the field of in vitro diagnostics (IVD)-grade quantitative reverse transcription PCR (qRT-PCR) assays for clinical use, initially developed in research laboratories, is still in its infancy. The incorporation of novel molecular biomarkers for clinical decision-making and patient management, i.e., diagnosis, prognosis, prediction, and monitoring of the therapeutic response or toxicity, need clear assay validation guidelines to be followed in the context of clinical research.In this context, basic and clinical researchers often resort to the use of laboratory-developed assays with variable and undefined quality, commercial research use only (RUO) assays or, in the best-case scenarios, laboratory-developed assays validated in accordance with guidance such as minimum information for the publication of quantitative real-time PCR experiments (MIQE) guidelines.5Bustin S.A. Benes V. Garson J.A. Hellemans J. Huggett J. Kubista M. Mueller R. Nolan T. Pfaffl M.W. Shipley G.L. et al.The MIQE guidelines: minimum information for publication of quantitative real-time PCR experiments.Clin. Chem. 2009; 55: 611-622Google Scholar The difference between such assays and certified IVD assays is significant. Laboratory-developed assays for clinical research are typically less controlled and standardized and do not need to comply with regulations, such as the European In Vitro Diagnostic Regulation (IVDR 2017/746). The European regulatory framework, based on the aforementioned IVDR and the Clinical Trials Regulation 2014/536, leaves a gray area relative to the status of laboratory assays that are used in the context of clinical trials. Poorly validated assays are not appropriate for large-scale clinical biomarker studies. Therefore, researchers would benefit from guidelines on the validation of what we refer to as clinical research (CR) assays, an assay type filling the gap between RUO and IVD that addresses the specific needs of researchers in the development of biomarkers. To some degree, such CR assays are similar to laboratory-developed test (LDT) assays in that they have undergone more thorough validation without reaching the status of a certified IVD assay.GlossaryAnalytical precision (or precision): closeness of two or more measurements to each otherAnalytical sensitivity: the ability of a test to detect the analyteAnalytical specificity: the ability of a test to distinguish target from nontarget analytesAnalytical trueness / analytical accuracy: closeness of a measured value to the true valueClinical research: in this article, “clinical encompasses clinical studies and their of which the are to therapeutic or understanding of disease Clinical typically to laboratory in clinical of that the appropriate use of a or a that the of validation associated with a development tool is to its context of the ability of a test to the absence of the disease in with test the ability of a test to disease in with / of that are / of that are a CR researchers can more out RUO assays that are and to in the stages of biomarker research to test or clinical laboratory This is in Here, we will focus on the necessary steps that need to be taken toward the appropriate validation of qRT-PCR for CR and clinical the of this review is not to regulatory guidance for with The is to and technical in the specific context of qRT-PCR for which are not to or are to researchers of the

Topics & Concepts