Artificial Intelligence Improves Pathologist Agreement for Fibrosis Scores in Nonalcoholic Steatohepatitis Patients

Abstract

Nonalcoholic fatty liver disease is increasingly recognized as a global health concern in tandem with metabolic syndrome. The term encompasses a spectrum of phenotypes, including simple steatosis and nonalcoholic steatohepatitis (NASH), which can be associated with fibrosis. The presence of significant fibrosis is the most important histologic feature associated with mortality in patients with NASH1Dulai P.S. et al.Hepatology. 2017; 65: 1557-1565Crossref PubMed Scopus (842) Google Scholar; reliably detecting its presence and severity has therefore been of great clinical and research interest. Despite advances in noninvasive assessment of fibrosis in patients with NASH, histologic diagnosis and assessment of NASH and fibrosis is currently still regarded as the gold standard and is included by the US Food and Drug Administration and the European Medicines Agency in their guidance for subject enrolment and primary end points in NASH clinical trials. However, interobserver agreement for categorical scores of liver fibrosis among pathologists has been reported to range from fair to moderate,2Davison B.A. et al.J Hepatol. 2020; 73: 1322-1332Abstract Full Text Full Text PDF PubMed Scopus (106) Google Scholar,3Kleiner D.E. et al.Hepatology. 2005; 41: 1313-1321Crossref PubMed Scopus (6754) Google Scholar which could adversely affect the outcomes of clinical trials in terms of study entry criteria and assessing treatment effects.2Davison B.A. et al.J Hepatol. 2020; 73: 1322-1332Abstract Full Text Full Text PDF PubMed Scopus (106) Google Scholar Thus, there have been increasing efforts to leverage on advances in artificial intelligence (AI) and digitized whole-slide images to develop various AI-assistive tools to improve/replace manual histopathologic interpretation. Recent endeavors include machine learning models based on deep convolutional neural networks requiring pixel-level annotations for supervised model training,4Taylor-Weiner A. et al.Hepatology. 2021; 74: 133-147Crossref PubMed Scopus (26) Google Scholar and algorithms based on second harmonic generation (SHG)/2-photon excitation fluorescence laser microscopy, which allows the identification, localization, and characterization of collagen fibers.5Xu S. et al.J Hepatol. 2014; 61: 260-269Abstract Full Text Full Text PDF PubMed Scopus (98) Google Scholar, 6Leow W.Q. et al.Diagnostics (Basel). 2020; 10: 643Crossref Scopus (5) Google Scholar, 7Wang Y. et al.Gut. 2020; 69: 1116-1126Crossref PubMed Scopus (14) Google Scholar qFibrosis, first established and validated on core biopsies from patients with chronic hepatitis B5 and recently refined for patients with NASH,6Leow W.Q. et al.Diagnostics (Basel). 2020; 10: 643Crossref Scopus (5) Google Scholar is one such example that provides an automated assessment of liver fibrosis with a stain-free approach, incorporating quantitation and architectural localization. In this pilot study, we aim to determine if AI-assistive tools, such as qFibrosis, can improve agreement of fibrosis assessment by pathologists on patients with NASH. Three pathologists assessed fibrosis on digitized hematoxylin-eosin and Masson trichrome–stained images (unassisted read), and again with additional SHG image, qFibrosis stage and continuous values quantitated using SHG/2-photon excitation fluorescence microscopy on unstained sections (assisted read), in 2 sessions separated by a wash-out period of at least 3 weeks (Supplementary Methods). The demographic characteristics of the 40 Chinese subjects included in this pilot study are shown in Supplementary Table 1. The samples comprised 10 F0, 10 F1, 10 F2, 5 F3, and 5 F4. Examples of each fibrosis stage in each modality (hematoxylin-eosin, Masson trichrome, SHG image with qFibrosis stage and continuous value) are illustrated in Supplementary Figure 1. The results of pathologist scoring across the unassisted reads and assisted reads were examined to determine the interrater reliability. With qFibrosis assistance, the concordance rate between pathologists improved substantially, with 0.82 mean linearly weighted Kappa, as compared with 0.72 for the unassisted review (Table 1). Correspondingly, the mean overall percentage agreement between pathologists improved from 89.38% to 92.93% (P = .032). The mean linearly weighted Kappa for intraobserver agreement was also higher, achieving 0.91 kappa compared with 0.79 for unassisted reads.Table 1Mean Percentage Agreement and Linearly Weighted Kappa Results for Interobserver and Intraobserver Unassisted and Assisted Reads of Liver Fibrosis Scoring in NASH PatientsLiver fibrosis scoringUnassisted readAssisted readInterobserverMean percentage agreement89.37%92.92%Mean linearly weighted Kappa0.720.82IntraobserverMean percentage agreement92.08%96.46%Mean linearly weighted Kappa0.790.91NASH, nonalcoholic steatohepatitis. Open table in a new tab NASH, nonalcoholic steatohepatitis. The effect of AI assistance on fibrosis assessment consistency was further examined by reviewing the percentage of patients who remained staged between F1 and F3 after the initial reads (retention rate). With qFibrosis assistance, on average, 95.3% of the patients were retained in the subsequent read versus 82% for the unassisted review. Pathologic scoring of fibrosis in liver biopsies is dependent on many variables. These include technical variances, such as section quality and collagen staining intensity, and sampling errors inherent in the biopsy procedure, which is further exacerbated by suboptimal samples.8Ooi G.J. et al.Surg Endosc. 2021; 35: 1210-1218Crossref PubMed Scopus (13) Google Scholar Furthermore, fibrosis is a continuous and dynamic process with heterogeneity even within a single liver biopsy. Assessment requires not just an estimation of collagen present but also the presence of architectural/vascular alterations. The standard categorical semiquantitative histologic scoring systems are inadequate in reflecting the complexity of the process. This is a likely contributor to intraobserver and interobserver variability in current pathologic fibrosis scoring: should a liver biopsy with only 1 regenerative hepatocytic nodule be scored F3 or F4? Recognition of these factors contributing to poor intrarater and interrater agreement is important to address concerns regarding suboptimal reliability of liver biopsy scoring adversely impacting subject enrolment and end point assessment in clinical trials. Assessment methods based on recent advances in technologies, such as AI and whole-slide images, hold much promise in mitigating some of the limitations of current pathologic scoring systems by identifying elements currently undetected by pathologist scoring,4Taylor-Weiner A. et al.Hepatology. 2021; 74: 133-147Crossref PubMed Scopus (26) Google Scholar,9Forlano R. et al.Clin Gastroenterol Hepatol. 2020; 18: 2081-2090Abstract Full Text Full Text PDF PubMed Scopus (40) Google Scholar including in samples less than 1.5–2 cm.10Wang B. et al.Hepatol Int. 2019; 13: 501-509Crossref PubMed Scopus (5) Google Scholar In this study, we have demonstrated that qFibrosis as an AI-assistive tool improves interpathologist-weighted Kappa to near-perfect agreement with 93% agreement and 95% retention rate among pathologists with varying experience. qFibrosis is a fully quantitative stain-free method for the automated assessment of liver fibrosis that incorporates spatial architectural features of pathologic relevance at the tissue level.5Xu S. et al.J Hepatol. 2014; 61: 260-269Abstract Full Text Full Text PDF PubMed Scopus (98) Google Scholar This approach is based on an understanding of the pathophysiology of fibrosis; the algorithm can identify perisinusoidal fibrosis and was also recently refined and validated in patients with NASH to identify F2 fibrosis with more accuracy.6Leow W.Q. et al.Diagnostics (Basel). 2020; 10: 643Crossref Scopus (5) Google Scholar Discrimination between F1 and F2 on connective tissue stains alone can be challenging for pathologists. The provision of qFibrosis staging and continuous values in the assisted reads in our study could have helped standardize pathologist assessment in such cases, contributing to a higher overall intrarater and interrater agreement. This is valuable to prevent incorrect classification and erroneous exclusion of understaged F2 patients in pivotal NASH clinical trials (noncirrhotic NASH with liver fibrosis). A larger validation study is planned to further evaluate the effects of AI-assistance in particular scenarios of clinical relevance (eg, F1 vs F2, F3 vs F4). Because qFibrosis itself may not achieve 100% recognition of architectural features, with situations arising whereby all 3 pathologists disagreed with the qFibrosis value, an adjudication panel of pathologists will also be included as an additional arm to establish the “ground truth” for fibrosis. With AI-assistive tools, such as qFibrosis, pathologic assessment can be further standardized and will remain an important element in determining subject eligibility and assessing treatment effects in NASH clinical trials. The authors thank HistoIndex, Singapore for their help with the technical aspects of this study and for performing data analysis. Deidentified liver biopsy samples from 40 untreated adult patients (≥18 years old) with biopsy-proven nonalcoholic fatty liver disease/NASH and a range of fibrosis scores (F0–F4) from Peking University People’s Hospital were included in this study. Patients with concomitant liver conditions, such as viral hepatitis, autoimmune hepatitis, or alcoholic or drug-induced liver disease, were excluded. This study was approved by the Ethics Committee of Peking University People’s Hospital (No. 2017PHB133-01). SHG/2-photon excitation fluorescence microscopy was performed on 1 unstained liver tissue section 4–5 μm thick using the commercially available Genesis system (HistoIndex Pte Ltd, Singapore) in methods previously described.6Leow W.Q. et al.Diagnostics (Basel). 2020; 10: 643Crossref Scopus (5) Google Scholar Briefly, samples were laser-excited at 780 nm, SHG signals were recorded at 390 nm, and 2-photon excitation fluorescence signals were recorded at 550 nm. Images were acquired at 20X magnification with 512 × 512 pixels resolution; each image tile had a dimension of 200 × 200 μm. Multiple adjacent image tiles were captured to encompass the whole tissue area. The unstained sections were subsequently stained with hematoxylin-eosin and Masson trichrome, digitized, and uploaded onto a whole-slide image platform. The median length of the biopsies was 15.69 mm (9.20–34.82). Three pathologists with experiences ranging from 5 to 40 years were blinded to the liver samples. In the unassisted read, pathologists independently scored the biopsies based on the hematoxylin-eosin and Masson trichrome digitized slides according to the NASH CRN scoring system (unassisted read). Subsequently, the pathologists rescored the biopsies, but with the added provision of the SHG image, qFibrosis stage, qFibrosis continuous value, and cutoff values for each biopsy (assisted read). This was repeated after a washout period of at least 3 weeks (total of 4 reads). All statistics were calculated using statistical software MATLAB R2021a. The linearly weighted Kappa and percentage agreement were calculated within and between each pair of pathologists (A, B, and C) for unassisted and assisted reads.Supplementary Table 1Demographic Characteristics of the 40 SubjectsAll subjects (n = 40)Mean age, y41.03 ± 13.88Male gender, n (%)22 (55.00)Chinese ethnicity, %100Body mass index, kg/m226.88 ± 3.14Albumin, g/L42.90 ± 5.61Bilirubin, μmol/L11.38 ± 6.27Alanine transaminase, U/L94.36 ± 66.56Aspartate transaminase, U/L59.05 ± 32.35γ-Glutamyltransferase, IU/L90.08 ± 68.27Alkaline phosphatase, U/L93.65 ± 28.33Platelet count, x109Forlano R. et al.Clin Gastroenterol Hepatol. 2020; 18: 2081-2090Abstract Full Text Full Text PDF PubMed Scopus (40) Google Scholar/L219.09 ± 63.66Glucose, mmol/L5.44 ± 1.29Triglyceride, mmol/L3.03 ± 1.72Histologic steatosis grade, n (%) No steatosis, S0 (<5% steatosis)0 (0) Mild steatosis, S1 (5%–33% steatosis)26 (65) Moderate steatosis, S2 (>33%–66% steatosis)8 (20) Severe steatosis, S3 (>66% steatosis)6 (15) Open table in a new tab

Topics & Concepts