Challenges to sharing sample metadata in computational genomics

Abstract

The genomic data deluge has led to challenges with sharing and integrating genomic data. While substantial effort has been devoted to making genomics data easier to share, one challenge that has received little attention is the related goal of sharing genomic metadata, or attributes of biological samples. Genomic metadata is distinct from genomic data in many important ways that affect the optimal way to share it. Here, we outline several challenges specific to sharing metadata associated with genomic data. We argue that sharing genomic metadata is an important and underserved area in genomics, and that addressing this strategically could lead to alternative sharing paradigms with potential to improve the overall computational genomics ecosystem.While much effort has been placed on the idea of sharing genomic data, it is helpful to distinguish between genomic data and metadata. In genomics, data is generally produced by a DNA sequencer, whereas metadata describes the sample from which these sequences were derived. Genomic data is inherently sample-centric: most genomic data is naturally derived from a biological sample. The attributes of these samples comprise the metadata. Metadata can be categorized into several types (Fig. 1A): 1) inherent attributes describe essential characteristics of a sample, such as species or cell line; 2) experimental attributes describe processing features, such as treatments, experimental conditions, or library preparation protocols; finally, 3) analytical attributes describe inputs or outputs to data analysis, such as parameters or reference genome used. For example, in an RNAseq experiment, the metadata may include inherent attributes like the sample cell type (K562), experimental attributes like treatment (DMSO), and analytical attributes like paths to data stored in .fastq files.Broad discussions of sharing genomic data don't always distinguish metadata from the data itself. This ignores important differences that affect the challenges and opportunities for sharing. The characteristics of the data and metadata are different enough that the two warrant different strategies for sharing. Three relevant differences with important sharing implications are 1) size; 2) source; and 3) use case. Each of these differences has important implications that change the optimal strategy for sharing (Fig. 1B).One of the major challenges in sharing genomic data is size. In fact, this is a driving factor that is shaping our sharing strategies and driving huge investments in infrastructure [1,2]. However, metadata is typically much smaller than the data it describes. While genomic data may contain many thousands of sequencing reads for a single sample, the metadata for the same sample might only require a few simple annotations. As such, although it makes sense to avoid data transfer by bringing compute to the data for large genomic datasets [1], this argument simply doesn't apply to metadata, which is relatively cheap to duplicate and distribute. Lumping the two together therefore creates unnecessary barriers to metadata sharing.Another important difference between data and metadata is that metadata is typically created and curated by humans, rather than by machines. Genomic data is overwhelmingly generated by high-throughput sequencers, which have matured to the point of producing standardized file formats which are computer-friendly from the beginning. The primary creator and consumer of these files is machines, as it's impractical for humans to manually explore hundreds of millions of genome fragments.This machine-centric quality creates a self-regulating standardization process for genomic data. In contrast, metadata is more frequently created and consumed by humans. This drastically increases the diversity of metadata, which makes it more challenging to consume, integrate, and analyze by computer. While sequences can all be integrated and processed similarly by software, metadata cannot. This leads to metadata-specific challenges in sharing. Finally, another key sharing-related difference between genomic data and metadata is how it is used. Of course, both data and metadata are likely required for any type of re-analysis, but metadata also has an additional specific use case: it is required for finding the data of interest in the first place. A search for data of interest is likely to need access to metadata in order to determine whether the data is relevant. Finding relevant data requires sifting through lots of potentially irrelevant datasets. As a result, metadata will be much more frequently viewed than data, making it even more important for sharing metadata to be simple, easy, fast, and cheap.Given the distinctions between data and metadata, it's clear that sharing metadata warrants a dedicated strategy. This strategy should address challenges specific to sharing metadata, which can be grouped into 6 areas: 1) standardizing terms; 2) standardizing formats; 3) distribution; 4) findability; 5) versioning; and 6) portability. These challenges span the life-cycle of metadata use, including discovery, access, harmonization, and processing (Fig. 1C).The first step to reusing data is finding it. However, because metadata are not centralized, but scattered across various servers and databases, finding relevant data can be a challenge. In addition to the general challenge of multiple sources to find data, the problem is exacerbated by the inability for computers to parse and index some metadata, such as PDFs or Excel files. Finally, authorization barriers inhibit findability. Though there has been some effort to create centralized search frameworks or open API-oriented systems [3], existing tools are still covering only a small amount of the possible search space.Distribution of genomic metadata is also a challenge. The status quo is ad hoc; there are a variety of different distribution mechanisms, and none is particularly machine-friendly. Much genomic metadata is deposited onto data-oriented databases, such as GEO or dbGap, where metadata is notoriously difficult to process, leading to a variety of dedicated tools for that purpose [4][5][6][7][8][9][10][11][12]. Furthermore, metadata may be deposited under the same access restrictions as the data itself for convenience, because the repository may not be set up to separate the two (Fig. 1D). This convenience violates a tenet of the FAIR philosophy, that metadata should be accessible even if the data itself has restricted access [13]. Another common distribution mechanism is through supplementary files attached to journal publications, but this is highly non-standard and is not amenable to easy meta-analysis or reuse. One attempt at making genomic metadata easier to distribute and parse is the GenoMetric Query Language (GMQL), a declarative language that provides abstractions of experimental, biological, and clinical metadata. [14].A major challenge to sharing and re-using genomic metadata is that terms must match. One way to address this challenge is with ontologies. Ontologies provide vocabularies with controlled terms and definitions. They may also provide information about relationships among those terms.Creating ontologies is laborintensive and requires coordination and community, but fortunately, many ontologies already exist for a variety of biomedical use cases [15][16][17][18]. Unfortunately, in practice, researchers do not necessarily use existing ontologies [19]. One barrier is that the benefits of ontologies may become most apparently only in integrative meta-analysis. Therefore, for an individual study, an ontology may be viewed as merely added cost.We can address this in two ways: either reducing the cost or increasing the value of using an ontology for individual studies. Reducing the cost means lowering the barrier to using controlled terms. There is an opportunity for tools that make it more user-friendly to use an ontology by suggesting controlled terms or mapping existing ontologies while metadata is being created. On the other side, we could approach the problem by adding value to an individual study that uses controlled terms. For instance, we could promote tools that will automatically integrate a new result with external data, even if this isn't the primary analysis of the study. An example could be gene identifiers: even for a standalone study, researchers want to analyze results in the context of existing public resources, so they must use standardized gene names. Another possibility is to use machine learning approaches to standardize terminology post hoc [10].It is not sufficient for two studies to use the same ontology and share controlled terms; they must also structure the data in the same way. Genomic metadata frequently adopts a tabular form, with rows corresponding to samples or files, and columns corresponding to attributes of the samples or files. However, genomic metadata may also adopt schema-less, document-based file formats. Furthermore, sample attributes are sometimes encoded in less machine-friendly ways, such as using text formatting or color to mark samples in Microsoft Excel files. Making metadata machine-understandable is a difficult challenge. Even if file formats and general structures are consistent, subtle differences may prevent integration. For example, CSV files can be one row per sample, or one row per file, or one row per sequencing lane (Fig. 1E). These distinctions make samples tables non-interoperable, which in turn makes it difficult or impossible to integrate, hindering integrative re-analysis of data. Several attempts have been made to address this issue in general, such as ISA-tab [20], the PEP framework [21], and META-BASE [22].Metadata can change. Inherent and experimental attributes are mostly stable but may be edited or added. Furthermore, analytical attributes are frequently added to a metadata table as analysis progresses. Despite clear mutability, metadata tables are often treated as static.Version control is well established for code and has a diverse and multifaceted history for data as well [23], but the question of versioning metadata specifically is distinct. A common strategy for versioning sample metadata is to use tools built for code versioning, such as git. However, the fit is not perfect; the line-based nature of git and other code version control systems is less suited to a sample table which may have long lines. A more tailored approach would use a table-cell-based framework, but a bespoke tool for table versioning does not exist. In lieu of this, a common approach is to develop a protocol for recording revisions, typically involving incrementing version numbers in file names [24]. Also common is to simply not version control metadata. There are clear opportunities for innovation, new standards, and tool development to support the specific needs of metadata versioning.A final challenge dealing with sample metadata is its portability. By portability, we mean whether relevance is retained if the data or metadata are moved to a different computing environment. Metadata often changes locations; e.g. from one collaborator's computer to another, from a high-performance computing environment to a web repository to a laptop. In this process, some attributes lose their relevance: Although inherent and experimental attributes tend to be portable, many analytical properties are not. For example, sample tables often include file paths, but paths typically refer to a specific computing environment. Another example is properties used as input to a pipeline. For instance, the reference genome used is often included as an attribute in a table; however, it not a property of a sample itself, but of a particular analysis. If the sample table is reprocessed, this attribute changes. This distinction between portable and non-portable metadata is typically ignored, so genomic metadata includes both in a single table, which renders the table specific to a computing environment and thereby reduces its portability. This problem motivated the Portable Encapsulated Project framework [21], which allows environment-specific settings to be extracted from the same table into a configuration file that can change with the environment. There is opportunity for new approaches to conceptualize sample attributes in ways to acknowledge this portability problem to treat metadata attributes according to their portability.Genomic metadata is clearly distinct enough from genomic data itself to warrant a specific sharing strategy, which should address metadata-specific challenges including findability, distribution, terminology, structure, identification, and portability. Perhaps the greatest challenge to sharing metadata is caused by the overwhelming complexity introduced by its human-curated nature.The primary goal of addressing these concerns is to improve the interoperability of sample metadata. Interoperability is a prerequisite for integration and drives reuse. Only by solving this challenge will we be able to benefit from the knowledge that emerges from studying existing published genomic datasets systematically, instead of as individual studies. Obviously, metadata sharing is just the beginning; ultimately, and important challenges remain with sharing the data itself. Nevertheless, our attempts to integrate data will remain limited until we address the challenges at metadata level that warrant specific attention. Increasing metadata interoperability will reduce a major barrier, taking us one step closer to the vision where genomic data integration is commonplace.

Topics & Concepts