This paper is in the following e-collection/theme issue:

Precision Medicine (63) Electronic Health Records (961) Genomics and Bioinformatics for Clinical Use (28)

Published on in Vol 23, No 4 (2021): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/26261, first published .
Fast Healthcare Interoperability Resources (FHIR)–Based Quality Information Exchange for Clinical Next-Generation Sequencing Genomic Testing: Implementation Study

Fast Healthcare Interoperability Resources (FHIR)–Based Quality Information Exchange for Clinical Next-Generation Sequencing Genomic Testing: Implementation Study

Fast Healthcare Interoperability Resources (FHIR)–Based Quality Information Exchange for Clinical Next-Generation Sequencing Genomic Testing: Implementation Study

Authors of this article:

Donghyeong Seong1 Author Orcid Image ;   Sungwon Jung1 Author Orcid Image ;   Sungchul Bae2 Author Orcid Image ;   Jongsuk Chung3 Author Orcid Image ;   Dae-Soon Son4 Author Orcid Image ;   Byoung-Kee Yi2, 5 Author Orcid Image
Article Authors Cited by (3) Tweetations (1) Metrics

    Original Paper

    1Department of Health Sciences and Technology, Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, Republic of Korea

    2Smart Healthcare Research Institute, Samsung Medical Center, Seoul, Republic of Korea

    3Samsung Genome Institute, Samsung Medical Center, Seoul, Republic of Korea

    4School of Big Data Science, Data Science Convergence Research Center, Hallym University, Chuncheon, Republic of Korea

    5Department of Digital Health, Samsung Advanced Institute for Health Sciences & Technology, Sungkyunkwan University, Seoul, Republic of Korea

    *these authors contributed equally

    Corresponding Author:

    Byoung-Kee Yi, PhD

    Smart Healthcare Research Institute

    Samsung Medical Center

    81 Irwon-ro

    Gangnam-gu

    Seoul, 06351

    Republic of Korea

    Phone: 82 2 3410 1944

    Email: byoungkeeyi@gmail.com


    Background: Next-generation sequencing (NGS) technology has been rapidly adopted in clinical practice, with the scope extended to early diagnosis, disease classification, and treatment planning. As the number of requests for NGS genomic testing increases, substantial efforts have been made to deliver the testing results clearly and unambiguously. For the legitimacy of clinical NGS genomic testing, quality information from the process of producing genomic data should be included within the results. However, most reports provide insufficient quality information to confirm the reliability of genomic testing owing to the complexity of the NGS process.

    Objective: The goal of this study was to develop a Fast Healthcare Interoperability Resources (FHIR)–based web app, NGS Quality Reporting (NGS-QR), to report and manage the quality of the information obtained from clinical NGS genomic tests.

    Methods: We defined data elements for the exchange of quality information from clinical NGS genomic tests, and profiled a FHIR genomic resource to enable information exchange in a standardized format. We then developed the FHIR-based web app and FHIR server to exchange quality information, along with statistical analysis tools implemented with the R Shiny server.

    Results: Approximately 1000 experimental data entries collected from the targeted sequencing pipeline CancerSCAN designed by Samsung Medical Center were used to validate implementation of the NGS-QR app using real-world data. The user can share the quality information of NGS genomic testing and verify the quality status of individual samples in the overall distribution.

    Conclusions: This study successfully demonstrated how quality information of clinical NGS genomic testing can be exchanged in a standardized format. As the demand for NGS genomic testing in clinical settings increases and genomic data accumulate, quality information can be used as reference material to improve the quality of testing. This app could also motivate laboratories to perform diagnostic tests to provide high-quality genomic data.

    J Med Internet Res 2021;23(4):e26261

    doi:10.2196/26261

    Keywords

    FHIR; clinical NGS genomic testing; clinical massive parallel sequencing; quality control; genomic reporting 


    Next-generation sequencing (NGS) technology has been rapidly adopted in clinical practice, with the scope extended to early diagnosis, disease classification, and treatment planning [1-3]. To implement clinical NGS applications, health care providers operate their own NGS laboratories or request genomic tests to external laboratories. The NGS genomic testing reports provide information regarding genomic variations and related data. However, different templates and data elements are used depending on the laboratory, and most reports are provided in text or PDF format [4,5]. As the number of requests for NGS genomic testing increases, considerable efforts have been made to deliver the testing results clearly and unambiguously [6-10]. Moreover, international standards development organizations have developed reporting standards such as the Fast Healthcare Interoperability Resources (FHIR) Genomics Reporting Implementation Guide and International Organization for Standardization (ISO) standards to exchange complex clinical genomic data and interpretations [11-14].

    For the legitimacy of clinical NGS genomic testing, the quality information from the process of producing the genomic data should be included within the test results [15-17]. The demand for genomic testing of small tissue samples and needle biopsies is increasing [18], and it remains challenging to determine the reliability of the genomic test results. When testing small quantities or low-quality samples, including low-purity specimens and formalin-fixed paraffin-embedded (FFPE) specimens, quality information can be considered for the interpretation of diagnostic results as evidence. Confirming that there is no specific variation is a particular challenge, especially without quality information. For the validity and utility of clinical NGS genomic testing, the US Centers for Medicare and Medicaid Services regulates all laboratory tests performed on humans through the Clinical Laboratory Improvement Amendments, and the US Food and Drug Administration (FDA) requires information regarding the clinical validity for genomic tests [19,20]. In Korea, the Ministry of Food and Drug Safety (MFDS) has initiated the clinical laboratory accreditation program since 2017 [21]; however, this provides accreditation of the entire NGS process rather than the reliability of individual samples. When a clinician performs NGS genomic testing for diagnostic purposes, quality information can be used to interpret the results for individual samples. In addition, when constructing a reference database using these samples, it is essential to include this quality information.

    The successful practice of precision medicine depends on clinical genomic data sharing and knowledge-based interpretations of genomic variant data at the point of care [11]. To improve interoperability as part of precision medicine, health care stakeholders encourage the use of application programming interfaces (APIs) and app-based ecosystems such as SMART on FHIR, CDS Hooks, and SMART Markers [22-26]. These platforms enable easy implementation for health care use cases and facilitate functional extensibility. There are several apps based on clinical genomics use cases on the SMART on FHIR platforms, such as the SMART Precision Cancer Medicine and SMART Cancer Navigator apps [27-29].

    In this study, we developed the NGS Quality Reporting (NGS-QR) app to exchange the quality information of clinical NGS genomic testing. To exchange the information in a standardized format, we profiled a FHIR genomic resource based on ISO/TS 22692:2020 Genomics Informatics-Quality Control Metrics for DNA sequencing, which defines the quality-related data for the entire NGS process, including sample preparation, library preparation, sequencing, and data processing [15]. This app enables the performance comparison of clinical NGS genomic testing and monitoring of the quality status of the current sample as the data accumulate.


    Overview

    This study describes the development of the NGS-QR app, a FHIR-based web app, to report, manage, and monitor the quality information from the process of producing genomic data. As shown in Table 1, the development was composed of the following phases: (1) requirement analysis, (2) design, (3) implementation, and (4) testing. In the requirement analysis phase, we defined use cases and selected data elements such as DNA purity and integrity, library input amount and size, and sequencing running quality, in accordance with standards and guidelines. In the design phase, we profiled a FHIR resource on the existing genomic resource to exchange quality information in a standardized manner. We also designed user interfaces and functions for the NGS-QR app. In the development phase, we developed the three following components: web app (NGS-QR app), FHIR server, and R Shiny server. The web app includes a FHIR resource handler to generate and parse FHIR resources. Finally, in the testing phase, the NGS-QR app was validated using real-world data collected from the NGS pipeline, a targeted sequencing platform.

    Table 1. Overview of development of the next-generation sequencing-quality reporting (NGS-QR) app.
    Development phaseDescription
    Requirement analysis

    		Use casesDefine use cases for quality information exchange of clinical NGSa genomic testing

    		Data elementsSelect data elements in accordance with standards and guidelines (eg, ISOb, US FDAc, ACMGd)
    Design

    		FHIRe genomic profileProfile a FHIR resource on the existing genomic resource, MolecularSequence

    		System componentsDesign user interfaces and functions of the NGS-QR app
    Implementation

    		Web app (NGS-QR app)Develop user interfaces and functions of the web app; develop a FHIR resource handler to generate and parse FHIR resources

    		FHIR serverDevelop a FHIR server and repository; apply the FHIR QcMetrics profile to the FHIR server

    		R Shiny serverDevelop R code for statistical analysis of NGS experimental data
    Testing

    		Data collectionCollect real-world data generated from the NGS pipeline

    		Data exchangeExchange real-world data between the FHIR server and NGS-QR app using FHIR APIsf

    		Quality managementCheck the quality information of NGS genomic testing using the dashboard and statistical analysis

    aNGS: next-generation sequencing.

    bISO: International Organization for Standardization.

    cUS FDA: US Food and Drug Administration.

    dACMG: American College of Medical Genetics.

    eFHIR: Fast Healthcare Interoperability Resources.

    fAPIs: application programming interfaces.

    Use Cases

    We defined two use cases: quality information reporting and quality management for clinical NGS genomic testing. The first use case is related to the quality information reporting for clinical NGS genomic testing, as shown in Figure 1a. The laboratory reports the results of NGS genomic testing to the hospital along with quality information. This use case assumes that the hospital information system provides the FHIR APIs. The laboratory requested for the NGS genomic testing sends the results using the NGS-QR app, which acts as an FHIR client. When the user fills out the reporting form in the NGS-QR app, a FHIR resource is created in JSON format and then the resource is sent to the server via the FHIR API. Hospitals that request NGS genomic testing to multiple laboratories can receive information in the same format through this standardized method.

    The second use case is related to the quality management for clinical NGS genomic testing, as shown in Figure 1b. Users such as health care providers and performers of NGS testing manage and monitor the quality status of the genomic data received from the laboratories. Through the NGS-QR app, users can retrieve experimental data from the FHIR server and view the summary such as the total number of genomic tests and the number of tests based on the year or specimen type for all genomic test results. Moreover, quality information such as DNA purity, integrity, and data quality for each sample can be compared through statistical analysis. This step can be used to determine the reliability of clinical NGS genomic testing.

    ‎
    Figure 1. Use cases for exchanging the quality information of clinical next-generation sequencing (NGS) genomic testing. (a) Quality reporting for clinical NGS genomic testing. (b) Quality management for clinical NGS genomic testing. FHIR: Fast Healthcare Interoperability Resources.
    View this figure

    Data Elements Selection

    The NGS workflow involves complex procedures consisting of several steps, which are broadly divided into sample preparation, library preparation, sequencing, and data processing. In this process, various types of experimental conditions and results are generated and captured. As shown in Table 2, we selected data elements based on the ISO 22692 [15] and guidelines such as those of the FDA [20], American College of Medical Genetics [30], and College of American Pathologists [31] for quality management and comparative analysis. For this study, approximately 30 data elements were selected.

    Table 2. Next-generation sequencing (NGS) workflow and data elements.
    NGS workflowData elements
    Sample preparation

    		Sample sequencing typesequencing type, target gene

    		Sample informationspecimen type, sampling date
    Library preparation

    		DNA extractionDNA extraction kit, DNA purity (eg, ODa 260/280, OD 260/230), DNA integrity (eg, DNA median size)

    		Library constructionlibrary input amount, library input size, library construction kit
    Sequencing

    		Sequencing informationsequencing instrument, read length, sequencing direction, running mode

    		Running qualityerror rate, percent data quality (>Q30)
    Data processing

    		Data qualitytotal reads, mean coverage, uniformity, on-target rate, Q30, PRb score

    		Sequencing alignmentmapping algorithm, sequencing alignment software

    		Variant callingvariant calling software, quality score, allelic read percentage

    		Variant filtering and annotationgermline filter criteria, reference database

    aOD: optical density.

    bPR: pass rate.

    FHIR Genomics Resource Profile

    We profiled a FHIR genomic resource by defining constraints and extensions for exchanging quality information with a standardized method. Since the results of NGS genomic testing include genomic variant information and its quality-related data, the FHIR R4 MolecularSequence resource [32] was used for tailoring the NGS data to the use cases in this study. We created a QcMetrics element (a FHIR extension) in the resource and added data elements for each process under QcMetrics. Textbox 1 shows an example of a FHIR genomics resource that includes the QcMetrics profile. We uploaded the profile to the public FHIR registry SIMPLIFHIR.NET [33].

    Example of the Fast Healthcare Interoperability Resources genomic resource that includes the QcMetrics profile.

    {

    “resourceType”: “MolecularSequence”,

    “extension”: [

    {

    “url”: “http://example.org/fhir/StructureDefinition/QcMetrics”,

    “extension”: [

    {

    “url”: “dnaExtraction”,

    “extension”: [

    {

    “url”: “dnaExtractionKit”,

    “valueString”: “qiagen allprep DNA/RNA mini kit”

    },

    {

    “url”: “dnaPurity”,

    “extension”: [

    {

    “url”: “od260280”,

    “valueDecimal”: 2.1

    },

    {

    “url”: “od260230”,

    “valueDecimal”: 2.3

    }]

    },

    {

    “url”: “dnaIntegrity”,

    “units”: “bp”,

    “valueDecimal”: 60000

    }]

    },

    …

    }

    Textbox 1. Example of the Fast Healthcare Interoperability Resources genomic resource that includes the QcMetrics profile.

    Implementation

    We developed the following three major components: the NGS-QR app (FHIR client), FHIR server, and R Shiny server, as shown in Figure 2. The NGS-QR app was developed using Node.js (v12.13.1), which is composed of user interfaces (UIs), a FHIR resource handler, and a REST API module. The app has been deployed to the App Gallery of SMART on FHIR, which is an open platform for substitutable third-party health apps to connect to electronic medical record (EMR)/electronic health record (EHR) systems with appropriate security guarantees [34,35]. The source code of the app is available at GitHub [36]. The FHIR server was locally installed using the HAPI FHIR server (v4.0.0), which is an open-source Java implementation of the FHIR specification. The QcMetrics profile was added to the FHIR server. The source code for operating the R Shiny server was written in RStudio Cloud (R version 3.6.3) and the server was deployed to the R cloud platform, Shinyapp.io [37]. The R Shiny server fetched the data used for statistical analysis from the FHIR server. The R source code is available at GitHub [38].

    FHIR has a set of security recommendations that identifies communications security, authentication, authorization, access control, and auditing [39]. We applied Transport Layer Security (TLS) and OAuth 2.0, which are industry-standard protocols for communications security and authentication [40,41]. The TLS was used for the encryption of FHIR resources transmitted between the NGS-QR app and the FHIR server. The OAuth 2.0 protocol was used to grant the NGS-QR app access to the FHIR server. The NGS-QR app requests an access token by authenticating with the Authorization Server. The Authorization Server authenticates the NGS-QR app and issues an access token. The access token was used for security credentials for the NGS-QR app to make API requests on behalf of a user.

    ‎
    Figure 2. Concept model of the NGS-QR app, FHIR server, and R Shiny server. FHIR: Fast Healthcare Interoperability Resources.
    View this figure

    Data Collection

    Samsung Medical Center (SMC) has developed and utilized a cancer panel sequencing pipeline, namely CancerSCAN, to determine effective treatment methods for patients based on data from more than 15,000 panel sequencing studies since 2014 [17,42,43]. The SMC received clinical laboratory accreditation by the MFDS in 2017, and CancerSCAN is clinically used for the diagnosis and prognosis of cancer patients [17]. For this study, approximately 1000 data entries were collected from CancerSCAN. They contained the experimental conditions and results for FFPE specimens, fresh cells, and cell lines.

    User Interfaces and Functions

    In the reporting UI, input data are converted to a FHIR genomic resource in JSON format using the FHIR resource generator, and the resource is sent to the FHIR server using the POST method, as shown in Figure 3. In the dashboard UI, genomic resources in the FHIR server are compiled using the GET method to display the summary of data such as the total number of genomic tests and the number of tests based on the year and specimen type for the complete genomic test results. In the statistics UI, the user can select the group of samples, and view the distribution and threshold of each experimental parameter. The user can also check the current quality status of each sample in the overall distribution, as shown in Figure 4.

    ‎
    Figure 3. Screenshots of the user interface: reporting quality information of clinical next-generation sequencing (NGS) genomic testing.
    View this figure
    ‎
    Figure 4. Screenshots of the user interface: quality management of clinical next-generation sequencing genomic testing.
    View this figure

    Data Statistics

    We validated the utility of the NGS-QR app using real-world data from the NGS pipeline CancerSCAN in SMC. All data were converted to FHIR genomic resources and sent to the FHIR server using the NGS-QR app. Figure 5 shows the statistics of the experimental data used in this study. As expected, fresh cell samples had better quality than FFPE samples in almost every category.

    ‎
    Figure 5. Statistics of the experimental data used in this study. FFPE: formalin-fixed paraffin-embedded; OD: optical density; PR: pass rate.
    View this figure

    Principal Results

    NGS technology has been widely adopted in clinical practice, which will play a prominent role in precision medicine. Although many countries and international institutions have developed NGS guidelines to implement clinical NGS applications, the quality criteria for determining the reliability of NGS genomic testing have not yet been standardized due to the complexity of the NGS technology. Since the scope and purpose of NGS applications are very diverse, it remains challenging to evaluate the validity and utility of clinical NGS genomic testing. At present, most individual laboratories have their own processes and criteria; thus, it is important to share quality information from the process of generating genomic data.

    In this study, we propose the NGS-QR app to exchange quality information for clinical NGS genomic testing. This study provides the following main contributions to the field. First, we demonstrated that the quality information managed only by individual laboratories could be shared using a standardized format. In the NGS workflow, the experimental conditions change depending on the purpose of the tests and the status of the samples. Thus far, it has been difficult to determine the conditions under which the genome data were generated. This study defined quality-related elements and profiled the FHIR genomic resource to report the experimental conditions and results produced in NGS genomic testing. Since our proposed method is based on ISO/TS 22692 and HL7 FHIR standards to interoperably share the quality-related data of clinical genomic testing, the app can communicate with any EMR/EHR systems that conform to these standards.

    Second, this study facilitated the verification of the quality status of each sample as experimental data accumulate. The quality of genome data is determined through comparative analysis based on the characteristics of NGS technology. As shown in Figure 6, the results from different types of specimens cannot be directly compared because this may lead to incorrect conclusions. Therefore, it is important to select the target groups and determine the performance of individual samples within them. The NGS-QR app allows users to select various experimental conditions and compare the produced data with the correct target group to check the exact data quality.

    ‎
    Figure 6. Example of comparative analysis based on specimen type using the NGS-QR app. FFPE: formalin-fixed paraffin-embedded; PR: pass rate.
    View this figure

    Conclusions

    This study successfully demonstrated how the quality information of clinical NGS genomic testing can be exchanged using a standardized method. As the demand for NGS genomic testing increases and genomic data accumulate, quality information can be used as reference material for improving the quality of testing. This approach can also motivate laboratories to perform diagnostic tests to provide high-quality genomic data.

    Acknowledgments

    This work was supported by the Industrial Strategic Technology Development Program (grant number 10078282) funded by the Ministry of Trade, Industry & Energy (MOTIE) and by the Korea Health Technology R&D Project (grant number HI19C1026) through the Korea Health Industry Development Institute (KHIDI) funded by the Ministry of Health & Welfare, Republic of Korea.

    Conflicts of Interest

    None declared.

    1. Kamps R, Brandão RD, Bosch BJVD, Paulussen ADC, Xanthoulea S, Blok MJ, et al. Next-generation sequencing in oncology: genetic diagnosis, risk prediction and cancer classification. Int J Mol Sci 2017 Jan 31;18(2):308 [FREE Full text] [CrossRef] [Medline]
    2. Biesecker LG, Green RC. Diagnostic clinical genome and exome sequencing. N Engl J Med 2014 Jun 19;370(25):2418-2425. [CrossRef] [Medline]
    3. Katsanis SH, Katsanis N. Molecular genetic testing and the future of clinical genomics. Nat Rev Genet 2013 Jun;14(6):415-426 [FREE Full text] [CrossRef] [Medline]
    4. Tarczy-Hornoch P, Amendola L, Aronson SJ, Garraway L, Gray S, Grundmeier RW, et al. A survey of informatics approaches to whole-exome and whole-genome clinical reporting in the electronic health record. Genet Med 2013 Oct;15(10):824-832 [FREE Full text] [CrossRef] [Medline]
    5. Davis KW, Hamby Erby L, Fiallos K, Martin M, Wassman ER. A comparison of genomic laboratory reports and observations that may enhance their clinical utility for providers and patients. Mol Genet Genomic Med 2019 Jul;7(7):e00551. [CrossRef] [Medline]
    6. Knoppers BM, Zawati MH, Sénécal K. Return of genetic testing results in the era of whole-genome sequencing. Nat Rev Genet 2015 Sep;16(9):553-559. [CrossRef] [Medline]
    7. Kearney HM, Thorland EC, Brown KK, Quintero-Rivera F, South ST, Working Group of the American College of Medical Genetics Laboratory Quality Assurance Committee. American College of Medical Genetics standards and guidelines for interpretation and reporting of postnatal constitutional copy number variants. Genet Med 2011 Jul;13(7):680-685. [CrossRef] [Medline]
    8. Richards S, Aziz N, Bale S, Bick D, Das S, Gastier-Foster J, ACMG Laboratory Quality Assurance Committee. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med 2015 May;17(5):405-424 [FREE Full text] [CrossRef] [Medline]
    9. Li MM, Datto M, Duncavage EJ, Kulkarni S, Lindeman NI, Roy S, et al. Standards and guidelines for the interpretation and reporting of sequence variants in cancer: A Joint Consensus Recommendation of the Association for Molecular Pathology, American Society of Clinical Oncology, and College of American Pathologists. J Mol Diagn 2017 Jan;19(1):4-23 [FREE Full text] [CrossRef] [Medline]
    10. Haga SB, Mills R, Pollak KI, Rehder C, Buchanan AH, Lipkus IM, et al. Developing patient-friendly genetic and genomic test reports: formats to promote patient engagement and understanding. Genome Med 2014;6(7):58 [FREE Full text] [CrossRef] [Medline]
    11. Alterovitz G, Heale B, Jones J, Kreda D, Lin F, Liu L, et al. FHIR Genomics: enabling standardization for precision medicine use cases. NPJ Genom Med 2020;5:13. [CrossRef] [Medline]
    12. ISO/TS 20428:2017 Health informatics — Data elements and their metadata for describing structured clinical genomic sequence information in electronic health records. ISO.   URL: https://www.iso.org/standard/67981.html [accessed 2021-04-20]
    13. ISO/PRF TS 22693 Genomics informatics — Structured clinical gene fusion report in electronic health records. ISO.   URL: https://www.iso.org/standard/73694.html [accessed 2021-04-20]
    14. Ryu B, Shin S, Baek R, Kim J, Heo E, Kang I, et al. Clinical genomic sequencing reports in electronic health record systems based on international standards: implementation study. J Med Internet Res 2020 Aug 10;22(8):e15040 [FREE Full text] [CrossRef] [Medline]
    15. ISO/TS 22692:2020 Genomics informatics— Quality control metrics for DNA sequencing. ISO.   URL: https://www.iso.org/standard/73693.html [accessed 2021-04-20]
    16. ISO/DTS 22690 Genomics informatics — Reliability assessment criteria for high-throughput gene-expression data. ISO.   URL: https://www.iso.org/standard/73691.html [accessed 2021-04-20]
    17. Seong D, Chung J, Lee K, Kim S, Kim B, Song J, et al. Benchmark database for process optimization and quality control of clinical cancer panel sequencing. Biotechnol Bioproc E 2019 Oct 25;24(5):793-798. [CrossRef]
    18. Patel NM, Jo H, Eberhard DA, Yin X, Hayward MC, Stein MK, et al. Improved tumor purity metrics in next-generation sequencing for clinical practice: the Integrated Interpretation of Neoplastic Cellularity and Sequencing Results (IINCaSe) approach. Appl Immunohistochem Mol Morphol 2019;27(10):764-772 [FREE Full text] [CrossRef] [Medline]
    19. Clinical Laboratory Improvement Amendments (CLIA). US Centers for Medicare and Medicaid Services (CMS).   URL: https://www.cms.gov/Regulations-and-Guidance/Legislation/CLIA [accessed 2021-04-20]
    20. Considerations for Design, Development, and Analytical Validation of Next Generation Sequencing-Based In Vitro Diagnostics Intended to Aim in the Diagnosis of Suspected Germline Diseases. US Food and Drug Administration (FDA). 2018.   URL: https://www.fda.gov/media/99208/download [accessed 2021-04-20]
    21. Kim J, Park W, Kim NKD, Jang SJ, Chun S, Sung C, Molecular Pathology Study Group of Korean Society of Pathologists. Good Laboratory Standards for clinical next-generation sequencing cancer panel tests. J Pathol Transl Med 2017 May;51(3):191-204. [CrossRef] [Medline]
    22. Lehne M, Sass J, Essenwanger A, Schepers J, Thun S. Why digital medicine depends on interoperability. NPJ Digit Med 2019;2:79. [CrossRef] [Medline]
    23. Mandl KD, Mandel JC, Kohane IS. Driving innovation in health systems through an apps-based information economy. Cell Syst 2015 Jul;1(1):8-13 [FREE Full text] [CrossRef] [Medline]
    24. Mandel JC, Kreda DA, Mandl KD, Kohane IS, Ramoni RB. SMART on FHIR: a standards-based, interoperable apps platform for electronic health records. J Am Med Inform Assoc 2016 Sep;23(5):899-908 [FREE Full text] [CrossRef] [Medline]
    25. Dolin RH, Boxwala A, Shalaby J. A pharmacogenomics clinical decision support service based on FHIR and CDS hooks. Methods Inf Med 2018 Dec;57(S02):e115-e123 [FREE Full text] [CrossRef] [Medline]
    26. Sayeed R, Gottlieb D, Mandl KD. SMART Markers: collecting patient-generated health data as a standardized property of health information technology. NPJ Digit Med 2020;3:9. [CrossRef] [Medline]
    27. Alterovitz G, Warner J, Zhang P, Chen Y, Ullman-Cullere M, Kreda D, et al. SMART on FHIR Genomics: facilitating standardized clinico-genomic apps. J Am Med Inform Assoc 2015 Nov;22(6):1173-1178. [CrossRef] [Medline]
    28. Warner JL, Rioth MJ, Mandl KD, Mandel JC, Kreda DA, Kohane IS, et al. SMART precision cancer medicine: a FHIR-based app to provide genomic information at the point of care. J Am Med Inform Assoc 2016 Jul;23(4):701-710 [FREE Full text] [CrossRef] [Medline]
    29. Warner JL, Prasad I, Bennett M, Arniella M, Beeghly-Fadiel A, Mandl KD, et al. SMART Cancer Navigator: A framework for implementing ASCO Workshop recommendations to enable precision cancer medicine. JCO Precis Oncol 2018;2018:PO.17.00292 [FREE Full text] [CrossRef] [Medline]
    30. Rehm HL, Bale SJ, Bayrak-Toydemir P, Berg JS, Brown KK, Deignan JL, Working Group of the American College of Medical GeneticsGenomics Laboratory Quality Assurance Commitee. ACMG clinical laboratory standards for next-generation sequencing. Genet Med 2013 Sep;15(9):733-747 [FREE Full text] [CrossRef] [Medline]
    31. Aziz N, Zhao Q, Bry L, Driscoll DK, Funke B, Gibson JS, et al. College of American Pathologists' laboratory standards for next-generation sequencing clinical tests. Arch Pathol Lab Med 2015 Apr;139(4):481-493 [FREE Full text] [CrossRef] [Medline]
    32. FHIR Resource MolecularSequence. HL7.org.   URL: https://www.hl7.org/fhir/molecularsequence.html [accessed 2021-04-20]
    33. Quality control metrics for DNA sequencing. SIMPLIFIER.NET.   URL: https://simplifier.net/qualtiycontorlmetricsfordnasequencing [accessed 2021-04-20]
    34. Deployment of NGS-QR app. Heroku.com.   URL: https://ngs-qr.herokuapp.com [accessed 2021-04-20]
    35. NGS Quality Reporting (NGS-QR) App. SMART App Gallery.   URL: https://apps.smarthealthit.org/app/ngs-quality-reporting-ngs-qr-app [accessed 2021-04-20]
    36. NGS Quality Reporting App. GitHub.com.   URL: https://github.com/dhseong/ngs-qr-app-client [accessed 2021-04-20]
    37. Deployment of Shiny app for statistical analysis in NGS-QR app. Shinyapp.io.   URL: https://dhseong.shinyapps.io/NGS-QR-App [accessed 2021-04-20]
    38. R code for statistical analysis in NGS-QR app. GitHub.com.   URL: https://github.com/dhseong/Statistics-for-NGS-QR-app [accessed 2021-04-20]
    39. FHIR Security. HL7.org.   URL: https://www.hl7.org/fhir/security.html [accessed 2021-04-20]
    40. Transport Layer Security (TLS) Protocol.   URL: https://tools.ietf.org/html/rfc5246 [accessed 2021-04-20]
    41. OAuth 2.0 Authorization Framework.   URL: https://tools.ietf.org/html/rfc6749 [accessed 2021-04-20]
    42. Kim ST, Kim K, Kim NKD, Park JO, Ahn S, Yun J, et al. Clinical application of targeted deep sequencing in solid-cancer patients and utility for biomarker-selected clinical trials. Oncologist 2017 Oct;22(10):1169-1177. [CrossRef] [Medline]
    43. Shin H, Choi Y, Yun JW, Kim NKD, Kim S, Jeon HJ, et al. Prevalence and detection of low-allele-fraction variants in clinical cancer samples. Nat Commun 2017 Nov 09;8(1):1377. [CrossRef] [Medline]

    ‎
    API: application programming interface
    EHR: electronic health record
    EMR: electronic medical record
    FDA: Food and Drug Administration
    FFPE: formalin-fixed paraffin-embedded
    FHIR: Fast Healthcare Interoperability Resource
    ISO: International Organization for Standardization
    MFDS: Ministry of Food and Drug Safety
    NGS: next-generation sequencing
    NGS-QR: Next-Generation Sequencing Quality Reporting
    SMC: Samsung Medical Center
    TLS: Transport Layer Security
    UI: user interface

    Edited by G Eysenbach; submitted 04.12.20; peer-reviewed by T Sagi, J Lee, L Guo; comments to author 29.12.20; revised version received 10.01.21; accepted 03.04.21; published 28.04.21

    Copyright

    ©Donghyeong Seong, Sungwon Jung, Sungchul Bae, Jongsuk Chung, Dae-Soon Son, Byoung-Kee Yi. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.04.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.