This paper is in the following e-collection/theme issue:

Digital Health Reviews (1177) Artificial Intelligence (1191) Obstetrics, Gynecology, and Reproductive Medicine (194)

Published on in Vol 24, No 4 (2022): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/35465, first published .
Contributions of Artificial Intelligence Reported in Obstetrics and Gynecology Journals: Systematic Review

Contributions of Artificial Intelligence Reported in Obstetrics and Gynecology Journals: Systematic Review

Contributions of Artificial Intelligence Reported in Obstetrics and Gynecology Journals: Systematic Review

Authors of this article:

Ferdinand Dhombres1, 2 Author Orcid Image ;   Jules Bonnard3 Author Orcid Image ;   Kévin Bailly3 Author Orcid Image ;   Paul Maurice1 Author Orcid Image ;   Aris T Papageorghiou4 Author Orcid Image ;   Jean-Marie Jouannic1, 2 Author Orcid Image
Article Authors Cited by (28) Tweetations (6) Metrics

    Review

    1Fetal Medicine Department, Armand Trousseau University Hospital, Sorbonne University, Paris, France

    2Laboratory in Medical Informatics and Knowledge Engineering in e-Health, Institut National de la Santé et de la Recherche Médicale, Sorbonne University, Paris, France

    3Institute for Intelligent Systems and Robotics, Sorbonne University, Paris, France

    4Oxford Maternal & Perinatal Health Institute, Green Templeton College, Oxford, United Kingdom

    Corresponding Author:

    Ferdinand Dhombres, MD, PhD

    Fetal Medicine Department

    Armand Trousseau University Hospital

    Sorbonne University

    26 Avenue du Dr Arnold Netter

    Paris, 75012

    France

    Phone: 33 4473 5117

    Email: ferdinand.dhombres@inserm.fr


    Background: The applications of artificial intelligence (AI) processes have grown significantly in all medical disciplines during the last decades. Two main types of AI have been applied in medicine: symbolic AI (eg, knowledge base and ontologies) and nonsymbolic AI (eg, machine learning and artificial neural networks). Consequently, AI has also been applied across most obstetrics and gynecology (OB/GYN) domains, including general obstetrics, gynecology surgery, fetal ultrasound, and assisted reproductive medicine, among others.

    Objective: The aim of this study was to provide a systematic review to establish the actual contributions of AI reported in OB/GYN discipline journals.

    Methods: The PubMed database was searched for citations indexed with “artificial intelligence” and at least one of the following medical subject heading (MeSH) terms between January 1, 2000, and April 30, 2020: “obstetrics”; “gynecology”; “reproductive techniques, assisted”; or “pregnancy.” All publications in OB/GYN core disciplines journals were considered. The selection of journals was based on disciplines defined in Web of Science. The publications were excluded if no AI process was used in the study. Review, editorial, and commentary articles were also excluded. The study analysis comprised (1) classification of publications into OB/GYN domains, (2) description of AI methods, (3) description of AI algorithms, (4) description of data sets, (5) description of AI contributions, and (6) description of the validation of the AI process.

    Results: The PubMed search retrieved 579 citations and 66 publications met the selection criteria. All OB/GYN subdomains were covered: obstetrics (41%, 27/66), gynecology (3%, 2/66), assisted reproductive medicine (33%, 22/66), early pregnancy (2%, 1/66), and fetal medicine (21%, 14/66). Both machine learning methods (39/66) and knowledge base methods (25/66) were represented. Machine learning used imaging, numerical, and clinical data sets. Knowledge base methods used mostly omics data sets. The actual contributions of AI were method/algorithm development (53%, 35/66), hypothesis generation (42%, 28/66), or software development (3%, 2/66). Validation was performed on one data set (86%, 57/66) and no external validation was reported. We observed a general rising trend in publications related to AI in OB/GYN over the last two decades. Most of these publications (82%, 54/66) remain out of the scope of the usual OB/GYN journals.

    Conclusions: In OB/GYN discipline journals, mostly preliminary work (eg, proof-of-concept algorithm or method) in AI applied to this discipline is reported and clinical validation remains an unmet prerequisite. Improvement driven by new AI research guidelines is expected. However, these guidelines are covering only a part of AI approaches (nonsymbolic) reported in this review; hence, updates need to be considered.

    J Med Internet Res 2022;24(4):e35465

    doi:10.2196/35465

    Keywords

    artificial intelligence; systematic review; knowledge bases; machine learning; obstetrics; gynaecology; perinatology; medical informatics 


    The foundations of artificial intelligence (AI) as a discipline were established in the 1950s, under the hypothesis formulated by John McCarthy as “Every aspect of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it” [1]. Developing AI was a 3-fold challenge: collect an unprecedented amount of data for training and validation of algorithms, build computers with sufficient computational power, and create algorithms to simulate human intelligence functions (eg, reasoning, learning, adaptation, vision, interaction).

    At the dawn of the 21st century, all 3 challenges have been taken up in many fields, leveraging different types of AI approaches. Two general directions in AI research approaches have been pursued: symbolic approaches and nonsymbolic approaches. The symbolic AI approach, also known as “Good Old-Fashioned AI” (GOFAI) [2], encompasses formal logic, knowledge representation, and rule-based and semantic reasoning. These approaches are generally explainable and human-readable, need human curation and design, and do not rely on a large amount of data to develop. The first GOFAI-related publications in the field of medicine emerged 60 years ago [3], and these approaches provided the first significant results with expert systems [4,5] and are now widely used in knowledge-based systems [6,7], mostly through the application of ontologies and semantic web technologies [8-10]. Nonsymbolic AI, defined as intelligence without specific knowledge representations, encompasses various approaches to simulate other human intelligence processes such as learning, perception, and pattern recognition. Machine learning (ML) has become the main approach in this area [11], mostly through algorithms such as artificial neural networks (ANNs), and relies on a large amount of high-quality data to learn, train, and validate, along with significant computational resources. This AI is generally “nonexplainable,” with the process occurring inside ANNs (architecture, layers, and connections) remaining in the form of a “black box” to the users.

    Publications in AI in medicine have grown rapidly during the last two decades: 119,325 citations are referenced in PubMed, 93% of which have been published since 2000 (Figure 1). The obstetrics and gynecology (OB/GYN) domain represents a wide range of medical activities (obstetrics, fetal medicine, open and endoscopic surgery, ultrasound and other imaging modalities, reproductive biology, and assisted reproductive technologies). These activities are leveraging various types of data (eg, textual data; 2D, 3D, and 4D imaging data; genomic and proteomic data; fetal monitoring data). However, it is only recently that AI concepts (ML principles) were described in an OB/GYN ultrasound imaging journal [12]. Interestingly, the general emergence of AI in the OB/GYN domain, and more specifically in OB/GYN journals, has not been investigated.

    Our aim was to establish the actual contributions of AI reported in OB/GYN journals with a systematic review to investigate, within all OB/GYN subdomains, the AI methods, sources of data, and the contribution and validation of the AI processes. Most of the recent publications about AI usually focus on ML, ANNs, and deep learning. In this review, we considered all AI contributions, including both symbolic and nonsymbolic AI.

    ‎
    Figure 1. Trend of the 119,325 citations in PubMed indexed with the MeSH (Medical Subject Heading) term “artificial intelligence” between 1951 and 2020.
    View this figure

    Design

    This systematic review was performed in accordance with the recommended PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines [13]. The PRISMA Checklist for this study is presented in Multimedia Appendix 1.

    Ethics Approval

    As no patients were involved in the study, this study was exempted from institutional review board approval.

    Literature Search Strategy

    The PubMed database was searched for citations published between January 1, 2000, and April 30, 2020. We used the National Library of Medicine Medical Subject Headings (MeSH) terms to search for citations indexed with “artificial intelligence” and at least one other MeSH term from the OB/GYN domain: “obstetrics”; “gynecology”; “reproductive techniques, assisted”; or “pregnancy.” This search was restricted to English-language publications with an abstract, using the following query: “artificial intelligence” [MeSH Terms] AND (“obstetrics” [MeSH Terms] OR “gynecology” [MeSH Terms] OR “reproductive techniques, assisted” [MeSH Terms] OR “pregnancy” [MeSH Terms]) AND 2000/1/1:2020/4/30 [Date Publication].

    The results of the query were considered final on November 30, 2020, to cover all publications with potential delayed indexation in PubMed.

    All retrieved citations were classified according to disciplines defined in Web of Science (WoS) for the Journal Citation Reports (JCR) and grouped in the following 9 discipline categories: OB/GYN core disciplines journals, other medical clinical disciplines journals, medical nonclinical disciplines journals, medical genetics/biology disciplines journals, medical imaging journals, medical informatics journals, computer science disciplines journals, engineering disciplines journals, and other science disciplines journals. The detailed disciplines and discipline categories are presented in Table S1 of Multimedia Appendix 2 for all journals.

    We included all publications from journals or conference proceedings of the core OB/GYN WoS disciplines, namely Obstetrics & Gynecology, Surgery, Oncology, Developmental Biology, Reproductive Biology, Andrology, or Urology & Nephrology. The publications were excluded if no AI process was used in the study. Review, editorial, and commentary articles were also excluded.

    The publication selection was independently performed by two researchers (FD, JB) by full-text review to assess the actual use of any AI process in the study. Discrepancies on AI process assessments between reviewers were resolved during meetings with KB and JMJ.

    Data Collection and Analysis

    The data collection and the qualitative analysis of the citations comprised six different tasks: (1) classification of publications into 5 OB/GYN domains (obstetrics, gynecology, assisted reproductive medicine, early pregnancy, and fetal medicine), (2) description of the AI method used in the study (eg, ML, knowledge base), (3) description of the AI algorithm used in the study (eg, ANN, support vector machine, bioinformatics knowledge bases), (4) description of the type of data used in the AI process (eg, image data set, omics data set), (5) contribution of the AI process (eg, new algorithm, hypothesis generation, fully functional software), and (6) description of the validation of the AI process (eg, validation on one data set, validation on more than one data set, clinical validation). The statistical synthesis of this systematic review was performed by computing the proportion of publications by groups defined in the qualitative analysis.

    The evolution over time of the scientific production related to AI in OB/GYN was assessed by a trend analysis of publications per year during the entire review period for OB/GYN core journals and other science journals. The respective contributions of all scientific disciplines in the retrieved citations were assessed by the analysis of their distribution across all WoS disciplines and the proportion of citations in each of the 9 science discipline categories.


    Study Selection

    The PubMed search retrieved 579 citations. The 161 publications from OB/GYN core disciplines journals were reviewed for eligibility assessment. A total of 66 publications met the selection criteria [14-79]. The flow chart of the publications reviewed is presented in Figure 2.

    All OB/GYN domains were represented in these selected publications (N=66): obstetrics (n=27, 41%), gynecology (n=2, 3%), assisted reproductive medicine (n=22, 33%), early pregnancy (n=1, 2%), and fetal medicine (n=14, 21%). The detailed distribution of the publications in these domains is presented in Figure 3.

    ‎
    Figure 2. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2020 flow diagram for the selection process of the studies included in this review.
    View this figure
    ‎
    Figure 3. Distribution of the 66 artificial intelligence publications in obstetrics and gynecology journals, across subdomains.
    View this figure

    Description of AI Methods

    The two main AI methods in the 66 selected papers were represented by ML methods [14-47,66-70] (59%, n=39) and by knowledge base methods [50-65,71-79] (38%, n=25). In obstetrics and in fetal medicine, ML was more common than other AI methods in 70% (19/27) and 71% (10/14) of the publications, respectively. ML methods and knowledge bases were used in 45% (10/22) and 55% (12/22) of the publications in in the assisted reproductive medicine domain, respectively.

    The ML methods comprised mainly ANNs (25/39, 64%) [14-25,27-30,33-35,41,43-45,47,68]. Diverse other ML approaches are used, ranging from classical ML tools such as support vector machine [26] and genetic algorithms [31] to more recent methods such as random forest [38,46] or gradient boosted trees [37,42,67]. Very recently, more evolved and combined neural networks are used in deep learning to process complex data for image segmentation (eg, [41,44]) or classification (eg, [39,43]). The knowledge base methods comprised bioinformatic processes involving mainly Gene Ontology (88%, 22/25) but also other omics knowledge bases, text-mining processes leveraging ontologies, and semantic reasoning processes based on domain ontologies.

    The data sets used with all AI methods in the selected studies are detailed in Table 1. ML methods dealt primarily with ultrasound imaging (2D, 3D, video), numerical, and clinical data sets, whereas knowledge base methods dealt mostly with omics data sets.

    The contribution of using AI methods were for algorithm development (53%), hypothesis generation (42%), or software development (3%).

    When using knowledge base methods, the main AI contribution was to generate hypotheses in physiology or physiopathology (reproduction and implantation, preeclampsia, fetal growth, or breast cancer). When using ML methods, the AI contribution was to build prediction algorithms (implantation success, neonatal outcome, preterm delivery, fetal weight, aneuploidy risk, or postpartum complications). The detailed contributions for all AI methods are presented in Table 2.

    Most ML methods were applied to one data set (87%, 34/39) and the use of two or more data sets was less common (13%, 5/39). No external clinical validation of ML methods was identified in the selected articles. Knowledge base methods were applied on one data set in all cases and validated in one clinical study.

    Table 1. Type of data and artificial intelligence methods used in the 66 selected articles.
    Type of dataArticles, n
    Knowledge base method data sets

    		cDNAa/RNA-sequencing16

    		Mixed (clinical and transcriptomic data)3

    		Proteomic/spectrometry2

    		Other: text (publications), imaging (2D ultrasound), mixed (clinical and proteomic data), genomic data repository4
    Machine learning method data sets

    		Clinical (numeric/categorical variables)16

    		Numeric (fetal biometry)4

    		Numeric (fetal heart monitoring/FSpO2b data)4

    		Image (microscopy)3

    		Video (fetoscopy)3

    		Image (2D ultrasound)2

    		Other: administrative (numerical/categorical variables), registry (numerical/categorical variables), numeric (electromyography), numeric (maternal EKGc), mixed (clinical and genomic data), DNA methylation, proteomic/spectrometry7
    Fuzzy logic data sets: numeric (fetal heart monitoring/FSpO2 data)1
    Other data sets, artificial intelligence method not specified (image dataset: 3D ultrasound)1

    acDNA: complementary DNA.

    bFSpO2: fetal oxygen saturation.

    cEKG: electrocardiogram.

    Table 2. Contributions of artificial intelligence methods used in the 66 selected articles.
    Contribution of artificial intelligence methodsArticles, n
    Knowledge base method contributions

    		Hypothesis generation: ARTa techniques/implantation physiology7

    		Hypothesis generation: preeclampsia physiopathology3

    		Hypothesis generation: reproduction physiology3

    		Hypothesis generation: breast cancer physiopathology2

    		Hypothesis generation: fetal growth/development physiology2

    		Method: variant characterization1

    		Method: guided ultrasound image analysis1

    		Other hypothesis generation: pregnancy physiology, diabetes physiopathology, preterm labor physiopathology, recurrent pregnancy loss physiopathology, stem cell profiling, candidate gene/variant6
    Machine learning method contributions

    		Algorithm: implantation/ART method success prediction6

    		Algorithm: neonatal outcome prediction4

    		Algorithm: preterm delivery prediction3

    		Algorithm: delivery route prediction3

    		Algorithm: fetal weight/growth abnormalities prediction3

    		Algorithm: aneuploidy prediction/aneuploidy risk assessment2

    		Algorithm: postpartum complications prediction2

    		Other algorithms: gestational age prediction, preeclampsia prediction, blastocyst grading, classification of lung disorders, muscle image segmentation5

    		Method: fetoscopic images annotation2

    		Other methods: placental blood vessels detection, preterm outcome risk assessment, fertility phenotyping3

    		Hypothesis generation: diabetes physiopathology, fetal alcohol disorder spectrum physiopathology, gastroschisis physiopathology, coagulation physiopathology, uterus physiology5

    		Prototype software: ART success prediction1
    Fuzzy logic method contributions: functional software (3D fetal heart analysis)1
    Other contributions, artificial intelligence method not specified: algorithm (neonatal outcome prediction)1

    aART: assisted reproductive technology.

    General Trend in AI Publications

    We observed a significant rising trend in the scientific production over the last two decades, mainly outside the OB/GYN core journals (Figure 4). A total of 67 science disciplines covered this scientific production (579 PubMed indexed citations), 18% of which were in OB/GYN core disciplines journals. The distribution of citations in the other discipline categories is shown in Table 3. The science discipline was not defined in WoS/JCR for 6% of the citations.

    ‎
    Figure 4. Trends in PubMed artificial intelligence citations between 2000 and 2020 in obstetrics and gynecology (OB/GYN) journals and in other scientific disciplines journals.
    View this figure
    Table 3. Distribution of the 579 PubMed artificial intelligence citations between 2000 and 2020 among the 67 science disciplines.
    Science disciplinesArticles (N=874)a, n (%)
    OB/GYNb core disciplines161 (18.4)
    Medical imaging discipline50 (5.7)
    Other medical clinical discipline47 (5.4)
    Medical nonclinical discipline115 (13.2)
    Medical informatics discipline60 (6.9)
    Medical genetics/biology disciplines58 (6.6)
    Engineering disciplines79 (9.0)
    Computer science disciplines66 (7.6)
    Other science disciplines181 (2.1)
    Absence of discipline in Web of Science57 (6.5)

    aSince some citations are multidisciplinary, the total is higher.

    bOB/GYN: obstetrics and gynecology.


    Main Findings

    In this review, we have demonstrated that AI contributions are emerging in OB/GYN journals and that a wide range of AI approaches (symbolic and nonsymbolic) are applied across all OB/GYN subdomains. ML is the most common nonsymbolic AI approach (59%) and articles are based mainly on ANNs (64%). Knowledge bases are the most common symbolic AI approach (38%) and are based on ontologies in most articles (88%).

    However, most of the AI publications related to AI in OB/GYN (82%) remain out of the scope of the usual OB/GYN journals. Additionally, formally validated AI contributions reported to date suffer from an overall poor level of validation (one data set in most cases and no external validation in all cases) and actual AI contributions remain at the level of “proof of concept” or “proof of feasibility.”

    Publications in OB/GYN Discipline Journals

    The reported AI contribution to OB/GYN in the core discipline journals was 18% in comparison with 82% in journals of other disciplines. This can be explained by the early stage of research in AI or by the absence of clinical validation, meaning that the results are more relevant for the AI and computer science community. When novel algorithms are developed, computer science journals are naturally preferred [80-84]; for example, one of the first convolutional neural networks able to perform automated plane recognition during a fetal ultrasound scan was reported in a computer science journal [85]. In addition a clinically validated ML-based fetal biometric system was reported in a general medical imaging journal, not in a core OB/GYN discipline journal [86]. Another contribution based on logic and semantic reasoning for early pregnancy diagnosis was reported in a medical informatics journal [87]. These examples illustrate that core OB/GYN discipline journals await clinical value demonstration of AI-based research rather than reporting on novel systems. This pattern might also suggest that the time has come for the OB/GYN community to take over some valuable early-stage AI contributions within its core discipline journals.

    Additionally, we have observed more advanced AI techniques and architectures applied to OB/GYN in computer science journals than in OB/GYN discipline journals. Moreover, the well-established and most robust ANN architectures (eg, U-net, ResNet) are no longer published in computer science journals and are largely published in OB/GYN discipline journals to present another application context [39,41,44]. As a result, a strong representation of experts in AI methods in editorial boards could improve editorial choices, which would help to fill in the delay of translation of advanced AI to the OB/GYN readership.

    Interestingly, reported AI methods are applied in unconnected data silos in the field of OB/GYN (images, omics data, clinical data, other data modalities) and mixed AI methods in the field of OB/GYN are in early stages. Thus, approaches involving both ML and knowledge bases is a new direction that we expect to emerge. For example, the Smart Ultrasound in Obstetrics and Gynecology (SUOG) initiative (EIT-Health Innovation program) [88] combines knowledge bases for differential diagnosis and ML for image analysis to develop an AI-based ultrasound diagnosis assistant.

    Quality of AI Research Reporting in the OB/GYN Field

    The low level of validation of AI processes in medicine has been previously reported [89]. We also observed significant heterogeneity in the description of AI processes in this review, with an overall limited level of description in most publications and with a poor level of clinical validation. This can be explained because, until recently, there were no AI-specific guidelines for medical publications. Indeed, most AI notions are new to the medical readership, medical editorial boards, and medical literature indexing. Some medical publications have proposed glossaries and definitions of basic AI notions, and the first reference guidelines for reporting medical studies involving AI were published in 2020 [90-95]. Although these initiatives should improve the reporting of AI-related publications, these guidelines only cover ML approaches. For example, the extension of SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) guidelines for clinical trial protocols using interventions involving AI (SPIRIT-AI) [92] lists the items of interest for AI publications but does not cover knowledge representations, ontologies, semantic reasoning, nor knowledge bases. In addition, we found that 38% of the articles in this review leverage these AI approaches. Consequently, a further extension of these recommendations could advantageously provide guidelines for symbolic AI approaches.

    Albeit not covered in the guidelines for AI-related research, some “routine” methods in statistics (eg, logistic regression, multivariate logistic regression) and in data visualization (eg, K-means clustering) are also considered as ML approaches [96]. In this review, we excluded studies based on these methods [97-99]. However, from a perspective of consistency, some statistical methods involving ML techniques could also be covered by AI-related research guidelines.

    There are recurrent debates on ethical and legal considerations in AI methods in the news and social media; therefore, we were surprised that most publications do not elaborate on these aspects. The majority use nonexplainable approaches such as ANNs; while using such nonexplainable methods is acceptable, some limitations need disclosure, and their reproducibility needs proper assessment. The most straightforward assessment method of reproducibility relies on external validation, which remains critical prior to application of all methods, but even more so if nonexplainable. Human responsibility in using AI-based processes also depends on the level of autonomy of the process [100] and on recommendations to use such processes [101].

    Limitations of MeSH Indexation in PubMed

    This is the first systematic review on AI contributions reported in OB/GYN core journals. This study was performed by a pluridisciplinary group of experts from both the OB/GYN and computer science communities [102]. We have limited our paper selection to citations in PubMed and used the science disciplines as defined by WoS/JCR, thus controlling potential bias in the definition of journal domains. Although our method is reproducible and complies with systematic review guidelines, it is by essence subject to bias in publication indexation. For example, articles with ML methods mentioned only in one paragraph (eg, [103]) are not covered in this study. In addition, for papers with a scope in decision support (eg, [104,105]), the indexation will not fall under the MeSH term “artificial intelligence” in PubMed but rather under the MeSH term “diagnosis, computer-assisted” that is a distinct notion. However, unlike systematic reviews of clinical therapies, this limitation is less of a problem as we were still able to ascertain general trends in this relatively novel field of study.

    All reviewed papers on robotic surgery were indexed in PubMed with the MeSH term “robotics” and under the MeSH term “artificial intelligence.” Currently, in MeSH, “robotics” is a subcategory of “artificial intelligence.” As a result, all robotic surgery papers are considered to be AI papers, which is not always the case. A revision of MeSH terms and/or indexation policies could be a solution for disambiguation. Additionally, the use of appropriate AI-oriented keywords provided by authors at the time of submission could improve the characterization of AI-based research.

    Conclusions

    Until mid-2020, mostly preliminary work in AI applied in OB/GYN has been reported and published outside the usual OB/GYN journals. When published in OB/GYN journals, multiple data set validation and clinical validation of AI processes remain unmet prerequisites. Clarification in AI methods could be achieved by improving the MeSH indexing of publications in PubMed. Additionally, the reporting of AI applications should be improved by the new 2020 guidelines and recommendations in medical research involving AI. These are promising for future clinically relevant and methodologically valid clinical trials publications. However, these guidelines are covering only a part of AI approaches involved in the articles reviewed in this study, and updates need to be considered, especially to cover symbolic AI approaches.

    Acknowledgments

    This work was partially supported by the EIT-HEALTH Innovation (SUOG project, BP2020#20062 and BP2021#211015; principal investigator FD) and by the Sorbonne University’s Institute of Technology for Health (IUIS 2019 Doctoral Program Grant to JB).

    Authors' Contributions

    FD designed the review and drafted the original manuscript. FD, JB, KB, and JMJ reviewed the articles. AP, JMJ, and PM interpreted the data and reviewed/edited the manuscript. All authors have reviewed the submitted version.

    Conflicts of Interest

    FD and JMJ are inventors of an ontology-based imaging protocol, patented by their university (Sorbonne University) and implemented in the SUOG project. ATP is a co-founder and senior scientific advisor for Intelligent Ultrasound. The other authors have no conflicts of interest to declare.

    ‎
    Multimedia Appendix 1

    PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) checklist.

    DOC File , 65 KB
    ‎
    Multimedia Appendix 2

    List of journals covered by the review, by science discipline (source Web of Science) and grouped by discipline categories.

    DOCX File , 41 KB
    1. McCarthy J, Minsky M, Rochester N, Shannon C. A proposal for the Dartmouth Summer Research Project on Artificial Intelligence, August 31, 1955. AI Magazine 2006 Aug 31;27(4):12. [CrossRef]
    2. Haugeland J. Artificial intelligence: the very idea. Cambridge, MA: MIT Press; 1985.
    3. Ledley RS, Lusted LB. Reasoning foundations of medical diagnosis; symbolic logic, probability, and value theory aid our understanding of how physicians reason. Science 1959 Jul 03;130(3366):9-21. [CrossRef] [Medline]
    4. Shortliffe EH, Axline SG, Buchanan BG, Merigan TC, Cohen SN. An artificial intelligence program to advise physicians regarding antimicrobial therapy. Comput Biomed Res 1973 Dec;6(6):544-560. [CrossRef] [Medline]
    5. Shortliffe EH, Davis R, Axline SG, Buchanan BG, Green C, Cohen SN. Computer-based consultations in clinical therapeutics: explanation and rule acquisition capabilities of the MYCIN system. Comput Biomed Res 1975 Aug;8(4):303-320. [CrossRef] [Medline]
    6. Dhombres F, Charlet J, Section Editors for the IMIA Yearbook Section on Knowledge RepresentationManagement. Formal medical knowledge representation supports deep learning algorithms, bioinformatics pipelines, genomics data analysis, and big data processes. Yearb Med Inform 2019 Aug 16;28(1):152-155 [FREE Full text] [CrossRef] [Medline]
    7. Dhombres F, Charlet J, Section Editors for the IMIA Yearbook Section on Knowledge RepresentationManagement. As ontologies reach maturity, artificial intelligence starts being fully efficient: findings from the section on knowledge representation and management for the Yearbook 2018. Yearb Med Inform 2018 Aug 29;27(1):140-145 [FREE Full text] [CrossRef] [Medline]
    8. The Gene Ontology Consortium. The Gene Ontology Resource: 20 years and still GOing strong. Nucleic Acids Res 2019 Jan 08;47(D1):D330-D338 [FREE Full text] [CrossRef] [Medline]
    9. Reference Genome Group of the Gene Ontology Consortium T. The Gene Ontology's Reference Genome Project: a unified framework for functional annotation across species. PLoS Comput Biol 2009 Jul 3;5(7):e1000431 [FREE Full text] [CrossRef] [Medline]
    10. Köhler S, Carmody L, Vasilevsky N, Jacobsen JOB, Danis D, Gourdine J, et al. Expansion of the Human Phenotype Ontology (HPO) knowledge base and resources. Nucleic Acids Res 2019 Jan 08;47(D1):D1018-D1027 [FREE Full text] [CrossRef] [Medline]
    11. Rajkomar A, Dean J, Kohane I. Machine learning in medicine. N Engl J Med 2019 Apr 04;380(14):1347-1358. [CrossRef] [Medline]
    12. Drukker L, Noble JA, Papageorghiou AT. Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology. Ultrasound Obstet Gynecol 2020 Oct;56(4):498-505. [CrossRef] [Medline]
    13. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009 Jul 21;6(7):e1000097 [FREE Full text] [CrossRef] [Medline]
    14. Lockwood CJ, Kuczynski E. Risk stratification and pathological mechanisms in preterm delivery. Paediatr Perinat Epidemiol 2001 Jul;15(Suppl 2):78-89. [CrossRef] [Medline]
    15. Macones GA, Hausman N, Edelstein R, Stamilio DM, Marder SJ. Predicting outcomes of trials of labor in women attempting vaginal birth after cesarean delivery: a comparison of multivariate methods with neural networks. Am J Obstet Gynecol 2001 Feb;184(3):409-413. [CrossRef] [Medline]
    16. Salamalekis E, Thomopoulos P, Giannaris D, Salloum I, Vasios G, Prentza A, et al. Computerised intrapartum diagnosis of fetal hypoxia based on fetal heart rate monitoring and fetal pulse oximetry recordings utilising wavelet analysis and neural networks. BJOG 2002 Oct;109(10):1137-1142. [CrossRef] [Medline]
    17. Maeda K, Noguchi Y, Matsumoto F. Evaluation of prolonged fetal monitoring with normal and pathologic outcome probabilities determined by artificial neural network. Fetal Diagn Ther 2003 Aug 15;18(5):370-375. [CrossRef] [Medline]
    18. Maeda K, Nagasawa T. Automatic computerized diagnosis of fetal sinusoidal heart rate. Fetal Diagn Ther 2005 Aug 19;20(5):328-334. [CrossRef] [Medline]
    19. Wald M, Sparks AE, Sandlow J, Van-Voorhis B, Syrop CH, Niederberger CS. Computational models for prediction of IVF/ICSI outcomes with surgically retrieved spermatozoa. Reprod Biomed Online 2005 Sep;11(3):325-331. [CrossRef] [Medline]
    20. Valensise H, Facchinetti F, Vasapollo B, Giannini F, Monte ID, Arduini D. The computerized fetal heart rate analysis in post-term pregnancy identifies patients at risk for fetal distress in labour. Eur J Obstet Gynecol Reprod Biol 2006 Apr 01;125(2):185-192. [CrossRef] [Medline]
    21. Shi S, Maner WL, Mackay LB, Garfield RE. Identification of term and preterm labor in rats using artificial neural networks on uterine electromyography signals. Am J Obstet Gynecol 2008 Feb;198(2):235 [FREE Full text] [CrossRef] [Medline]
    22. Elaveyini U, Devi SP, Rao KS. Neural networks prediction of preterm delivery with first trimester bleeding. Arch Gynecol Obstet 2011 May 7;283(5):971-979. [CrossRef] [Medline]
    23. Ma Y, Chen B, Wang H, Hu K, Huang Y. Prediction of sperm retrieval in men with non-obstructive azoospermia using artificial neural networks: leptin is a good assistant diagnostic marker. Hum Reprod 2011 Feb 07;26(2):294-298. [CrossRef] [Medline]
    24. Tejera E, Jose Areias M, Rodrigues A, Ramõa A, Manuel Nieto-Villar J, Rebelo I. Artificial neural network for normal, hypertensive, and preeclamptic pregnancy classification using maternal heart rate variability indexes. J Matern Fetal Neonatal Med 2011 Sep 21;24(9):1147-1151. [CrossRef] [Medline]
    25. Cheng Y, Yan G, Chiu YH, Chang F, Chang C, Chung K. Efficient fetal size classification combined with artificial neural network for estimation of fetal weight. Taiwan J Obstet Gynecol 2012 Dec;51(4):545-553 [FREE Full text] [CrossRef] [Medline]
    26. Santos Filho E, Noble JA, Poli M, Griffiths T, Emerson G, Wells D. A method for semi-automatic grading of human blastocyst microscope images. Hum Reprod 2012 Sep;27(9):2641-2648. [CrossRef] [Medline]
    27. Cheng Y, Chiu YH, Wang H, Chang F, Chung K, Chang C, et al. Using Akaike information criterion and minimum mean square error mode in compensating for ultrasonographic errors for estimation of fetal weight by new operators. Taiwan J Obstet Gynecol 2013 Mar;52(1):46-52 [FREE Full text] [CrossRef] [Medline]
    28. Manna C, Nanni L, Lumini A, Pappalardo S. Artificial intelligence techniques for embryo and oocyte classification. Reprod Biomed Online 2013 Jan;26(1):42-49. [CrossRef] [Medline]
    29. Ramasamy R, Padilla WO, Osterberg EC, Srivastava A, Reifsnyder JE, Niederberger C, et al. A comparison of models for predicting sperm retrieval before microdissection testicular sperm extraction in men with nonobstructive azoospermia. J Urol 2013 Feb;189(2):638-642. [CrossRef] [Medline]
    30. Guzmán-Bárcenas J, Hernández JA, Arias-Martínez J, Baptista-González H, Ceballos-Reyes G, Irles C. Estimation of umbilical cord blood leptin and insulin based on anthropometric data by means of artificial neural network approach: identifying key maternal and neonatal factors. BMC Pregnancy Childbirth 2016 Jul 21;16(1):179 [FREE Full text] [CrossRef] [Medline]
    31. Papageorghiou AT, Kemp B, Stones W, Ohuma EO, Kennedy SH, Purwar M, International FetalNewborn Growth Consortium for the 21st Century (INTERGROWTH-21st). Ultrasound-based gestational-age estimation in late pregnancy. Ultrasound Obstet Gynecol 2016 Dec;48(6):719-726. [CrossRef] [Medline]
    32. Veeramani SK, Muthusamy E. Detection of abnormalities in ultrasound lung image using multi-level RVM classification. J Matern Fetal Neonatal Med 2016 Jul 02;29(11):1844-1852. [CrossRef] [Medline]
    33. Bahado-Singh RO, Sonek J, McKenna D, Cool D, Aydas B, Turkoglu O, et al. Artificial intelligence and amniotic fluid multiomics: prediction of perinatal outcome in asymptomatic women with short cervix. Ultrasound Obstet Gynecol 2019 Jul 09;54(1):110-118. [CrossRef] [Medline]
    34. Beksac MS, Tanacan A, Bacak HO, Leblebicioglu K. Computerized prediction system for the route of delivery (vaginal birth versus cesarean section). J Perinat Med 2018 Oct 25;46(8):881-884. [CrossRef] [Medline]
    35. Neocleous AC, Syngelaki A, Nicolaides KH, Schizas CN. Two-stage approach for risk estimation of fetal trisomy 21 and other aneuploidies using computational intelligence systems. Ultrasound Obstet Gynecol 2018 Apr 04;51(4):503-508. [CrossRef] [Medline]
    36. Kuhle S, Maguire B, Zhang H, Hamilton D, Allen AC, Joseph KS, et al. Comparison of logistic regression with machine learning methods for the prediction of fetal growth abnormalities: a retrospective cohort study. BMC Pregnancy Childbirth 2018 Aug 15;18(1):333. [CrossRef] [Medline]
    37. Lussier AA, Morin AM, MacIsaac JL, Salmon J, Weinberg J, Reynolds JN, et al. DNA methylation as a predictor of fetal alcohol spectrum disorder. Clin Epigenetics 2018 Jan 12;10(1):5-14 [FREE Full text] [CrossRef] [Medline]
    38. Blank C, Wildeboer RR, DeCroo I, Tilleman K, Weyers B, de Sutter P, et al. Prediction of implantation after blastocyst transfer in in vitro fertilization: a machine-learning perspective. Fertil Steril 2019 Feb;111(2):318-326. [CrossRef] [Medline]
    39. Vasconcelos F, Brandão P, Vercauteren T, Ourselin S, Deprest J, Peebles D, et al. Towards computer-assisted TTTS: Laser ablation detection for workflow segmentation from fetoscopic video. Int J Comput Assist Radiol Surg 2018 Oct 27;13(10):1661-1670 [FREE Full text] [CrossRef] [Medline]
    40. Liang B, Gao Y, Xu J, Song Y, Xuan L, Shi T, et al. Raman profiling of embryo culture medium to identify aneuploid and euploid embryos. Fertil Steril 2019 Apr;111(4):753-762. [CrossRef] [Medline]
    41. Sadda P, Imamoglu M, Dombrowski M, Papademetris X, Bahtiyar MO, Onofrey J. Deep-learned placental vessel segmentation for intraoperative video enhancement in fetoscopic surgery. Int J Comput Assist Radiol Surg 2019 Feb 27;14(2):227-235 [FREE Full text] [CrossRef] [Medline]
    42. Betts KS, Kisely S, Alati R. Predicting common maternal postpartum complications: leveraging health administrative data and machine learning. BJOG 2019 May;126(6):702-709. [CrossRef] [Medline]
    43. Tran D, Cooke S, Illingworth PJ, Gardner DK. Deep learning as a predictive tool for fetal heart pregnancy following time-lapse incubation and blastocyst transfer. Hum Reprod 2019 Jun 04;34(6):1011-1018 [FREE Full text] [CrossRef] [Medline]
    44. van den Noort F, van der Vaart CH, Grob ATM, van de Waarsenburg MK, Slump CH, van Stralen M. Deep learning enables automatic quantitative assessment of puborectalis muscle and urogenital hiatus in plane of minimal hiatal dimensions. Ultrasound Obstet Gynecol 2019 Aug 26;54(2):270-275. [CrossRef] [Medline]
    45. Vogiatzi P, Pouliakis A, Siristatidis C. An artificial neural network for the prediction of assisted reproduction outcome. J Assist Reprod Genet 2019 Jul 19;36(7):1441-1448 [FREE Full text] [CrossRef] [Medline]
    46. Weber KA, Yang W, Carmichael SL, Padula AM, Shaw GM. A machine learning approach to investigate potential risk factors for gastroschisis in California. Birth Defects Res 2019 Mar 01;111(4):212-221 [FREE Full text] [CrossRef] [Medline]
    47. Bano S, Vasconcelos F, Vander Poorten E, Vercauteren T, Ourselin S, Deprest J, et al. FetNet: a recurrent convolutional network for occlusion identification in fetoscopic videos. Int J Comput Assist Radiol Surg 2020 May 29;15(5):791-801 [FREE Full text] [CrossRef] [Medline]
    48. Czabański R, Jezewski J, Wróbel J, Sikora J, Jezewski M. Application of fuzzy inference systems for classification of fetal heart rate tracings in relation to neonatal outcome. Ginekol Pol 2013 Jan;84(1):38-43. [CrossRef] [Medline]
    49. Yeo L, Romero R. Fetal Intelligent Navigation Echocardiography (FINE): a novel method for rapid, simple, and automatic examination of the fetal heart. Ultrasound Obstet Gynecol 2013 Sep 02;42(3):268-284. [CrossRef] [Medline]
    50. Adona PR, Leal CL, Biase FH, De Bem TH, Mesquita LG, Meirelles FV, et al. In vitro maturation alters gene expression in bovine oocytes. Zygote 2016 Feb 17;24(4):624-633. [CrossRef]
    51. Fang P, Zeng P, Wang Z, Liu M, Xu W, Dai J, et al. Estimated diversity of messenger RNAs in each murine spermatozoa and their potential function during early zygotic development. Biol Reprod 2014 May;90(5):94. [CrossRef] [Medline]
    52. Franczak A, Wojciechowicz B, Kotwica G. Transcriptomic analysis of the porcine endometrium during early pregnancy and the estrous cycle. Reprod Biol 2013 Sep;13(3):229-237. [CrossRef] [Medline]
    53. Forde N, Maillo V, O'Gaora P, Simintiras CA, Sturmey RG, Ealy AD, et al. Sexually dimorphic gene expression in bovine conceptuses at the initiation of implantation. Biol Reprod 2016 Oct 03;95(4):92-92 [FREE Full text] [CrossRef] [Medline]
    54. Santonocito M, Vento M, Guglielmino MR, Battaglia R, Wahlgren J, Ragusa M, et al. Molecular characterization of exosomes and their microRNA cargo in human follicular fluid: bioinformatic analysis reveals that exosomal microRNAs control pathways involved in follicular maturation. Fertil Steril 2014 Dec;102(6):1751-1761. [CrossRef] [Medline]
    55. Peng G, Suo S, Chen J, Chen W, Liu C, Yu F, et al. Spatial transcriptome for the molecular annotation of lineage fates and cell identity in mid-gastrula mouse embryo. Dev Cell 2016 Mar 21;36(6):681-697 [FREE Full text] [CrossRef] [Medline]
    56. Hao Y, Zhang Z, Han D, Cao Y, Zhou P, Wei Z, et al. Gene expression profiling of human blastocysts from in vivo and 'rescue IVM' with or without melatonin treatment. Mol Med Rep 2017 Aug;16(2):1278-1288 [FREE Full text] [CrossRef] [Medline]
    57. Légaré C, Akintayo A, Blondin P, Calvo E, Sullivan R. Impact of male fertility status on the transcriptome of the bovine epididymis. Mol Hum Reprod 2017 Jun 01;23(6):355-369. [CrossRef] [Medline]
    58. Maurice P, Dhombres F, Blondiaux E, Friszer S, Guilbaud L, Lelong N, et al. Towards ontology-based decision support systems for complex ultrasound diagnosis in obstetrics and gynecology. J Gynecol Obstet Hum Reprod 2017 May;46(5):423-429. [CrossRef] [Medline]
    59. Aghaeepour N, Lehallier B, Baca Q, Ganio EA, Wong RJ, Ghaemi MS, et al. A proteomic clock of human pregnancy. Am J Obstet Gynecol 2018 Mar;218(3):347. [CrossRef] [Medline]
    60. Hu X, Ao J, Li X, Zhang H, Wu J, Cheng W. Competing endogenous RNA expression profiling in pre-eclampsia identifies hsa_circ_0036877 as a potential novel blood biomarker for early pre-eclampsia. Clin Epigenetics 2018 Apr 10;10(1):48 [FREE Full text] [CrossRef] [Medline]
    61. Cao C, Wen Y, Wang X, Fang N, Yuan S, Huang X. Testicular piRNA profile comparison between successful and unsuccessful micro-TESE retrieval in NOA patients. J Assist Reprod Genet 2018 May 19;35(5):801-808 [FREE Full text] [CrossRef] [Medline]
    62. Lykoudi A, Kolialexi A, Lambrou GI, Braoudaki M, Siristatidis C, Papaioanou GK, et al. Dysregulated placental microRNAs in early and late onset preeclampsia. Placenta 2018 Jan;61:24-32. [CrossRef] [Medline]
    63. Lu C, Yan Z, Song X, Xu Y, Zheng X, Li R, et al. Effect of exogenous gonadotropin on the transcriptome of human granulosa cells and follicular fluid hormone profiles. Reprod Biol Endocrinol 2019 Jun 24;17(1):49 [FREE Full text] [CrossRef] [Medline]
    64. Ntostis P, Kokkali G, Iles D, Huntriss J, Tzetis M, Picton H, et al. Can trophectoderm RNA analysis predict human blastocyst competency? Syst Biol Reprod Med 2019 Aug 27;65(4):312-325 [FREE Full text] [CrossRef] [Medline]
    65. Santucci-Pereira J, Zeleniuch-Jacquotte A, Afanasyeva Y, Zhong H, Slifker M, Peri S, et al. Genomic signature of parity in the breast of premenopausal women. Breast Cancer Res 2019 Mar 28;21(1):46 [FREE Full text] [CrossRef] [Medline]
    66. Hamilton EF, Dyachenko A, Ciampi A, Maurel K, Warrick PA, Garite TJ. Estimating risk of severe neonatal morbidity in preterm births under 32 weeks of gestation. J Matern Fetal Neonatal Med 2020 Jan 18;33(1):73-80. [CrossRef] [Medline]
    67. Kamphuis C, Duenk P, Veerkamp RF, Visser B, Singh G, Nigsch A, et al. Machine learning to further improve the decision which boar ejaculates to process into artificial insemination doses. Theriogenology 2020 Mar 01;144:112-121 [FREE Full text] [CrossRef] [Medline]
    68. Lipschuetz M, Guedalia J, Rottenstreich A, Novoselsky Persky M, Cohen SM, Kabiri D, et al. Prediction of vaginal birth after cesarean deliveries using machine learning. Am J Obstet Gynecol 2020 Jun;222(6):613. [CrossRef] [Medline]
    69. Venkatesh KK, Strauss RA, Grotegut CA, Heine RP, Chescheir NC, Stringer JSA, et al. Machine learning and statistical models to predict postpartum hemorrhage. Obstet Gynecol 2020 Apr;135(4):935-944. [CrossRef] [Medline]
    70. Miyagi Y, Tada K, Yasuhi I, Maekawa Y, Okura N, Kawakami K, et al. New method for determining fibrinogen and FDP threshold criteria by artificial intelligence in cases of massive hemorrhage during delivery. J Obstet Gynaecol Res 2020 Feb 24;46(2):256-265. [CrossRef] [Medline]
    71. Shi C, Han HJ, Fan LJ, Guan J, Zheng XB, Chen X, et al. Diverse endometrial mRNA signatures during the window of implantation in patients with repeated implantation failure. Hum Fertil 2018 Sep 19;21(3):183-194. [CrossRef] [Medline]
    72. Song Y, Liu J, Huang S, Zhang L. Analysis of differentially expressed genes in placental tissues of preeclampsia patients using microarray combined with the Connectivity Map database. Placenta 2013 Dec;34(12):1190-1195. [CrossRef] [Medline]
    73. Su Y, Zhang Y. Identification of biological processes and genes for gestational diabetes mellitus. Arch Gynecol Obstet 2015 Sep 4;292(3):635-640. [CrossRef] [Medline]
    74. Triche EW, Uzun A, DeWan AT, Kurihara I, Liu J, Occhiogrosso R, et al. Bioinformatic approach to the genetics of preeclampsia. Obstet Gynecol 2014 Jun;123(6):1155-1161 [FREE Full text] [CrossRef] [Medline]
    75. Weng S, Chang S, Cheng Y, Wang H, Wang T, Chang MD, et al. Comparative transcriptome analysis reveals a fetal origin for mesenchymal stem cells and novel fetal surface antigens for noninvasive prenatal diagnosis. Taiwan J Obstet Gynecol 2011 Dec;50(4):447-457 [FREE Full text] [CrossRef] [Medline]
    76. Xue D, Zhang Y, Wang Y, Wang J, An F, Sun X, et al. Quantitative proteomic analysis of sperm in unexplained recurrent pregnancy loss. Reprod Biol Endocrinol 2019 Jul 09;17(1):52 [FREE Full text] [CrossRef] [Medline]
    77. Ye K, Dai J, Liu L, Peng M. Network‑based gene function inference method to predict optimal gene functions associated with fetal growth restriction. Mol Med Rep 2018 Sep 29;18(3):3003-3010. [CrossRef] [Medline]
    78. Yuan W, Heesom K, Phillips R, Chen L, Trinder J, López Bernal A. Low abundance plasma proteins in labour. Reproduction 2012 Oct;144(4):505-518. [CrossRef] [Medline]
    79. Zhou Q, Sun E, Ling L, Liu X, Zhang M, Yin H, et al. Bioinformatic analysis of computational identified differentially expressed genes in tumor stoma of pregnancy‑associated breast cancer. Mol Med Rep 2017 Sep;16(3):3345-3350. [CrossRef] [Medline]
    80. Wang G, Zuluaga MA, Li W, Pratt R, Patel PA, Aertsen M, et al. DeepIGeoS: A deep interactive geodesic framework for medical image segmentation. IEEE Trans Pattern Anal Mach Intell 2019 Jul 1;41(7):1559-1572. [CrossRef]
    81. Meng Q, Housden J, Matthew J, Rueckert D, Schnabel JA, Kainz B, et al. Weakly supervised estimation of shadow confidence maps in fetal ultrasound imaging. IEEE Trans Med Imaging 2019 Dec;38(12):2755-2767. [CrossRef]
    82. Mohseni Salehi SS, Khan S, Erdogmus D, Gholipour A. Real-time deep pose estimation with geodesic loss for image-to-template rigid registration. IEEE Trans Med Imaging 2019 Feb;38(2):470-481. [CrossRef]
    83. Rueda S, Fathima S, Knight CL, Yaqub M, Papageorghiou AT, Rahmatullah B, et al. Evaluation and comparison of current fetal ultrasound image segmentation methods for biometric measurements: a grand challenge. IEEE Trans Med Imaging 2014 Apr;33(4):797-813. [CrossRef]
    84. Rahmatullah B, Papageorghiou A, Noble J. Integration of local and global features for anatomical object detection in ultrasound. Med Image Comput Comput Assist Interv 2012;15(Pt 3):402-409. [CrossRef] [Medline]
    85. Baumgartner CF, Kamnitsas K, Matthew J, Fletcher TP, Smith S, Koch LM, et al. SonoNet: real-time detection and localisation of fetal standard scan planes in freehand ultrasound. IEEE Trans Med Imaging 2017 Nov;36(11):2204-2215. [CrossRef]
    86. Ambroise Grandjean G, Hossu G, Bertholdt C, Noble P, Morel O, Grangé G. Artificial intelligence assistance for fetal head biometry: assessment of automated measurement software. Diagn Interv Imaging 2018 Nov;99(11):709-716 [FREE Full text] [CrossRef] [Medline]
    87. Dhombres F, Maurice P, Guilbaud L, Franchinard L, Dias B, Charlet J, et al. A novel intelligent scan assistant system for early pregnancy diagnosis by ultrasound: clinical decision support system evaluation study. J Med Internet Res 2019 Jul 03;21(7):e14286 [FREE Full text] [CrossRef] [Medline]
    88. Smart Ultrasound in Obstetrics and Gynecology (SUOG) project.   URL: https://www.suog.org/ [accessed 2022-04-13]
    89. Liu X, Faes L, Kale AU, Wagner SK, Fu DJ, Bruynseels A, et al. A comparison of deep learning performance against health-care professionals in detecting diseases from medical imaging: a systematic review and meta-analysis. Lancet Digit Health 2019 Oct;1(6):e271-e297 [FREE Full text] [CrossRef] [Medline]
    90. Collins GS, Moons KGM. Reporting of artificial intelligence prediction models. Lancet 2019 Apr 20;393(10181):1577-1579. [CrossRef] [Medline]
    91. Andaur Navarro CL, Damen JAAG, Takada T, Nijman SWJ, Dhiman P, Ma J, et al. Protocol for a systematic review on the methodological and reporting quality of prediction model studies using machine learning techniques. BMJ Open 2020 Nov 11;10(11):e038832 [FREE Full text] [CrossRef] [Medline]
    92. Cruz Rivera S, Liu X, Chan A, Denniston AK, Calvert MJ, SPIRIT-AICONSORT-AI Working Group, SPIRIT-AICONSORT-AI Steering Group, SPIRIT-AICONSORT-AI Consensus Group. Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension. Nat Med 2020 Sep;26(9):1351-1363 [FREE Full text] [CrossRef] [Medline]
    93. Liu X, Rivera SC, Moher D, Calvert MJ, Denniston AK, SPIRIT-AICONSORT-AI Working Group. Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI Extension. BMJ 2020 Sep 09;370:m3164 [FREE Full text] [CrossRef] [Medline]
    94. Norgeot B, Quer G, Beaulieu-Jones BK, Torkamani A, Dias R, Gianfrancesco M, et al. Minimum information about clinical artificial intelligence modeling: the MI-CLAIM checklist. Nat Med 2020 Sep 09;26(9):1320-1324 [FREE Full text] [CrossRef] [Medline]
    95. Sounderajah V, Ashrafian H, Aggarwal R, De Fauw J, Denniston AK, Greaves F, et al. Developing specific reporting guidelines for diagnostic accuracy studies assessing AI interventions: The STARD-AI Steering Group. Nat Med 2020 Jun 08;26(6):807-808. [CrossRef] [Medline]
    96. Hastie T, Tibshirani R, Friedman J. The elements of statistical learning: data mining, inference, and prediction. New York, NY: Springer; 2009.
    97. Ballester M, Oppenheimer A, d'Argent EM, Touboul C, Antoine J, Coutant C, et al. Nomogram to predict pregnancy rate after ICSI-IVF cycle in patients with endometriosis. Hum Reprod 2012 Feb 23;27(2):451-456. [CrossRef] [Medline]
    98. Bottomley C, Van Belle V, Kirk E, Van Huffel S, Timmerman D, Bourne T. Accurate prediction of pregnancy viability by means of a simple scoring system. Hum Reprod 2013 Jan;28(1):68-76. [CrossRef] [Medline]
    99. Souza RT, Cecatti JG, Passini R, Pacagnella RC, Oliveira PF, Silva CM, Brazilian Multicentre Study on Preterm Birth study group. Cluster analysis identifying clinical phenotypes of preterm birth and related maternal and neonatal outcomes from the Brazilian Multicentre Study on Preterm Birth. Int J Gynaecol Obstet 2019 Jul 21;146(1):110-117. [CrossRef] [Medline]
    100. Jaremko JL, Azar M, Bromwich R, Lum A, Alicia Cheong LH, Gibert M, Canadian Association of Radiologists (CAR) Artificial Intelligence Working Group. Canadian Association of Radiologists White Paper on Ethical and Legal Issues Related to Artificial Intelligence in Radiology. Can Assoc Radiol J 2019 May 01;70(2):107-118 [FREE Full text] [CrossRef] [Medline]
    101. Price WN, Gerke S, Cohen IG. Potential liability for physicians using artificial intelligence. JAMA 2019 Nov 12;322(18):1765-1766. [CrossRef] [Medline]
    102. Littmann M, Selig K, Cohen-Lavi L, Frank Y, Hönigschmid P, Kataka E, et al. Validity of machine learning in biology and medicine increased through collaborations across fields of expertise. Nat Mach Intell 2020 Jan 13;2(1):18-24. [CrossRef]
    103. Jugović D, Tumbri J, Medić M, Jukić MK, Kurjak A, Arbeille P, et al. New Doppler index for prediction of perinatal brain damage in growth-restricted and hypoxic fetuses. Ultrasound Obstet Gynecol 2007 Sep;30(3):303-311. [CrossRef] [Medline]
    104. Huang Y, Chen D, Jiang Y, Kuo S, Wu H, Moon WK. Computer-aided diagnosis using morphological features for classifying breast lesions on ultrasound. Ultrasound Obstet Gynecol 2008 Sep;32(4):565-572. [CrossRef] [Medline]
    105. Huang Y, Kuo S, Chang C, Liu Y, Moon WK, Chen D. Image retrieval with principal component analysis for breast cancer diagnosis on various ultrasonic systems. Ultrasound Obstet Gynecol 2005 Oct 08;26(5):558-566. [CrossRef] [Medline]

    ‎
    AI: artificial intelligence
    ANN: artificial neural network
    GOFAI: Good Old-Fashioned Artificial Intelligence
    JCR: Journal Citation Reports
    MeSH: Medical Subject Heading
    ML: machine learning
    OB/GYN: obstetrics and gynecology
    PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
    SPIRIT-AI: Standard Protocol Items: Recommendations for Interventional Trials involving Artificial Intelligence
    SUOG: Smart Ultrasound in Obstetrics and Gynecology
    WoS: Web of Science

    Edited by A Mavragani; submitted 05.12.21; peer-reviewed by R Rastmanesh, H Ayatollahi; comments to author 18.12.21; revised version received 11.02.22; accepted 15.03.22; published 20.04.22

    Copyright

    ©Ferdinand Dhombres, Jules Bonnard, Kévin Bailly, Paul Maurice, Aris T Papageorghiou, Jean-Marie Jouannic. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 20.04.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.