This paper is in the following e-collection/theme issue:

Chatbots and Conversational Agents (458)

Published on in Vol 21, No 11 (2019): November

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14672, first published .
Conversational Interfaces for Health: Bibliometric Analysis of Grants, Publications, and Patents

Conversational Interfaces for Health: Bibliometric Analysis of Grants, Publications, and Patents

Conversational Interfaces for Health: Bibliometric Analysis of Grants, Publications, and Patents

Authors of this article:

Zhaopeng Xing1 Author Orcid Image ;   Fei Yu1, 2 Author Orcid Image ;   Jian Du3 Author Orcid Image ;   Jennifer S. Walker2 Author Orcid Image ;   Claire B. Paulson1 Author Orcid Image ;   Nandita S. Mani2 Author Orcid Image ;   Lixin Song4, 5 Author Orcid Image
Article Authors Cited by (10) Tweetations (7) Metrics

    Original Paper

    1Carolina Health Informatics Program, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States

    2Health Science Library, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States

    3National Institute of Health Data Science, Peking University, Beijing, China

    4School of Nursing, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States

    5Lineberger Comprehensive Cancer Center, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States

    Corresponding Author:

    Zhaopeng Xing, MPS

    Carolina Health Informatics Program

    University of North Carolina at Chapel Hill

    Health Science Library, 335 S Columbia St

    Chapel Hill, NC

    United States

    Phone: 1 919 260 5844

    Email: zhaopeng@live.unc.edu


    Background: Conversational interfaces (CIs) in different modalities have been developed for health purposes, such as health behavioral intervention, patient self-management, and clinical decision support. Despite growing research evidence supporting CIs’ potential, CI-related research is still in its infancy. There is a lack of systematic investigation that goes beyond publication review and presents the state of the art from perspectives of funding agencies, academia, and industry by incorporating CI-related public funding and patent activities.

    Objective: This study aimed to use data systematically extracted from multiple sources (ie, grant, publication, and patent databases) to investigate the development, research, and fund application of health-related CIs and associated stakeholders (ie, countries, organizations, and collaborators).

    Methods: A multifaceted search query was executed to retrieve records from 9 databases. Bibliometric analysis, social network analysis, and term co-occurrence analysis were conducted on the screened records.

    Results: This review included 42 funded projects, 428 research publications, and 162 patents. The total dollar amount of grants awarded was US $30,297,932, of which US $13,513,473 was awarded by US funding agencies and US $16,784,459 was funded by the Europe Commission. The top 3 funding agencies in the United States were the National Science Foundation, National Institutes of Health, and Agency for Healthcare Research and Quality. Boston Medical Center was awarded the largest combined grant size (US $2,246,437) for 4 projects. The authors of the publications were from 58 countries and 566 organizations; the top 3 most productive organizations were Northeastern University (United States), Universiti Teknologi MARA (Malaysia), and the French National Center for Scientific Research (CNRS; France). US researchers produced 114 publications. Although 82.0% (464/566) of the organizations engaged in interorganizational collaboration, 2 organizational research-collaboration clusters were observed with Northeastern University and CNRS as the central nodes. About 112 organizations from the United States and China filed 87.7% patents. IBM filed most patents (N=17). Only 5 patents were co-owned by different organizations, and there was no across-country collaboration on patenting activity. The terms patient, child, elderly, and robot were frequently discussed in the 3 record types. The terms related to mental and chronic issues were discussed mainly in grants and publications. The terms regarding multimodal interactions were widely mentioned as users’ communication modes with CIs in the identified records.

    Conclusions: Our findings provided an overview of the countries, organizations, and topic terms in funded projects, as well as the authorship, collaboration, content, and related information of research publications and patents. There is a lack of broad cross-sector partnerships among grant agencies, academia, and industry, particularly in the United States. Our results suggest a need to improve collaboration among public and private sectors and health care organizations in research and patent activities.

    J Med Internet Res 2019;21(11):e14672

    doi:10.2196/14672

    Keywords

    conversational interfaces; conversational agents; chatbots; artifical intelligence; healthcare; bibliometrics; social network; grants; publications; patents 


    The emergence of conversational interfaces (CIs) enables users to talk to a machine [1]. Using conventional pattern match or natural language processing, CIs simulate human conversation through various interaction modalities [2-4]. For example, text-based CIs, also known as chatbots, are commonly presented on messaging platforms where users can converse with bots using textual input (eg, Facebook Messenger and Slack). Voice-based CIs incorporate a speech channel into the interface and are preferred over traditional graphical interfaces (eg, keyboard and screen) in visual or hands-off tasks [5]. Recent examples of voice-based CIs include intelligent personal assistants (eg, Apple Siri, Amazon Alexa, and Microsoft Cortana) and CIs with multimodalities such as embodied conversational agents (ECAs). Multimodality can be designed to engage verbal and nonverbal interactions (eg, gaze movement, facial expression, and gesture) between the CI system and users [6,7]. This mode can also be implemented using an embodied character and menu-based dialogue module to simulate a dialogue flow [8].

    In health care, CIs have been adopted to provide complementary therapy and health behavioral interventions [9-11], assist patient self-management [12-15], and support clinical decision making [16-18]. With the growth of CIs for health, there have been increasing efforts in appraising research in CI design and applications. Laranjo et al [2] identified 17 studies that have investigated 14 unique CI applications for general health-related purposes. These studies reported that CIs have produced positive outcomes related to patient engagement and adherence and decreased self-reported symptoms. A review examined 40 studies regarding CI technologies in hospital settings and proposed a taxonomy that involved interaction context, dialogue types, and architecture attributes. Researchers reported that CIs were primarily designed for physician education and patient counseling purposes [3]. A review of 8 studies that applied CIs to treat mental illness [4] reported positive outcomes and user satisfaction yet inconsistent evaluation of CI technologies.

    Despite the rapidly growing research evidence supporting the potentials of CI applications in health care, there is a lack of systematic investigation that goes beyond the published research and incorporates grant and patent activities. This will demonstrate the state of the art from the perspectives of funding agencies, academia, and industry. Research grants reflect funding agencies’ attention to and support for emerging research domains [19,20]. Publications and patents have been widely used to map the emergence of technologies and reveal research and development (R&D) activities [21,22]. A broad scope review using data from multiple sources, such as scientific publications, funding instruments, and patents, will thus provide an overview of the domain landscape and inform stakeholders [23,24] for better CI utilization and research. In addition, a recent report on artificial intelligence (AI) by World Intelligence Property Organization (WIPO) [25] identified the untapped opportunities that AI technologies have brought to health care and called for a broad collaboration and/or coordination among funding agencies, policymakers, researchers, and entrepreneurs. As one of the most promising health care applications of AI, CIs should be given sufficient attention. This study aimed to use data systematically extracted from multiple sources (ie, grant, publication, and patent databases) to investigate the development and research of health CIs. We examined the following 5 research questions:

    • RQ1: Basic statistics of the collected records: how many grants, publications, and patents exist for CIs used for health purposes?
    • RQ2: Analysis of grants: which funding agencies have granted the largest amount of funds in CIs and which organizations have received funds in CIs?
    • RQ3: Analysis of research publications: who are the top contributors to research publications, who are their main collaborators, and how are they distributed geographically?
    • RQ4: Analysis of patents: where were the patents filed (ie, country) and who were the most active patent assignees (ie, organizations)?
    • RQ5: Analysis of topic terms: what terms were frequently addressed in grants, publications, and patents and what potential gaps can be identified for future research?

    Data Collection

    To identify grants, publications, and patents, we created a search strategy adapted from previously published works that include CI-related terms [1,2,21,26], such as spoken dialogue system, conversational agent, chatbot, social robot, virtual agent, and question-answer system. The health-related terms covered both generic health term variations and key stakeholders, such as health, healthcare, medicine, clinic, physician, patient, and caregiver. In addition, we used a snowball process to collect synonyms, term variations, and relevant terms to build a search term thesaurus for this study [27]. Search queries were modified for different databases (see Multimedia Appendix 1).

    We systematically searched 9 databases to identify granted projects, research publications, and patents from January 2008 to December 2018 that were for health purposes (Table 1). Initial searches were conducted in August 2018 and updated in January 2019. The databases for grant search included Federal RePORTER and Community Research and Development Information Service (CORDIS). Federal RePORTER is a publicly available database of scientific funding projects by federal agencies, including National Institutes of Health (NIH), National Science Foundation (NSF), the Agency for Healthcare Research and Quality (AHRQ), Department of Agriculture, National Aeronautics and Space Administration, Department of Defense, Environmental Protection Agency, and Department of Education. For a broader coverage of funding records, we also searched the Europe Commission (EC) by using CORDIS, a primary information source of funded projects in the European Union countries.

    The publication search was conducted in 6 bibliographic databases that index literature in technology, biomedical, and health sciences (Table 1). Publications were limited to conference papers, journal articles, and book sections and excluded meeting abstracts, editorials, letters, or lecture notes. We retrieved the filed and granted patents from Derwent Innovation Index (DII). Sponsored by Clarivate Analytics, DII is a widely used database in patent bibliometric analysis and has a broad coverage of over 37 million inventions and 70 million patents from 52 authorities in the world [24,28].

    After searches were completed, the retrieved records from the 9 selected databases were aggregated and deduped in EndNote (Clarivate Analytics). Particularly, if a record had a duplication, we only kept the one which had complete data fields. We also cross-checked other data resources for the missing parts of the records.

    Table 1. Databases of grants, publications, and patents.
    Data typeDatabase searched
    GrantsFederal RePORTER and Community Research and Development Information Service
    PublicationsWeb of Science, PubMed, Scopus, EMBASE, Association for Computing Machinery Digital Library, and Cumulative Index of Nursing and Allied Health
    PatentsDerwent Innovation Index

    Data Screening

    We included grants, scientific publications, and patents that were relevant to the design, development, improvement, deployment, or evaluation of CIs for health purposes. A record was excluded if (1) it was awarded, published, or filed before 2008; (2) on cross-check, it missed any of the following fields: organization recipients of a grant, grant agency, grant size, or budget year; title, abstract, or author affiliation; and country, patent assignee, or priority date; and (3) it was irrelevant to CI technologies for health purposes. For example, we included the records that reported systems, programs, and interfaces, which enabled users to issue command requests or engage in dialogue/chats in different input modalities (eg, text based, spoken, menu selection, or multimodal). We excluded the records that did not involve or specify the abovementioned modes, such as pet robot, nonverbal virtual robots, and the Internet of Things. We also excluded the records that examined CIs for nonhealth purposes. For patents, we included the records that claimed CI inventions that can be used for health purposes. After deduping and cleaning the data, ZX manually screened the titles and abstracts of the records, and the records with uncertainty were discussed at team meetings and resolved by co-authors (Figure 1).

    ‎
    Figure 1. Data collection and screening.
    View this figure

    Data Analysis

    We employed bibliometric analysis, social network analysis (SNA), and term co-occurrence analysis techniques to analyze the abstracted data (Table 2). Bibliometric analysis is the quantitative analysis of scientific publications [29]. It has been widely used for measuring research and patenting performance [21,30] and recently extended to measuring research funding dynamics [23,24]. SNA was originally developed for social structure study. Using bibliographic data, SNA expands the scope of bibliometrics by revealing the co-authorship among different research units [31,32]. Built on network theory and bibliometrics, the term co-occurrence analysis represents the knowledge components of a document by key terms and their co-occurrence relationship [33]. Researchers have used the term co-occurrence approach to explore potential research topics in scientific publications [34,35].

    Table 2. Extracted data fields, analysis methods, and metrics.
    Data sampleData fieldsAnalysis methodsMetrics
    GrantTitle, abstract, grant agency, granted organization, grant size, and grant start yearBibliometric analysis, social network analysis, and term co-occurrence analysisNumber of funded projects, grant size, project counts, project duration, funding and recipient agency and organizations, and topic co-occurrence
    PublicationTitle, abstract, affiliation, and publication yearBibliometric analysis, social network analysis, and term co-occurrence analysisNumber of publications, organizational network, and topic co-occurrence
    PatentTitle, abstract, patent assignee, and the priority dateaBibliometric analysis, social network analysis, and term co-occurrence analysisNumber of patents, organizational collaboration network, and topic co-occurrence

    aWe used the priority date as the time stamp for patent data, which is the date when the first patent application in a patent family was filed at a patent office (priority application) and which is often used to establish the priority of an invention [36].

    We used both VOSviewer (version 1.6.9) and Tableau for data analysis and visualization. VOSviewer is a bibliometric network analysis software to conduct SNA for collaboration networks and term co-occurrence analysis for topic analysis [37,38]. For SNA, each organization was represented as a node, and each collaboration between 2 nodes was represented by a link. For the term co-occurrence analysis, key terms were extracted from the title and abstract of the records. We used a density map to visualize the disclosed key terms. The importance of a topic was measured by frequency of terms (ie, counts of a term) and its co-occurrence with other terms. The synonyms of extracted key terms were consolidated for visualization. For example, the extracted terms child, children, and kid were merged and standardized as child.


    The Number of Grants, Research Publications, and Patents

    After data screening, we obtained a dataset with 42 grants, 428 research publications, and 162 patents related to CIs for health purposes. As displayed in Figure 2, the number of grants has remained steady from 2008 to 2018, with an average of 3.8 projects per year. The number of publications increased since 2010, with an average growth rate of 22.1% per year, which then reached a peak in 2015. The number of patents also grew from 2010 to 2017, with an annual growth rate of 22%. The US patent applications filed on and after 2018 are not publicly available until 18 months after the application's earliest filing date [36], causing the drop-in number of patent records in 2018.

    ‎
    Figure 2. Number of grants, research publications, and patents between 2008 and 2018.
    View this figure

    Analysis of Grants

    Among the 42 funded research projects focusing on CI technologies for health purposes, the total dollar amount of grants awarded was US $30,297,932, of which US $13,513,473 was awarded by funding agencies in the United States, and US $16,784,459 was awarded by the EC (Table 3 and Figure 3). The top 3 funding agencies in the United States were NSF, NIH, and AHRQ, which overall funded 45.2% of the included projects. Among the 6 EC-funded projects identified in this study, the EC grants were allocated to 10 countries, that is, the United Kingdom, Spain, France, Italy, Germany, Norway, Belgium, Israel, Latvia, and Romania (Table 4), for either individual or collaborative projects.

    Table 3. Grant count, size, and average duration of projects.
    AgencyProject countPercentage of total countGrant sizePercentage of total grantsMonths of project, mean (SD)
    National Science Foundation (the United States)1945.2US $8,686,66928.6733.9 (15.9)
    National Institutes of Health (the United States)1638.1US $4,645,61215.3334.9 (17.4)
    Agency for Healthcare Research and Quality (the United States)12.4US $181,1920.6024.0 (N/Aa)
    Europe Commission (European Union)614.3US $16,784,45955.4033.0 (6.7)

    aNot applicable.

    ‎
    Figure 3. Grant size (USD) and project duration (months) from 2008 to 2018. AHRQ: Agency for Healthcare Research and Quality; EC: Europe Commission; NIH: National Institutes of Health; NSF: National Science Foundation.
    View this figure
    Table 4. The grant size and number of grant recipients by country.
    CountryOrganization recipients countTotal grant size
    United States27US $13,513,473
    United Kingdom9US $3,914,721
    Spain8US $3,048,271
    France5US $2,350,823
    Italy5US $2,567,397
    Germany4US $2,120,652
    Norway2US $1,392,323
    Belgium1US $330,315
    Israel1US $218,738
    Latvia1US $639,540
    Romania1 US $201,680

    The 42 funded projects involve a total of 64 granted organizations (Figure 4). Among the 27 US grantee organizations, 26 were research institutes. Boston Medical Center was awarded the largest grant size with 3 projects funded by NIH with a total amount of US $2,065,245 and 1 project funded by AHRQ with the amount of US $181,192. These 4 projects were granted for designing and evaluating CIs to improve patients’ engagement [39], for palliative care of patients with advanced illness, for reducing cardiopulmonary rehospitalization [40], and for treating comorbid depression (no outcomes reported). In Europe, the EC funded 18 private companies and 19 academic institutes. The University of Edinburg received the largest grant for a project that applied CI technology to support the treatment of major depression (ie, US $1,758,786) [41].

    The project duration varied from 3 to 69 months. For example, Auburn University and the University of South Florida were granted 3-month funding from NSF and NIH, respectively. The University of Delaware received funding that lasted for 69 months.

    ‎
    Figure 4. Grant agencies and granted organizations network. AHRQ: Agency for Healthcare Research and Quality; EC: Europe Commission; NIH: National Institutes of Health; NSF: National Science Foundation.
    View this figure

    Analysis of Research Publications

    This study identified a total of 428 publications authored by researchers from 58 countries. As shown in Table 5, the researchers from the United States published 114 manuscripts on CI technologies for health purposes. The other countries that have published the most about this line of work included Japan (npublication=34), France (npublication =33), China (npublication=28), the United Kingdom (npublication=24), Italy (npublication=21), Malaysia (npublication=20), Germany (npublication=17), and the Netherlands (npublication=16). Overall, researchers in European countries produced about 31.1% of the total publications compared with 19.2% by researchers in Asian countries.

    Among the 566 organizations with which the authors were affiliated, 95 organizations (16.8%) contributed at least two publications, including 85 academic institutions, 6 hospitals, and 4 companies or private organizations. As shown in Table 5, the top 3 most productive institutes were Northeastern University, the Universiti Teknologi MARA System in Malaysia, and the French National Center for Scientific Research (also known as and shown as CNRS).

    Regarding organizational collaboration, 246 of 428 publications (57.5%) were co-authored by researchers from at least two organizations. Overall, 463 organizations participated in interorganizational collaborations, whereas 303 organizations had more than 1 external collaborator. The top 5 institutes were CNRS (20 collaborators), Inserm (14 collaborators), Osaka University (13 collaborators), Northeastern University (11 collaborators), and the University of Naples Federico II (11 collaborators; Figure 5). A total of 2 collaboration clusters were identified from the organizational collaboration network with Northeastern University and CNRS as the central nodes.

    Table 5. The number of publications by country and organization (Top 10), N=248.
    RankPublications by countryPublication by organization
    CountryPublication count, n (%)OrganizationPublication count, n (%)
    1United States114 (26.6)Northeastern University (US)15 (3.5)
    2Japan34 (7.9)University Teknologi MARA (Malaysia)13 (3.0)
    3France33 (7.7)CNRS (France)12 (2.8)
    4China28 (6.5)University of South California (US)9 (2.1)
    5United Kingdom24 (5.1)University of Gothenburg (Sweden)8 (1.9)
    6Italy21 (4.9)Delft University of Technology (Netherlands)6 (1.4)
    7Spain21 (4.9)Istanbul Technical University (Turkey)6 (1.4)
    8Malaysia20 (4.7)University Carlos III de Madrid (Spain)5 (1.2)
    9Germany17 (4.0)Universiti Teknikal Malaysia Melaka (Malaysia)5 (1.2)
    10Nether-lands16 (3.7)German Research Center for Artificial Intelligence (Germany)5 (1.2)
    ‎
    Figure 5. Publication organizational co-authorship network at country level. Each node represents an organization, and the size of the node represents publication count. CNRS: National Center for Scientific Research.
    View this figure

    In addition to academic institutions, 5 hospitals/medical centers (ie, Chiba Rosal Hospital, Mahak Hospital, Boston Medical Center, Spectrios Institute for Low Vision, and National Rehabilitation Center for Persons with Disabilities in Japan), and 3 nonacademic institutions (ie, TNO, IBM, and Chicago Lighthouse) were involved in the collaboration network. The only collaboration across 3 types of organizations was among Loyola University, Chicago Lighthouse (a nonprofit social service organization), and the Spectrios Institute for Low Vision (Figure 6).

    ‎
    Figure 6. Publication co-authorship network at the organizational level. The organizations shown are those with more than 1 publication.
    View this figure

    Analysis of Patents

    Among the 162 patents reviewed in this study, the United States and China filed the majority of the patent inventions, 78 and 69 respectively, and they are followed by Japan (npatent =7) and South Korea (npatent =6). Regarding the patent assignees, 112 organizations were identified, including 98 private companies and 14 research institutes. Among research institutes, 9 were based in China, 2 in Japan, 2 in the United States, and 1 in South Korea. IBM filed more patents (npatent =17) than all the other organizations or companies. Organizations that filed more than 2 patents were Vocollect Healthcare (npatent =6), Google (npatent =4), Next It Corp (npatent =3) and Samsung (npatent =3).

    With regard to the collaboration among patent assignees, we only identified 5 collaboration patents that were co-assigned to 5 pairs of organizations in the United States, Japan, and China. They included (1) Puretech Management Inc and Bose Corp (United States), (2) Next It Corp and Verint Americas Inc (United States), (3) University of Kyoto and Toyota (Japan), (4) Jiangsu province Hospital and Nanjing Zhongyue Information Technology Company (China), and (5) University of South China Technology and Guanzhou Lvsong Biotechnology Company (China). Furthermore, this study did not find any cross-country patent collaborations among identified assignees.

    Analysis of Topic Terms

    Grants

    The most commonly occurring terms (Figure 7) included patient, child, intervention, robot, and speech. The grants primarily focused on the patient, and 7 projects targeted the child or elderly populations. In addition, the term asd (ie, autism spectrum disorder [ASD]) was in very close proximity with child and occurred in 5 funded projects. For example, an NIH-sponsored project at the University of Connecticut and the University of Delaware in 2009 used social robots to support children with ASD. The term elderly was also addressed in 7 grants. For example, Auburn University and Clemson University proposed a project, funded by NSF in 2009, aimed to use CI to improve older adults’ quality of life. Furthermore, speech seems to be a preferred interaction modality in the proposals. For example, the University of Texas Dallas was funded by NSF to create a speech-enabled CI, which aimed to help individuals with hearing impairment and autistic children improve communication skills.

    ‎
    Figure 7. Heatmap of the topic terms that occurred in more than 4 granted projects.
    View this figure
    Publications

    The terms with top occurrence in publications were robot, patient, and child (Figure 8). In addition, asd appeared together with child in 46 publications [42,43]. Other disease-related terms discussed in the publications included aphasia (3.7%, 16/428) [44,45], cerebral palsy (2.8%, 12/428) [46,47], stroke (2.6%, 11/428) [48,49], cancer (1.4%, 6/428) [50,51]. The key term elderly was addressed in 42 publications [52,53]. The term multimodal occurred in 39 publications [54,55], and specific terms associated with multimodal interaction were gesture (10.3%, 44/428), voice (10.0%, 43/428), and facial expression (5.1%, 22/428). Furthermore, NAO robot, a humanoid robot developed by SoftBank Robotics, was investigated in 87 publications (20.3%) [42,43].

    ‎
    Figure 8. Heatmap of the topic terms that occurred in more than 9 publications.
    View this figure
    Patents

    The terms with top occurrence in patents (Figure 9) included patient, voice, and electronic device. Other key terms reflected the system components of CIs [56,57], such as speaker (18.5%, 30/162), microphone (15.4%, 25/162), and communication module (6.2%, 10/162). In addition, the terms robot (npatent=34, 21.0%, 34/162), sensor (14.8%, 24/162), and voice interaction (42.6%, 69/162) were closely positioned and co-occurred frequently [58,59]. Another frequently occurred term was question (16%, 26/162), which also appeared closely with question answering systems, (9.3%, 15/162) [60,61]. The stakeholders addressed in the patents included not only patient but also physician (3.1%, 5/162) and elderly (2.5%, 4/162) [62,63].

    ‎
    Figure 9. Heatmap of the topic terms that occurred in more than 7 patents.
    View this figure

    Principal Findings

    This is the first study to systematically examine emerging CI technologies for health purposes using grants, research publications, and patent data. We found increasing efforts in recent years in exploring health-related applications of CI.

    There has been an increase in government funding support in this field in the United States and European countries since 2008. The total dollar amount of EC grants was higher than that of US grants. Research institutes and private organizations have been involved and awarded almost equally in European countries, whereas research institutions were the only type of awardee in the United States. These results may suggest that funding agencies in the United States need to encourage more collaboration between academia and industry.

    The research publications are mainly from a few countries and institutions, and there is a lack of international collaborations across countries. US researchers have been leading in the number of publications about CIs for health. Northeastern University was the leading institute that has focused on the design and evaluation of CIs for health education and behavioral intervention in clinical settings, such as information access to clinical trials for cancer [39], depression therapy [64], or spinal cord injury recovery [65]. This university has established a wide research collaboration network, but its collaborators are mainly domestic institutions. Researchers in European countries produced about one-third of the included publications. CNRS, a prominent French research institute, has applied CIs to tackle health issues such as substance use disorder [66] and depression [67]. The collaboration in European countries is centered on France, and collaboration with US institutions is rarely observed, suggesting that CI research is still geographically isolated, and more cross-country collaboration could be encouraged. The Universiti Teknologi MARA system in Malaysia was the major contributor to research publications in Asia. Their research has focused on adopting the robot NAO in interventions and rehabilitation for patients with mental or brain disorders (eg, ASD [68] and cerebral palsy [69]). Osaka University in Japan had the most research collaborators compared with other Asian institutes. The institutions in Malaysia and China, on the other hand, were not actively involved in CI research collaboration. In addition to the lack of collaboration between countries, there is also a lack of collaboration between researchers and health care professionals. Several hospitals and medical centers engaged in interorganizational collaborations, but the number of such collaborations is still small. This may explain the existing challenge of insufficient clinical evaluation of CIs, inconsistent results, and high concern for patient safety, which were disclosed by previous studies [2,70,71]. In addition, the function of CIs in patient care requires substantial domain knowledge and an adequate understanding of human emotion [72]. Given these facts, the involvement of health care providers is important in design and clinical evaluation for better user-centered CI functions and improved patient outcomes.

    The United States and China were the 2 most active countries in patenting activities, but the major players were different. In the United States, private companies (eg, IBM, Vocollect Healthcare, and Google) have dominated patent applications, whereas in China, academic institutions are most actively engaged in patent applications. This finding is consistent with results from the WIPO’s report on AI [25], which suggested that China has paid increasing attention to research-innovation translation. In addition, co-owned patents among university and companies were rarely identified in this study, which may suggest that academia-industry collaboration in patent activities related to CIs for health is currently uncommon, and further efforts are required to bridge this gap.

    This study also presented key topic terms regarding the targeted users, health issues, and interaction modality. Overall, both patient and robot were addressed by all 3 types of records. Child and elderly appear to be 2 major user-related terms frequently discussed. For example, Northeastern University investigated computer-based ECA, which served as a humanoid assistant for patients with low literacy [73,74]. IBM's inventions, which primarily concerned clinical QA systems, helped physicians and other medical professionals to search medication evidence and supported their clinical decisions [60,61].

    In terms of targeted health issues, this study found CIs were mainly used for addressing mental health and chronic diseases. The asd is the most frequently investigated term in grant applications and publications. It often occurred along with children and robot. This finding is consistent with the results from previous studies [2,75,76] that robots have been widely used to treat children with ASD by promoting their social behaviors and improving their communication skills. Our results also revealed that CIs were adopted to manage chronic diseases, such as stroke [46], cancer [47], and dementia [77]. These health issues usually require long-term treatment and intensive care. CIs, as a supplement to health care providers, can be used to promote communication and provide a support companion for patients with ASD [42,43] and to deliver interventions and management for patients with chronic diseases [48-51].

    Regarding the human-CI interaction modality, we found that the CI interface was favorably presented as robot and that multimodality was widely discussed in publications and patents. For example, NAO robots interacted with users in both verbal and nonverbal modes and provided face-to-face communication and physical touch. More than 80 publications reported findings about the applications of this humanoid robot among patients with various health issues, including improving communication skills for children with ASD (eg, [42,43]), reducing distress for cancer patients (eg, [50]), and providing functional and emotional support for the elderly (eg, [52,78]). NAO’s hybrid communication channels (eg, speech, eye contact, and body movement) contributed to users’ in-depth engagement to achieve desired behavior change. This may suggest that the multimodal robot has become one of the main forms of CIs for health purposes. However, it does not mean that this is the only or best way to design or implement CIs for health. Instead, different approaches in presenting CIs for health can potentially benefit health care resolutions in different clinical scenarios [2,3]. Overall, empirical evidence has confirmed the positive effects of CIs’ multimodal interface on medication adherence, social activity, and elderly patients’ learning processes [3]. Nevertheless, there is a lack of consistency and evidence of long-term clinical effects of the CIs [75]. Therefore, future research should deepen the understanding of user interaction with multimodal CI agents and the corresponding impacts on patient outcomes.

    Limitations

    The study has several limitations. First, the selection of databases in this study introduced bias because we included only records released in English. Therefore, the results did not reflect any grants, research, and patenting activities reported in other languages. Second, during the data screening process, although the uncertain cases were reviewed and discussed by co-authors, only the first author screened the entire retrieved records, which may have led to bias in records inclusion and exclusion and consequently affected the results. Third, privately funded projects were not included in the analysis of grants data because of lack of access to private funding databases. Finally, the 3 types of data (ie, grants, publications, and patents) were retrieved and analyzed independently. There were no direct data mapping and integration across these data sources. Thus, results in this study should not be interpreted with respect to the relationship between public research investment, research outcomes, and patenting activities.

    Conclusions

    This study systematically analyzed CI technologies for health purposes using data extracted from multiple information sources. Our findings provided an overview of the countries, organizations, and topic terms in funding activities as well as the authorship, collaboration, contents, and related information of research publications and patents. Overall, the inclusion of grants and patents in addition to publications has presented complementary insights into the R&D landscape of the use of CIs for health purposes. Our results have shown that there is a lack of cross-sector collaboration among grantees in the United States. In addition, international collaboration in research should be encouraged among the institutes to sustain the growth of CI application in health. The academic-industrial collaboration should also be fostered in patenting activities. Although current CIs have focused extensively on mental health problems and chronic diseases, future research needs to extend CI technologies to more diverse health issues via safer and more engaging multimodal interfaces.

    Acknowledgments

    This work was supported by Health Science Library at the University of North Carolina at Chapel Hill, by R01NR016990 National Institute of Nursing Research (principal investigator: LS), by R21CA212516 National Cancer Institute (principal investigator: LS) and by University Cancer Research Fund of the Lineberger Comprehensive Cancer Center at the UNC-CH (principal investigator: LS).

    Authors' Contributions

    ZX developed the study design with inputs from FY, JD, and LS. ZX, FY, and JSW collected records and formulated the query with assistance from Jesse Akman and Lee Richardson in the Health Science Library at the University of North Carolina at Chapel Hill. ZX screened the title and abstract and resolved cases with uncertainty with FY and LS. ZX and FY prepared the initial draft of the manuscript. All authors provided contributions to the final version of the paper and approved it.

    Conflicts of Interest

    LS is an editorial board member for JMIR Aging.

    ‎
    Multimedia Appendix 1

    Search query.

    DOCX File , 20 KB
    1. McTear M, Callejas Z, Griol D. Conversational interfaces: past and present. In: The Conversational Interface. Cham: Springer International Publishing; 2016:51-72.
    2. Laranjo L, Dunn AG, Tong HL, Kocaballi AB, Chen J, Bashir R, et al. Conversational agents in healthcare: a systematic review. J Am Med Inform Assoc 2018 Sep 1;25(9):1248-1258 [FREE Full text] [CrossRef] [Medline]
    3. Montenegro JL, da Costa CA, Righi R. Survey of conversational agents in health. Expert Syst Appl 2019 Sep;129:56-67. [CrossRef]
    4. Vaidyam AN, Wisniewski H, Halamka JD, Kashavan MS, Torous JB. Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can J Psychiatry 2019 Jul;64(7):456-464. [CrossRef] [Medline]
    5. Cohen PR, Oviatt SL. The role of voice input for human-machine communication. Proc Natl Acad Sci USA 1995 Oct 24;92(22):9921-9927 [FREE Full text] [CrossRef] [Medline]
    6. Cassell J. Embodied Conversational Agents. In: MIT press. Cambridge, MA: The MIT Press; 2000.
    7. Oviatt S. Multimodal interfaces. In: Jacko JA, editor. Human Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications. Second Edition. Boca Raton, Florida: CRC Press; 2007:286-304.
    8. Bickmore TW, Mitchell SE, Jack BW, Paasche-Orlow MK, Pfeifer LM, Odonnell J. Response to a relational agent by hospital patients with depressive symptoms. Interact Comput 2010 Jul 1;22(4):289-298 [FREE Full text] [CrossRef] [Medline]
    9. Tanaka H, Negoro H, Iwasaka H, Nakamura S. Embodied conversational agents for multimodal automated social skills training in people with autism spectrum disorders. PLoS One 2017;12(8):e0182151 [FREE Full text] [CrossRef] [Medline]
    10. Russo A, D'Onofrio G, Gangemi A, Giuliani F, Mongiovi M, Ricciardi F, et al. Dialogue systems and conversational agents for patients with dementia: the human-robot interaction. Rejuvenation Res 2019 Apr;22(2):109-120. [CrossRef] [Medline]
    11. Fitzpatrick KK, Darcy A, Vierhile M. Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Ment Health 2017 Jun 6;4(2):e19 [FREE Full text] [CrossRef] [Medline]
    12. Brixey J, Hoegen R, Lan W, Rusow J, Singla K, Yin X. SHIHbot: A Facebook chatbot for Sexual Health Information on HIV/AIDS. In: Proceedings of the 18th Annual SIGdial Meeting on DiscourseDialogue Stroudsburg, PA. 2017 Aug Presented at: SIGDIAL'17; 2017; Saarbrücken, Germany p. 370-373. [CrossRef]
    13. Cheng A, Raghavaraju V, Kanugo J, Handrianto Y, Shang Y. Development and Evaluation of a Healthy Coping Voice Interface Application Using the Google Home for Elderly Patients With Type 2 Diabetes. In: Proceedings of the 2018 15th IEEE Annual Consumer Communications & Networking Conference. 2018 Presented at: CCNC'18; January 12-15, 2018; Las Vegas, NV, USA. [CrossRef]
    14. Fadhil A. A conversational interface to improve medication adherence: towards AI support in patient's treatment. arXiv 2018:- [FREE Full text]
    15. Fadhil A, Gabrielli S. Addressing Challenges in Promoting Healthy Lifestyles: the AI-Chatbot Approach. In: Proceedings of the 11th EAI International Conference on Pervasive Computing Technologies for Healthcare. 2017 Presented at: PervasiveHealth'17; May 23-26, 2017; Barcelona, Spain p. 261-265. [CrossRef]
    16. Beveridge M, Fox J. Automatic generation of spoken dialogue from medical plans and ontologies. J Biomed Inform 2006 Oct;39(5):482-499 [FREE Full text] [CrossRef] [Medline]
    17. Barker DJ, van Schaik P, Simpson DS, Corbett WA. Evaluating a spoken dialogue system for recording clinical observations during an endoscopic examination. Med Inform Internet Med 2003 Jun;28(2):85-97. [Medline]
    18. Kenny PG, Parsons TD. Embodied conversational virtual patients. In: Perez-Marin D, Pascual-Nieto I, editors. Conversational Agents and Natural Language Interaction: Techniques and Effective Practices. Hershey, PA: IGI Global; 2011:254-281.
    19. Lane J. Science innovation. Assessing the impact of science funding. Science 2009 Jun 5;324(5932):1273-1275. [CrossRef] [Medline]
    20. Sanyal P. Understanding patents: The role Of R&D funding sources and the patent office. Econ Innov New Tech 2003;12(6):1. [CrossRef]
    21. Goeldner M, Herstatt C, Tietze F. The emergence of care robotics — A patent and publication analysis. Technol Forecast Soc Change 2015 Mar;92:115-131. [CrossRef]
    22. Daim TU, Rueda G, Martin H, Gerdsri P. Forecasting emerging technologies: use of bibliometrics and patent analysis. Technol Forecast Soc Change 2006 Oct;73(8):981-1012. [CrossRef]
    23. Chen H, Roco M, Son J, Jiang S, Larson C, Gao Q. Global nanotechnology development from 1991 to 2012: patents, scientific publications, and effect of NSF funding. J Nanopart Res 2013;15(9):1-21. [CrossRef]
    24. Du J, Tang XL. Natural products against cancer: a comprehensive bibliometric study of the research projects, publications, patents and drugs. J Cancer Res Ther 2014 Aug;10(Suppl 1):27-37 [FREE Full text] [CrossRef] [Medline]
    25. WIPO. WIPO Technology Trends 2019 Artificial Intelligence. Geneva, Switzerland: World Intellectual Property Organization; 2019.   URL: https://www.wipo.int/publications/en/details.jsp?id=4386
    26. Gao J, Galley M, Li L. Neural Approaches to Conversational AI. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics: Tutorial Abstracts. 2018 Presented at: ACL'18; July 16th–18th, 2018‎; Melbourne, Australia p. 1371-1374. [CrossRef]
    27. Wohlin C. Guidelines for Snowballing in Systematic Literature Studies and a Replication in Software Engineering. In: Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering. 2014 Presented at: EASE'14; May 13-14, 2014; London, England, United Kingdom. [CrossRef]
    28. Lv PH, Wang G, Wan Y, Liu J, Liu Q, Ma F. Bibliometric trend analysis on global graphene research. Scientometrics 2011;88(2):399-419. [CrossRef]
    29. Narin F. Patent bibliometrics. Scientometrics 1994 May;30(1):147-155. [CrossRef]
    30. Guan J, He Y. Patent-bibliometric analysis on the Chinese science — technology linkages. Scientometrics 2007;72(3):403-425. [CrossRef]
    31. Otte E, Rousseau R. Social network analysis: a powerful strategy, also for the information sciences. J Inf Sci 2002 Oct;28(6):441-453. [CrossRef]
    32. Perianes-Rodríguez A, Olmeda-Gómez C, Moya-Anegón F. Detecting, identifying and visualizing research groups in co-authorship networks. Scientometrics 2010;82(2):307-319. [CrossRef]
    33. Callon M, Courtial J, Turner WA, Bauin S. From translations to problematic networks: an introduction to co-word analysis. Soc Sci Inf 1983 Mar 1;22(2):191-235. [CrossRef]
    34. van Eck NJ, Waltman L. Visualizing bibliometric networks. In: Ding Y, Rousseau R, Wolfram D, editors. Measuring Scholarly Impact: Methods and Practice. Cham: Springer; 2014:285-320.
    35. Radhakrishnan S, Erbis S, Isaacs JA, Kamarthi S. Novel keyword co-occurrence network-based methods to foster systematic reviews of scientific literature. PLoS One 2017;12(3):e0172778 [FREE Full text] [CrossRef] [Medline]
    36. United States Patent and Trademark Office. 211 Claiming the Benefit of an Earlier Filing Date Under 35 USC 120 and 119(e) [R-07.2015].   URL: https://www.uspto.gov/web/offices/pac/mpep/s211.html [accessed 2019-04-19] [WebCite Cache]
    37. van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics 2010 Aug;84(2):523-538 [FREE Full text] [CrossRef] [Medline]
    38. van Eck NJ, Waltman L. Text mining and visualization using VOSviewer. arXiv preprint arXiv:1109.2058 2011 Sep:1-5 [FREE Full text]
    39. Bickmore TW, Utami D, Matsuyama R, Paasche-Orlow MK. Improving access to online health information with conversational agents: a randomized controlled experiment. J Med Internet Res 2016 Jan 4;18(1):e1 [FREE Full text] [CrossRef] [Medline]
    40. Greenwald JL, Jack BW. Preventing the preventable: reducing rehospitalizations through coordinated, patient-centered discharge processes. Prof Case Manag 2009;14(3):135-40; quiz 141 [FREE Full text] [CrossRef] [Medline]
    41. Burton C, Tatar AS, McKinstry B, Matheson C, Matu S, Moldovan R, Help4Mood Consortium. Pilot randomised controlled trial of Help4Mood, an embodied virtual agent-based system to support treatment of depression. J Telemed Telecare 2016 Sep;22(6):348-355. [CrossRef] [Medline]
    42. Ioannou A, Kartapanis I, Zaphirisocial P. Social robots as co-therapists in autism therapy sessions: a single-case study. In: Tapus A, André E, Martin JC, Ferland F, Ammi M, editors. Social Robotics. Cham: Springer; 2015:255-263.
    43. Baraka K, Melo F, Veloso M. ‘Autistic robots’ for embodied emulation of behaviors typically seen in children with different autism severities. In: Kheddar A, Yoshida E, Ge SS, Suzuki K, Cabibihan JJ, Eyssel F, editors. Social Robotics. Cham: Springer International Publishing; 2017:105-114.
    44. Kuwabara K, Shimode Y, Miyamoto S. Agent-based Remote Conversation Support for People With Aphasia. In: Proceedings of the 4th KES international conference on Agent and multi-agent systems: technologies and applications. 2010 Presented at: KES-AMSTA'10; June 23-25, 2010; Gdynia, Poland p. 371-380. [CrossRef]
    45. Kuwabara K, Hayashi S, Uesato T, Umadome K, Takenaka K. Remote conversation support for people with aphasia: some experiments and lessons learned. In: Stephanidis C, editor. Universal Access in Human-Computer Interaction. Addressing Diversity. Berlin, Heidelberg: Springer; 2009:375-384.
    46. Malik NA, Yussof H, Hanapiah FA, Rahman RA, Basri HH. Human-robot interaction for children with cerebral palsy: reflection and suggestion for interactive scenario design. Procedia Comput Sci 2015;76:388-393. [CrossRef]
    47. Borovac B, Gnjatovic M, Savic S, Rakovic M, Nikolic M. Human-like robot MARKO in the rehabilitation of children with cerebral palsy. In: Wenger P, Chevallereau C, Pisla D, Bleuler H, Rodic A, editors. New Trends in Medical and Service Robots: Human Centered Analysis, Control and Design. Cham: Springer; 2016:191-203.
    48. Zannatha JM, Tamayo AJ, Sánchez AD, Delgado JE, Cheu LE, Arévalo WA. Development of a system based on 3D vision, interactive virtual environments, ergonometric signals and a humanoid for stroke rehabilitation. Comput Methods Programs Biomed 2013 Nov;112(2):239-249. [CrossRef] [Medline]
    49. Jung H, Takahashi T, Choe Y, Baird J, Foster T, Grupen R. Towards extended virtual presence of the therapist in stroke rehabilitation. IEEE Int Conf Rehabil Robot 2013 Jun;2013:6650345. [CrossRef] [Medline]
    50. Alemi M, Ghanbarzadeh A, Meghdari A, Moghadam LJ. Clinical application of a humanoid robot in pediatric cancer interventions. Int J of Soc Robotics 2016;8(5):743-759. [CrossRef]
    51. Hassoon A, Schrack J, Naiman D, Lansey D, Baig Y, Stearns V, et al. Increasing physical activity amongst overweight and obese cancer survivors using an Alexa-based intelligent agent for patient coaching: protocol for the physical activity by technology help (PATH) trial. JMIR Res Protoc 2018 Feb 12;7(2):e27 [FREE Full text] [CrossRef] [Medline]
    52. Liu T, Meng M. Study on cooperation between humanoid robot Nao and Barrett WAM. In: 2012 IEEE International Conference on Robotics and Biomimetics. 2012 Presented at: ROBIO'12; December 11-14, 2012; Guangzhou, China p. 980-983. [CrossRef]
    53. Carelli L, Gaggioli A, Pioggia G, De Rossi F, Riva G. Affective robot for elderly assistance. Stud Health Technol Inform 2009;144:44-49. [Medline]
    54. Burger B, Ferrané I, Lerasle F, Infantes G. Two-handed gesture recognition and fusion with speech to command a robot. Auton Robot 2012;32(2):129-147. [CrossRef]
    55. Garzotto F, Gelsomini M, Kinoe Y. Puffy: A mobile inflatable interactive companion for children with neurodevelopmental disorder. In: Bernhaupt R, Dalvi G, Joshi A, Balkrishan D, O'Neill J, Winckler M, editors. Human-Computer Interaction - INTERACT 2017. Cham: Springer; 2017:467-492.
    56. Wang F, Chen L, Qu W, Zhu Y, Inventors. Elderly care service robot. Chinese Patent CN201810749966.3A 2018 Jul 10:NA-NA.
    57. Fountaine D. Google Patents. Voice Controlled Assistance for Monitoring Adverse Events of a User and/or Coordinating Emergency Actions Such as Caregiver Communication.   URL: https://patents.google.com/patent/US10258295B2/en [accessed 2019-11-05]
    58. Wu K, Wu L, Cui H, Inventor. Health-service robot capable of achieving electrocardiograph detection. Chinese Patent CN105310686A 2015 Aug 24:NA-NA.
    59. Chen M, Tian Q, Li X, Zhang L, Inventor. Intelligent limb motion monitor and application method. Chinese Patent CN106267730A 2016 Aug 12:NA-NA.
    60. Kozloski J, Murdock J, Pickover P, Walker G. Google Patents. Empathy Injection for Question-Answering Systems.   URL: https://patents.google.com/patent/US9298766B2/en [accessed 2019-11-05]
    61. Anumalasetty K, Ayyagari P, Bhide M. Patent Scope. Personal Assistant Computing System Monitoring.   URL: https:/​/patentscope.​wipo.int/​search/​en/​detail.​jsf;jsessionid=3EAB4FBD23824DC2313883CD6F122CDE.​wapp2nC?docId=US240620712&recNum=1080&office=&queryString=&prevFilter=&sortOption=Pub+Date+Desc&maxRec=74636364 [accessed 2019-11-05]
    62. Bagchi S, Ferrucci D, Levas A, Mueller E. Patent Scope. A Decision-Support Application and System for Problem Solving Using a Question-Answering System.   URL: https:/​/patentscope.​wipo.int/​search/​en/​detail.​jsf?docId=WO2012122198&recNum=184&docAn=US2012027942&queryString=((num& [accessed 2019-11-05]
    63. Kawahara T, Watabe S, Nakano Y. Voice interactive method and voice interactive system. Japan Patent JP2016038501A 2014 Aug 08:NA-NA.
    64. Alemi M, Meghdari A, Ghanbarzadeh A, Moghadam L, Ghanbarzadeh A. Effect of Utilizing a Humanoid Robot as a Therapy-assistant in Reducing Anger, Anxiety, and Depression. In: Proceedings of the 2014 Second RSI/ISM International Conference on Robotics and Mechatronics. 2014 Presented at: ICRoM'14; October 15-17, 2014; Tehran, Iran p. 748-753. [CrossRef]
    65. Shamekhi A, Trinh H, Bickmore T, DeAngelis T, Ellis T, Houlihan B. A Virtual Self-care Coach for Individuals with Spinal Cord Injury. In: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility. Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS'' ?16 New York, New York, USA; 2016 Presented at: ASSETS'16; October 24-26, 2016; Reno, Nevada, USA p. 327-328. [CrossRef]
    66. Auriacombe M, Moriceau S, Serre F, Denis C, Micoulaud-Franchi J, de Sevin E, et al. Development and validation of a virtual agent to screen tobacco and alcohol use disorders. Drug Alcohol Depend 2018 Dec 1;193:1-6. [CrossRef] [Medline]
    67. Philip P, Micoulaud-Franchi J, Sagaspe P, Sevin ED, Olive J, Bioulac S, et al. Virtual human as a new diagnostic tool, a proof of concept study in the field of major depressive disorders. Sci Rep 2017 Feb 16;7:42656 [FREE Full text] [CrossRef] [Medline]
    68. Aziz AA, Moganan FF, Ismail A, Lokman AM. Autistic children's kansei responses towards humanoid-robot as teaching mediator. Procedia Comput Sci 2015;76:488-493. [CrossRef]
    69. Abdul MN, Yussof H, Hanapiah F, Anne S. Human Robot Interaction (HRI) between a humanoid robot and children with Cerebral Palsy: Experimental framework and measure of engagement. In: Proceedings of the 2014 IEEE Conference on Biomedical Engineering and Sciences. 2014 Presented at: IECBES'14; December 8-10, 2014; Kuala Lumpur, Malaysia p. 430-435. [CrossRef]
    70. Bickmore T, Trinh H, Asadi R, Olafsson S. Safety first: conversational agents for health care. In: Moore RJ, Szymanski MH, Arar R, Ren GJ, editors. Studies in Conversational UX Design. Cham: Springer; 2018:33-57.
    71. Miner AS, Milstein A, Schueller S, Hegde R, Mangurian C, Linos E. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Intern Med 2016 May 1;176(5):619-625 [FREE Full text] [CrossRef] [Medline]
    72. Palanica A, Flaschner P, Thommandram A, Li M, Fossat Y. Physicians' perceptions of chatbots in health care: cross-sectional web-based survey. J Med Internet Res 2019 Apr 5;21(4):e12887 [FREE Full text] [CrossRef] [Medline]
    73. Bickmore TW, Pfeifer LM, Paasche-Orlow MK. Using computer agents to explain medical documents to patients with low health literacy. Patient Educ Couns 2009 Jun;75(3):315-320 [FREE Full text] [CrossRef] [Medline]
    74. Bickmore T, Pfeifer L, Jack B. Taking the Time to Care: Empowering Low Health Literacy Hospital Patients With Virtual Nurse Agents. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 2009 Presented at: CHI'09; April 4–9, 2009; Boston, MA, USA p. 1265-1274. [CrossRef]
    75. Provoost S, Lau HM, Ruwaard J, Riper H. Embodied conversational agents in clinical psychology: a scoping review. J Med Internet Res 2017 May 9;19(5):e151 [FREE Full text] [CrossRef] [Medline]
    76. Pennisi P, Tonacci A, Tartarisco G, Billeci L, Ruta L, Gangemi S, et al. Autism and social robotics: a systematic review. Autism Res 2016 Feb;9(2):165-183. [CrossRef] [Medline]
    77. Sarabia M, Young N, Canavan K, Edginton T, Demiris Y, Vizcaychipi MP. Assistive robotic technology to combat social isolation in acute hospital settings. Int J of Soc Robotics 2018;10(5):607-620. [CrossRef]
    78. Yu N, Xu C, Wang K, Yang Z, Liu J. Gesture-based Telemanipulation of a Humanoid Robot for Home Service Tasks. In: Proceedings of the 2015 IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems. 2015 Presented at: CYBER'15; June 8-12, 2015; Shenyang, China p. 1923-1927. [CrossRef]

    ‎
    AHRQ: Agency for Healthcare Research and Quality
    ASD: autism spectrum disorder
    CIs: conversational interfaces
    CNRS: National Center for Scientific Research
    CORDIS: Community Research and Development Information Service
    DII: Derwent Innovation Index
    EC: Europe Commission
    EU: European Union
    NIH: National Institutes of Health
    NSF: National Science Foundation
    PI: principal investigator
    R&D: research and development
    WIPO: World Intelligence Property Organization

    Edited by G Eysenbach; submitted 13.05.19; peer-reviewed by S Zhou, D Carvalho; comments to author 28.05.19; revised version received 10.10.19; accepted 19.10.19; published 18.11.19

    Copyright

    ©Zhaopeng Xing, Fei Yu, Jian Du, Jennifer S. Walker, Claire B. Paulson, Nandita S. Mani, Lixin Song. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 18.11.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.