This paper is in the following e-collection/theme issue:

Participatory Medicine & E-Patients (758) Consumer & Patient Education and Shared-Decision Making (636) Information Retrieval (271) User Needs and Competencies (270) Information Seeking, Information Needs (419) Users' and Patients' Needs for Mental Health Services (378) Pregnancy Information, Education and Lifestyle Interventions (281)

Published on in Vol 22, No 3 (2020): March

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14095, first published .
Medical Help-Seeking Strategies for Perinatal Women With Obstetric and Mental Health Problems and Changes in Medical Decision Making Based on Online Health Information: Path Analysis

Medical Help-Seeking Strategies for Perinatal Women With Obstetric and Mental Health Problems and Changes in Medical Decision Making Based on Online Health Information: Path Analysis

Medical Help-Seeking Strategies for Perinatal Women With Obstetric and Mental Health Problems and Changes in Medical Decision Making Based on Online Health Information: Path Analysis

Authors of this article:

Kyungmi Chung1, 2 Author Orcid Image ;   Hee Young Cho3 Author Orcid Image ;   Young Ran Kim3 Author Orcid Image ;   Kyungun Jhung4 Author Orcid Image ;   Hwa Seon Koo5 Author Orcid Image ;   Jin Young Park1, 2 Author Orcid Image
Article Authors Cited by (13) Tweetations (4) Metrics

    Original Paper

    1Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea

    2Department of Psychiatry, Yonsei University College of Medicine, Severance Hospital, Yonsei University Health System, Seoul, Republic of Korea

    3Department of Obstetrics and Gynecology, CHA Bundang Medical Center, CHA University School of Medicine, Seongnam, Republic of Korea

    4Department of Psychiatry and Behavioral Neuroscience, International St. Mary's Hospital, Catholic Kwandong University, Incheon, Republic of Korea

    5Fertility Center, CHA Bundang Medical Center, CHA University School of Medicine, Seongnam, Republic of Korea

    Corresponding Author:

    Jin Young Park, MD, PhD

    Department of Psychiatry

    Yonsei University College of Medicine

    Severance Hospital, Yonsei University Health System

    50-1 Yonsei-ro, Seodaemun-gu

    Seoul,

    Republic of Korea

    Phone: 82 2 2228 0972

    Fax:82 2 2227 6558

    Email: empathy@yuhs.ac


    Background: Previous studies have revealed that most pregnant women rarely discuss informal information found on the internet with health professionals and have frequently expressed concerns for medical experts’ reactions to the online information they shared, as well as the lack of time to consult the medical experts in general. To date, little information is available on the effect of individual differences in utilizing medical help-seeking strategies on their medical decisions during the perinatal period.

    Objective: The objectives of this study were (1) to determine associations among perinatal women’s medical help-seeking strategies, changes in medical decision making, and online health information utilization with a focus on the mediating effect of self-efficacy in perinatal health literacy on the intent to consult health professionals, and (2) to clarify these associations in perinatal women with two different medical problems: obstetric and mental health.

    Methods: A total of 164 perinatal women aged 24 to 47 years (mean 34.64, SD 3.80) repeatedly completed the Problem Solving in Medicine and Online Health Information Utilization questionnaires to examine the moderating effect of two types of medical problems on their decision-making processes. To validate the hypothesized relationships in the proposed conceptual model encompassing obstetric and mental health problem-solving models, path analyses were performed.

    Results: This study found that some perinatal women, who use an online informal medical help-seeking (OIMH) strategy, would be more likely to change their medical decisions based only on internet-based information without consulting health professionals (P<.001), compared to other women using different medical help-seeking strategies. Particularly, this concern is significantly prevalent when encountering obstetric problems during the perinatal period (obstetric problem-solving: P<.001; mental health problem-solving: P=.02). Furthermore, perinatal women with mental health issues using the OIMH strategy showed a significant difference in intent to consult health professionals based on online health information when the medical problem they had to solve was different (obstetric problem-solving: P=.94; mental health problem-solving: P=.003).

    Conclusions: Despite the positive mediating effects of perinatal women’s enhanced health literacy on the intent to discuss personal medical issues with health professionals based on online health information, the strategy used is of fundamental importance for understanding their help-seeking and decision-making processes during the perinatal period. Beyond a short consultation to steer patients quickly and authoritatively towards an obstetric doctor’s choice of action, it is recommended in this study that obstetricians attempt to provide their patients with needed context for the information found online. To fully explain this information with an open mind, they should actively develop or support information and communications technology (ICT)-based health information services.

    J Med Internet Res 2020;22(3):e14095

    doi:10.2196/14095

    Keywords

    perinatal care; obstetrics; mental health; information seeking behavior; help-seeking behavior; self efficacy; health literacy; consultation; decision making; internet 


    Background

    Medical help-seeking can be defined as the ability to actively seek help from others to cope with medical problems or painful experiences [1-3]. The help-seeking process is a transition from the personal domain (being aware of personal needs, thoughts, and feelings) to the interpersonal domain (being willing to share and disclose one’s needs to others) [2,3]. Internet health-seeking complements rather than replaces in-person health-seeking [4,5]. Particularly, online health information has been used to fill an informational void that promotes self-efficacy and affects medical decisions and actions about how to treat an illness or other medical conditions; this information can be discussed with friends, family members, and health care providers [6]. Physical and mental health professionals have serious concerns about the quality of online health information [6], and women in the perinatal period from conception through pregnancy to childbirth and the postpartum stage become more overwhelmed with false or misleading health information from multiple online sources, which in turn causes them to feel frightened and anxious rather than informed and empowered [7-11] and vise versa [12]. Even though internet searching allows pregnant women with the need for empirical medical information specific to their own situations to readily access experienced women’s birth stories on social media and provides social support to anxious women at the same time [13], women with a less experienced or robust capacity to make sense of conflicting health information from informal online sources may challenge their pre-existing ideations and aspirations of birth processes and address issues that they have not yet considered [11,14]. Hence, the provision of appropriate online health information can be considered an important step toward patient education and empowerment [6,15].

    When it comes to dealing with difficulty assessing the accuracy, reliability, and credibility of relevant medical information on the internet, the evidenced-based information provided by health professionals will play a critical role in guiding perinatal women’s medical decisions on the right course of treatment [16]. For prenatal women, the association between trust in health providers and medical information sources should be carefully considered. Unlike many chronic disease patients who maintain the primacy of information acquisition from their doctors [17], mothers with lower levels of trust in health providers use more informal sources such as friends, family, other parents, the internet, and alternative medicine providers and do not regard their doctors as the main source of medical information [18]. Furthermore, most pregnant women have rarely discussed informal information found on the internet with health professionals [19], having frequently expressed concerns on medical experts’ negative reactions to the online information they brought to the outpatient clinic [20] as well as on the lack of time to consult them [21,22]. Without either perceiving self-efficacy in perinatal health literacy or contacting health professionals as the last bastion of medical decision based on online health information, women who solve medical problems by online informal consultation or search results can be more likely to make the wrong medical decisions, fail to receive appropriate medical treatments, or be confronted with new decision-making situations [23]. In this regard, this study focused on the association between perinatal women’s intentions of online informal medical help-seeking for health problems and their medical decision-making processes based on health information from the internet.

    To address this issue, individual differences in using medical help-seeking strategies should be considered because the majority of pregnant women (83.7%) tend to use multiple information sources whose conflicting information increases anxiety and leads those with no chronic diseases to decide not to use medicines more than those with chronic diseases [24]. Based on the Chen and colleagues’ research model [25], this study categorizes perinatal women into four types of health information seekers with different medical help-seeking strategies, considering the formality (formal: querying doctors, medical experts, and health care professionals vs informal: querying family members, friends or relatives, and experienced people with previous medical problem-solving experiences) and channel (online: email, internet medical forum, instant messenger, and websites or online blogs vs nononline: face-to-face visit and communication) of the information source: (1) nononline formal medical help-seeking (NFMH), (2) online formal medical help-seeking (OFMH), (3) nononline informal medical help-seeking (NIMH), and (4) online informal medical help-seeking (OIMH). Further, when encountering two different obstetric and mental health problems, whether perinatal women with the four distinct medical help-seeking strategies will make medical decisions and change their decisions in a different manner, particularly based on online health information, has yet to be determined by comparing the proposed research model within the same Korean sample.

    Objectives

    Given the potential impact of the information on a patient’s medical decision-making process that will lead to a change in their health-seeking behaviors, the personal, cognitive, and social skills that determine an individual’s motivation and ability to obtain, evaluate, and use health care information is of great importance for making appropriate medical decisions and promoting and maintaining good health [26]. The objective of this study was to develop and validate a medical problem-solving model to (1) determine associations among perinatal women’s medical help-seeking strategies, changes in medical decision making, and online health information use with a focus on the mediating effect of self-efficacy in perinatal health literacy on the intent to consult health professionals and (2) clarify these associations in perinatal women with two different medical problems (obstetric and mental health).

    Research Model and Hypotheses

    As shown in Figure 1, this study proposes a medical problem-solving model with a 2-step mediation approach that includes promoting self-efficacy in perinatal health literacy (PS) and consulting others (CO), both based on online health information perinatal women obtained, as potential mediators of the effects of perinatal women’s medical help-seeking strategies on changes in their medical decisions based on online health information (CD). Furthermore, the moderating role of two types of medical problems (obstetric vs mental health) will be further examined in this conceptual research model. According to the model, the following hypotheses are stated in the literature review.

    ‎
    Figure 1. Medical problem-solving model. NFMH: nononline formal medical help-seeking; NIMH: nononline informal medical help-seeking; OFMH: online formal medical help-seeking; OIMH: online informal medical help-seeking; PS: promoting self-efficacy in perinatal health literacy based on online health information; CO: consulting others (health professionals) based on online health information; CD: changing their medical decisions based on online health information.
    View this figure

    In terms of the formality of medical help-seeking strategies, patients who prefer to seek medical help from formal sources have good health literacy [27]. Particularly, patients seeking online formal medical help achieve better health literacy [28] than those seeking nononline formal medical help in that the OFMH strategy is beneficial to compare, cross-check, and evaluate health-related information from multiple reliable information sources by looking for consistent results. However, as far as patients’ trust in their doctors is concerned, perinatal women with high trust levels will promote self-efficacy in perinatal health literacy based on their doctors’ guidelines, while those with low trust levels will do so based on online health information because they do not regard their doctors as the main source of medical information [18]. Thus, we posit the following hypotheses:

    • H1/H2: Perinatal women’s strategies of (1) NFMH and (2) OFMH will be positively associated with PS.

    Additionally, a recent study of patient samples, not specifically consisting of perinatal women, revealed that patients employing an NFMH strategy when facing medical problems had a significantly positive association with consulting others about relevant issues based on the online health information they obtained [25]. In the perinatal period, women who have maintained a long-term doctor-patient relationship through rapport building can be motivated to directly address their medical concerns sourced from the internet in order to reduce anxiety before and after a visit to the health professional [19,20,29-34]. As a facilitating factor in the decision to communicate with health professionals about difficulties in the perinatal period [35], a woman’s social network (eg, partner, friends, relatives, family members, and experienced mothers) will influence the help-seeking process by encouraging professional help-seeking, participating in the decision-making process, and initiating help-seeking [36]. Accordingly, the following hypotheses are proposed:

    • H3/H4: Perinatal women’s strategies of (3) NFMH and (4) NIMH will be positively associated with CO.

    Among the various benefits of using the internet, providing perinatal women with personal and emotional support from other women in similar situations is of unquestionable importance [13], since they are vulnerable to undergoing unknown, unpredictable events. In fact, finding people with the same medical problems is not easy online, but it will be helpful to share their concerns and doubts with other women and take positive energy from one another in online forums and social media platforms [37], thereby reducing anxiety and increasing confidence in making medical decisions for themselves and their babies after such internet use [20,21]. In addition, the internet can help expert users identify more questions they wanted to ask a health professional, be more involved in the decision-making process, make better decisions, and feel they had more control over their decisions than novice users [21]. Hence, we postulate the following hypotheses:

    • H5: The perinatal women’s strategy of OIMH will be positively associated with PS.
    • H6: The perinatal women’s strategy of OIMH will be positively associated with CO.
    • H7: The perinatal women’s strategy of OIMH will be positively associated with CD.

    According to a continuum of health literacy including basic or functional health literacy, communicative/interactive health literacy, and critical health literacy, there is a need to focus more on developing the skills and confidence to make well-informed health choices than on limiting the transmission of information [38]. The internet can empower people, including pregnant women, to communicate with their health professionals and play a critical role in making decisions [21]. However, the association between promoting self-efficacy in perinatal health literacy and changing perinatal women’s medical decisions might interfere with their anxiety levels. First-time pregnant women show a significant negative association between the level of maternal anxiety and feelings of control during labor [39]. As fears can be caused by negative moods and stories told by others about alarming information, diseases, and child-related problems in primiparas as well as negative experiences of a previous pregnancy, childbirth, and a baby’s health and care in multiparas, it will not be easy to change their views to align with the online health information they obtained without communicating with health professionals [40]. Therefore, we hypothesize the following:

    • H8: PS will be positively associated with CO.
    • H9: PS will be negatively associated with CD.
    • H10: CO will be positively associated with CD.

    Participant Recruitment

    To test the proposed hypotheses, we administered an internet-based questionnaire built with Google Forms (Google Inc). Perinatal women aged 19 to 59 years who were being prepared for pregnancy, in fertility treatment, pregnant, or postpartum (up to 2 years after delivery) and had the ability to read, understand, and respond to the questionnaire items were eligible to be enrolled in this study. Those who failed to meet the inclusion criteria were not listed in our survey pool. In total, 168 women responded to the questionnaire for a 50.5% response rate, as the total survey pool consisted of 333 patients from the Fertility Center and/or Department of Obstetrics Gynecology, CHA Bundang Women’s Hospital, South Korea. This study was approved by the institutional review board of CHA Bundang Medical Center, CHA University.

    The questionnaire was delivered to the participants’ mobile phones via a Google Forms link embedded in a short message service or multimedia messaging service text message or chat in KakaoTalk (Kakao Corp). Participants were allowed to access the survey link during a 2-month period from November 1, 2018, to December 31, 2018. To identify participants who met the criteria, a unique identification (ID) number was assigned to each patient with the survey link, and all patients were instructed to enter the preassigned ID number into the Google Form. Before receiving access to the online questionnaires, patients were asked to read and understand the aims of this study and relevant information, confirming that participating in the survey would be considered the equivalent of giving informed consent.

    Measurement Instruments

    To assess the associations between perinatal women’s strategies of medical help-seeking to solve obstetric and mental health problems, medical decision making, and their online medical information use, the Problem Solving in Medicine and Online Health Information Utilization questionnaires developed by Chen and colleagues [25] were implemented twice in this study. The wording of the original questionnaire items was slightly modified to reflect the context of medical problems to be solved.

    The Problem Solving in Medicine questionnaire consisted of 2 constructs with 6 observed variables measuring 27 latent variables: medical help-seeking (NFMH: 4 items, OFMH: 4 items, NIMH: 5 items, and OIMH: 5 items) and health information searching (nononline health information search [4 items] and online health information search [5 items]). The Online Health Information Utilization questionnaire included 2 constructs with 3 observed variables measuring 12 latent variables: medical decision making (PS: 4 items and CO: 4 items) and changes in medical decision making (CD: 4 items) based on online health information. All items were rated on a 7-point Likert scale ranging from 1=strongly disagree to 7=strongly agree. The reliability, validity (Table 1), and descriptive statistics (Table 2) were assessed for the measurement instruments.

    Pregnancy-related anxiety was measured with a 10-item scale [41] that assessed the extent to which women felt concerned about their health, their baby’s health, labor and delivery, and caring for a baby. In this study, responses were made on a 5-point Likert scale ranging from 1=not at all to 5=very much. Items on the pregnancy-related anxiety scale were as follows: (1) I am confident of having a normal childbirth (reverse-coded), (2) I think my labor and delivery will go normally (reverse-coded), (3) I have a lot of fear regarding the health of my baby, (4) I am worried that the baby could be abnormal, (5) I am afraid that I will be harmed during delivery, (6) I am worried about how the baby is growing and developing inside me, (7) I am worried about losing the baby, (8) I am worried about having a hard or difficult labor and delivery, (9) I am worried about taking care of a new baby, and (10) I am worried about developing medical problems during my pregnancy.

    Table 1. The internal reliability and convergent validity of the measurement instruments.
    Factor and itemFactor loadingCronbach alphaComposite reliabilityaAverage variance extracted
    NFMHb




    		NFMH10.84.800.860.67

    		NFMH20.79———

    		NFMH30.83———
    OFMHc




    		OFMH10.84.850.850.58

    		OFMH20.80———

    		OFMH30.76———

    		OFMH40.64———
    NIMHd




    		NIMH20.85.830.870.68

    		NIMH40.72———

    		NIMH50.90———
    OIMHe




    		OIMH10.71.860.860.55

    		OIMH20.66———

    		OIMH30.72———

    		OIMH40.82———

    		OIMH50.78———
    PSf




    		PSOHI10.68.860.890.67

    		PSOHI20.89———

    		PSOHI30.87———

    		PSOHI40.82———
    COg




    		COOHI10.59.780.830.62

    		COOHI20.89———

    		COOHI40.85———
    CDh




    		CDOHI10.83.930.940.80

    		CDOHI20.91———

    		CDOHI30.93———

    		CDOHI40.91———

    aComposite reliability is defined for each factor by the ratio of the total variance (squared sum of standardized factor loadings for items) to the observed variance (squared sum of standardized factor loadings for items + sum of error variances). As Cronbach alpha is sensitive to the number of items in the scale and tends to underestimate the internal consistency reliability except under restricted assumptions of equal common factor loadings and uncorrelated measurement errors, composite reliability is measured as the second check for internal consistency.

    bNFMH: nononline formal medical help-seeking.

    cOFMH: online formal medical help-seeking.

    dNIMH: nononline informal medical help-seeking.

    eOIMH: online informal medical help-seeking.

    fPS: promoting self-efficacy in health literacy based on online health information (PSOHI).

    gCO: consulting others based on online health information (COOHI).

    hCD: changing medical decisions based on online health information (CDOHI).

    Table 2. Descriptive statistics and discriminant validity of the measurement instruments.
    FactorMean (SD)NFMHaOFMHbNIMHcOIMHdPSeCOfCDg
    NFMH5.44 (1.26)0.82h——————
    OFMH3.44 (1.56)0.300.76h—————
    NIMH4.82 (1.41)0.250.130.83h————
    OIMH2.95 (1.43)0.090.570.300.74h———
    PS4.29 (1.11)0.190.300.180.270.82h——
    CO4.84 (1.07)0.250.190.250.240.330.79h—
    CD4.09 (1.28)0.050.080.150.260.030.280.90h

    aNFMH: nononline formal medical help-seeking.

    bOFMH: online formal medical help-seeking.

    cNIMH: nononline informal medical help-seeking.

    dOIMH: online informal medical help-seeking.

    ePS: promoting self-efficacy in health literacy based on online health information.

    fCO: consulting others based on online health information.

    gCD: changing medical decisions based on online health information.

    hDiagonal elements represents the square roots of the average variance extracted.

    Kaisor-Meyer-Olkin and Bartlett sphericity tests, conducted to measure the sampling adequacy, showed that the samples met the criteria for factor analysis (0.83, P<.001). According to the results of an exploratory factor analysis using SPSS Statistics 18.0 software (IBM Corp) with varimax rotation on these 27 items, online health information search and nononline health information search were not clearly defined by the intended items and were excluded from the data analysis. We eliminated the following 4 items with loading values less than 0.50: NFMH4 (When I have an obstetric or mental health problem, I will ask the pharmacist of a nearby pharmacy store for information), NIMH1 (When I have an obstetric or mental health problem, I will seek help from a drugstore), NIMH3 (When I have an obstetric or mental health problem, I will go to the temple praying to god for advice), and COOHI3 (I will discuss relevant issues with my family or friends based on the health information on the internet). On the basis of the results of the exploratory factor analysis, the factor CO, which is an abbreviation for consulting others including a doctor (COOHI2), family members or friends (COOHI3), and other medical experts (COOHI4), did not include the item COOHI3, indicating that perinatal women of this study sample did not consider their family and friends as an important reference when making their medical decisions.


    Participant Characteristics

    For the data analysis, 4 out of the 168 respondents were excluded (2.4%): 2 did not enter their given IDs and 2 did not have easy access to the internet when needed. A total of 164 perinatal women were included ranging in age from 24 to 47 years, with a mean age of 34.64 (SD 3.80) years. Perinatal women showed a mild to moderate level of fear and anxiety about their own health, their baby’s health, and labor and delivery outcomes (mean 2.85 [SD 0.69]), and the majority of them were aged from 30 to 49 years (150/164, 91.5%). As presented in Table 3, the sociodemographic and clinical characteristics of this study sample were documented as well as relevant information-seeking data more in detail.

    Table 3. Sociodemographic and clinical characteristics of this study sample (n=164).
    CharacteristicsParticipants, n (%)
    Gender

    		Female164 (100)
    Age of perinatal women in years

    		19-2914 (8.5)

    		30-39134 (81.7)

    		40-4916 (9.8)
    Age of their partners in years

    		19-298 (4.9)

    		30-39116 (70.7)

    		40-4939 (23.8)

    		50-591 (0.6)
    Marital status

    		Unmarried2 (1.2)

    		Married162 (98.8)
    Highest level of education completed

    		High school14 (8.5)

    		Technical college (2-3 years)37 (22.6)

    		Undergraduate degree (bachelor’s: 4-5 years)76 (46.3)

    		Postgraduate degree (master’s)32 (19.5)

    		Postgraduate degree (PhD)5 (3.0)
    Religious status

    		Nonreligious69 (42.1)

    		Christian (Protestant)49 (29.9)

    		Christian (Catholic)27 (16.5)

    		Buddhist19 (11.6)
    Professional status

    		Unemployed (housewife)72 (43.9)

    		Employed92 (56.1)
    Parity

    		Nullipara8 (4.9)

    		Primipara120 (73.2)

    		Multipara36 (22.0)
    Fetal plurality

    		Before birth23 (14.0)

    		Singleton131 (79.9)

    		Twins8 (4.9)

    		Singleton + twins2 (1.2)
    Number of children

    		Before birth23 (14.0)

    		197 (59.1)

    		239 (23.8)

    		34 (2.4)

    		41 (0.6)
    Number of sources for searching pregnancy/delivery information

    		One13 (7.9)

    		Multiple151 (92.1)
    Most important aspect of pregnancy/delivery information seeking

    		Accuracy and reliability of information115 (70.1)

    		Diversity of information13 (7.9)

    		Recency of information4 (2.4)

    		Use of information3 (1.8)

    		Ease of use of media25 (15.2)

    		Interactivity of media4 (2.4)
    Digital device used for information seeking (multiple response question)

    		Computer (desktop/laptop)34 (16.8)

    		Tablet7 (3.5)

    		Smartphone161 (79.7)

    Measurement Model

    A confirmatory factor analysis performed with SPSS Amos 18.0 software (IBM Corp) indicated that the measurement model provided an acceptable fit to the data (χ2278.00=686.03, normed chi-square [χ2/df]=2.47, comparative fit index [CFI] 0.92, Tucker-Lewis index [TLI] 0.90, standardized root mean square residual [SRMR] 0.06, and root mean square error of approximation [RMSEA] 0.07). All values with an adequate level of construct validity were achieved by the recommended fit indices of the measurement model: χ2/df<3.0 [42], CFI>0.90 [43], TLI>0.90 [44], SRMR<0.08 [45], and RMSEA<0.08 [46].

    The reliability and validity results (Tables 1 and 2) indicated that the questionnaire items had acceptable internal consistency with Cronbach alpha>.70 and composite reliability values>0.60, convergent validity values with factor loadings (all significant at P<.001) and average variance extracted values>0.50, and discriminant validity with the square roots of the average variance extracted values being higher than the correlations between the factors.

    Structural Model and Hypothesis Testing

    To validate the hypothesized relationships in the proposed conceptual model (Figure 1), path analyses were performed. As mentioned previously, the same-fit indices of the measurement model were used to assess the goodness of fit of structural models. The results of the path analysis for pregnant women’s medical help-seeking strategies, decision making, and any changes based on online information showed that the conceptual model had satisfactory levels of fit indices: χ25.00=5.07, P=.41, χ2/df=1.01, CFI 1.00, TLI 1.00, SRMR 0.02, and RMSEA 0.01. As predicted in H1, H2, and H5, the strategies of NFMH (H1: β=.12, P=.02), OFMH (H2: β=.17, P=.01), and OIMH (H5: β=.17, P=.009) had positive effects on PS. Consistent with H3, H4, and H6, the strategies of NFMH (H3: β=.16, P=.003), NIMH (H4: β=.13, P=.02), and OIMH (H6: β=.12, P=.03) had positive effects on CO. Among those strategies for medical help-seeking, OIMH (H7: β=.23, P<.001) appeared as the only significant determinant of CD. H8, H9, and H10 were also supported by the results as CO was positively affected by PS (H8: β=.25, P<.001); however, CD was negatively affected by PS (H9: β=–.12, P=.03) and positively affected by CO (H10: β=.26, P<.001; Figure 2).

    ‎
    Figure 2. Medical problem-solving model with standardized path coefficients (*P<.05, **P<.01, ***P<.001). NFMH: nononline formal medical help-seeking; NIMH: nononline informal medical help-seeking; OFMH: online formal medical help-seeking; OIMH: online informal medical help-seeking; PS: promoting self-efficacy in perinatal health literacy based on online health information; CO: consulting others (health professionals) based on online health information; CD: changing their medical decisions based on online health information.
    View this figure

    In addition, the other structural model for the moderating effect of different types of medical problems that pregnant women might encounter showed a satisfactory model fit: χ210.00=10.34, P=.41, χ2/df=1.03, CFI 1.00, TLI 1.00, SRMR 0.02, and RMSEA 0.01. The results of the multiple group path analysis (Table 4) revealed that the only significant difference was found in 2 paths, H6_1 and H6_2, for obstetric and mental health problem-solving models, respectively (OIMHCO; critical ratio [CR] 2.01, significant at P<.05 [if CR>1.96]).

    In the obstetric problem-solving model, the following hypotheses were rejected: H1_1 (β=.10, P=.20), H3_1 (β=.10, P=.18), H5_1 (β=.12, P=.17), H6_1 (β=.01, P=.94), and H9_1 (β=–.11, P=.15). As hypothesized, H2_1 (β=.21, P=.02) and H4_1 (β=.18, P=.02) were supported. In the mental health problem-solving model, the following hypotheses were rejected: H1_2 (β=.14, P=.07), H2_2 (β=.10, P=.29), H4_2 (β=.09, P=.24), and H9_2 (β=–.13, P=.10). Unlike the results of hypothesis testing for the obstetric problem-solving model, H3_2 (β=.20, P=.007), H5_2 (β=.24, P=.01), and H6_2 (β=.22, P=.003) were supported. In both models, the findings indicate that online health information was considered an important reference only for those with an OIMH strategy (H7_1: β=.27, P<.001; H7_2: β=.19, P=.02) when changing their medical decisions. Based on the online health information use, PS had positive effects on CO (H8_1: β=.26, P<.001; H8_2: β=.23, P=.002); furthermore, CO had positive effects on CD (H10_1: β=.25, P=.001; H10_2: β=.28, P<.001; Figure 3).

    Table 4. Summary of the hypothesis testing results.
    Model and hypothesesβaP valueSEbCRcSupported
    Medical problem solving (n=328)





    		H1: NFMHd→PSh.12.020.052.25Yes

    		H2: OFMHe→PS.17.010.052.56Yes

    		H3: NFMH→COi.16.0030.042.99Yes

    		H4: NIMHf→CO.13.020.042.36Yes

    		H5: OIMHg→PS.17.0090.052.61Yes

    		H6: OIMH→CO.12.030.042.12Yes

    		H7: OIMH→CDj.23<.0010.054.16Yes

    		H8: PS→CO.25<.0010.054.72Yes

    		H9: PS→CD–.12.030.07–2.16Yes

    		H10: CO→CD.26<.0010.074.72Yes
    Obstetric problem solving (n=164)





    		H1_1: NFMH→PS.10.200.071.27No

    		H2_1: OFMH→PS.21.020.062.38Yes

    		H3_1: NFMH→CO.10.180.061.34No

    		H4_1: NIMH→CO.18.020.052.29Yes

    		H5_1: OIMH→PS.12.170.071.36No

    		H6_1: OIMH→COk.01.940.060.07No

    		H7_1: OIMH→CD.27<.0010.073.58Yes

    		H8_1: PS→CO.26<.0010.073.42Yes

    		H9_1: PS→CD–.11.150.09–1.43No

    		H10_1: CO→CD.25.0010.103.24Yes
    Mental health problem solving (n=164)





    		H1_2: NFMH→PS.14.070.071.83No

    		H2_2: OFMH→PS.10.290.071.05No

    		H3_2: NFMH→CO.20.0070.062.68Yes

    		H4_2: NIMH→CO.09.240.061.17No

    		H5_2: OIMH→PS.24.010.072.50Yes

    		H6_2: OIMH→COk.22.0030.062.96Yes

    		H7_2: OIMH→CD.19.020.072.38Yes

    		H8_2: PS→CO.23.0020.083.12Yes

    		H9_2: PS→CD–.13.100.09–1.64No

    		H10_2: CO→CD.28<.0010.093.46Yes

    aβ: standardized coefficient.

    bSE: standard error.

    cCR: critical ratio.

    dNFMH: nononline formal medical help-seeking.

    eOFMH: online formal medical help-seeking.

    fNIMH: nononline informal medical help-seeking.

    gOIMH: online informal medical help-seeking.

    hPS: promoting self-efficacy in health literacy based on online health information.

    iCO: consulting others based on online health information.

    jCD: changing medical decisions based on online health information.

    kA multigroup path analysis was performed between H6_1 and H6_2.

    ‎
    Figure 3. Obstetric (top) and mental health (bottom) problem-solving models with standardized path coefficients (*P<.05, **P<.01, ***P<.001). NFMH: nononline formal medical help-seeking; NIMH: nononline informal medical help-seeking; OFMH: online formal medical help-seeking; OIMH: online informal medical help-seeking; PS: promoting self-efficacy in perinatal health literacy based on online health information; CO: consulting others (health professionals) based on online health information; CD: changing their medical decisions based on online health information.
    View this figure

    Finally, mediation analyses were conducted by examining the direct and indirect effects of perinatal women’s medical help-seeking strategies and online health information use with CO and CD, with a focus on PS. It was revealed in the entire conceptual model that PS partially mediates the relation between CO and the strategies of NFMH (β=.03, P=.01) and OIMH (β=.04, P=.005), fully mediates the relation between OFMH and CO (β=.04, P=.003), and fully mediates the relation between NFMH and CD (β=.03, P=.049). In the obstetric problem-solving model, PS fully mediated the relation between the strategy of OFMH and CO (β=.06, P=.01), and in the mental health problem-solving model, PS partially mediated the relation between OIMH and CO (β=.06, P=.006).


    Principal Findings

    As the goodness-of-fit indices between the measurement and structural models and the significance of all the hypothesized paths were statistically confirmed, the proposed conceptual model is expected to extend the understanding of the medical decision-making process in perinatal women with the 4 different medical help-seeking strategies. However, the decision-making process among perinatal women relied on which medical problem they would encounter, obstetrics or mental health, because there were substantial differences in the paths to self-efficacy in perinatal health literacy and intent to consult health professionals from their different medical help-seeking strategies, illustrated in the path diagrams (Figure 3). In the medical problem-solving model, the medical problem type had a partial moderating effect between OIMH and CO (H6_1 vs H6_2). Contrary to when the perinatal women using the OIMH strategy needed to solve mental health problems, no significant direct path from their adoption of the OIMH strategy to the intent to CO was found when they needed to solve obstetric problems. Furthermore, the direct paths from perinatal women’s adoption of the OIMH strategy to CD were significant in both obstetric and mental health problem-solving models.

    When encountering obstetric problems, perinatal women with the OIMH strategy were more likely to change their medical decisions based on numerous conflicting and confusing online information sources without consulting health professionals compared with those with the same strategy to solve mental health problems. In this study, the majority of the perinatal women (151/164, 92.1%) preferred to search pregnancy/delivery information from multiple sources to compensate for a lack of accuracy and reliability of information, and only 4 out of the remaining 13 women (30.8%) preferred to obtain the information from doctors, nurses, and other health professionals, indicating they did not consider their doctors to be a main source of medical information, showed lower levels of trust in medical experts [18], and used multiple information sources whose conflicting information increases anxiety [24]. Considering that perinatal women were not willing to discuss the pregnancy-related information obtained from the internet with health professionals in the previous studies [16,19,31], they will be more exposed to the risks of either applying treatment that is not evidence-based or missing the right time to initiate the appropriate medical treatment with the best chance to achieve better pregnancy and delivery outcomes.

    As evidenced by the findings of previous studies [21,35], perinatal women who use the NIMH strategy to solve obstetric problems tended to validate the online health information by communicating with health professionals. Lagan and colleagues [21] reported that almost all (96.2%) discussed the internet-based information with their partner or husband, and 70.8% discussed the information with at least one health professional, reflecting that the presence of a partner or relatives facilitates perinatal women’s communication with health professionals about their perceptions of being at risk [35]. Despite the associated support, informal help from partners, friends, and family members has disadvantages such as stigma or inappropriate support caused by their lack of relevant medical knowledge; therefore, making or changing medical decisions following suggestions provided by those with insufficient knowledge can be unhelpful or even harmful [47]. Meanwhile, those preferring to use the NFMH strategy did not make or change their medical decisions regarding obstetric problems by referring to online health information. As 70.1% of this study sample (115/164) considered accuracy and reliability to be the most important criteria in their evaluation when seeking pregnancy/delivery information, it could be inferred by our findings that the internet was considered a supplementary information source before or after contacting health professionals as a primary information source.

    Our study revealed a positive relationship between self-efficacy in perinatal health literacy and intent to consult health professionals in both models. Once the level of perinatal women’s health literacy increases, they can be more easily encouraged to identify questions they wish to ask a health professional, be involved in the decision-making process, make better decisions, and exert more control over their decisions [21]. Searching online health information before paying a visit to their health professionals and after their consultation [30] is common as a means to deal with doubts and navigate other women’s pregnancy-related decisions, which can also result in increased confusion, anxiety, and fright [48]. With the perinatal women’s concerns on the lack of time to sufficiently discuss personal medical issues with health professionals [21,22] and the professionals’ reactions to the information they shared [20], a number of previous studies found that the majority of women believed that health professionals should suggest suitable and reliable internet websites [20,21] where they could find relevant information with appropriate context and avoid erroneous or misleading claims [48]. As revealed in the obstetric problem-solving model, the OFMH strategy was positively related to PS, which reflects the implications of other studies.

    When it comes to solving mental health problems, perinatal women showed a different attitude toward online information use. Those using the NFMH strategy disclosed their intent to discuss relevant issues with mental health professionals without the precondition of enhanced self-efficacy in perinatal health literacy. Generally, psychiatrists and other mental health professionals, whose goals are to provide each patient with the opportunity to discuss problems with a sympathetic listener by helping to establish a rapport between the patient and doctor, educate and motivate the patient, and take a clinical examination of the mental state, depend on a psychiatric interview. In comparison with the significance of the direct and indirect paths from NFMH to CO (ie, NFMHPS, NFMHPSCO, and NFMHCO) in the two different medical problem-solving models shown in Figure 3 and Table 4, the findings of this study suggest that what women expected from the interaction with obstetricians or psychiatrists during their clinic visits would be different during the perinatal period. Accordingly, these findings can well explain why it is important for health professionals not to discourage their patients to share obstetric concerns and treatment suggestions retrieved from the internet.

    Limitations and Future Direction

    There are several limitations of this study that should be considered in future research. The sample size and its demographic composition limited our opportunity to examine the possible age difference, relationship with one’s doctor, number of births, and history of experienced obstetric and mental health problems because this study focused on the entire perinatal period, encompassing conception, pregnancy, childbirth, and postpartum. Given the small sample size, it would result in an insufficient statistical power, particularly as reflected by significantly low standardized path coefficients. For a more accurate estimate, a larger sample size would be required in the future studies. In terms of age distribution, the proportion of participants aged 19 to 29 years (14/164, 8.5%) was considerably less than that of those aged 30 to 39 years (134/164, 81.7%). Given the age differences of people obtaining online health information [49], the possibility that younger women may differ in how they make and change their medical decisions by using online health information compared with older women cannot be ruled out. As this study sample, enrolled from a large hospital-based sample of patients, was willing to complete an online questionnaire, the participants might be more likely to have a good rapport with their doctors or be internet-savvy patients accessing the internet more frequently for solving medical problems than the others who did not so. Finally, whether perinatal women have given birth to at least one child or had a history of obstetric and mental health problems may be associated with their intent to either share their own experiences with other mothers in trouble or trust and follow the doctor’s instructions only. Taken together, there is a need to control for these determinants in future studies.

    Conclusions

    Despite the positive mediating effects of perinatal women’s enhanced health literacy on the intent to discuss personal medical problems with health professionals, the findings from this study revealed that some women using an OIMH strategy would be more likely to change their medical decisions not by consulting health care professionals but by referring to online health information compared with other women using different medical help-seeking strategies. Particularly, this concern would be prevalent when encountering participants with obstetric problems during the perinatal period. Beyond a short consultation to quickly and authoritatively steer patients toward an obstetric doctor’s choice of action, it is recommended that obstetricians provide their patients with needed context for information found online. To fully explain this with an open mind, they need to actively develop or support information and communications technology–based health information services such as mobile apps and chatbots.

    Acknowledgments

    This research was supported by a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (grant number: HI18C0911).

    Conflicts of Interest

    None declared.

    1. Featherstone B, Broadhurst K. Engaging parents and carers with family support services: What can be learned from research on help-seeking? Child Fam Soc Work 2003 Nov;8(4):341-350. [CrossRef]
    2. Rickwood D, Deane FP, Wilson CJ, Ciarrochi J. Young people's help-seeking for mental health problems. Aust eJ Advancem Mental Health 2005 Dec;4(3):218-251. [CrossRef]
    3. Cornally N, McCarthy G. Help-seeking behaviour: a concept analysis. Int J Nurs Pract 2011 Jun;17(3):280-288. [CrossRef] [Medline]
    4. Conrad P, Stults C. The internet and the experience of illness. In: Bird CE, Conrad P, editors. Handbook of Medical Sociology, 6th Edition. Nashville: Vanderbilt University Press; 2010:179-191.
    5. Slauson-Blevins KS, McQuillan J, Greil AL. Online and in-person health-seeking for infertility. Soc Sci Med 2013 Dec;99:110-115. [CrossRef] [Medline]
    6. Morahan-Martin JM. How internet users find, evaluate, and use online health information: a cross-cultural review. Cyberpsychol Behav 2004 Oct;7(5):497-510. [CrossRef] [Medline]
    7. Fenwick J, Hauck Y, Downie J, Butt J. The childbirth expectations of a self-selected cohort of Western Australian women. Midwifery 2005 Mar;21(1):23-35. [CrossRef] [Medline]
    8. Carolan M. Health literacy and the information needs and dilemmas of first-time mothers over 35 years. J Clin Nurs 2007 Jun;16(6):1162-1172. [CrossRef] [Medline]
    9. Lagan BM, Sinclair M, Kernohan WG. What is the impact of the Internet on decision-making in pregnancy? A global study. Birth 2011 Dec;38(4):336-345. [CrossRef] [Medline]
    10. Fleming SE, Vandermause R, Shaw M. First-time mothers preparing for birthing in an electronic world: internet and mobile phone technology. J Reprod Infant Psychol 2014 Mar 19;32(3):240-253. [CrossRef]
    11. Sanders RA, Crozier K. How do informal information sources influence women's decision-making for birth? A meta-synthesis of qualitative studies. BMC Pregnancy Childbirth 2018 Jan 10;18(1):21 [FREE Full text] [CrossRef] [Medline]
    12. Dako-Gyeke P, Aikins M, Aryeetey R, Mccough L, Adongo PB. The influence of socio-cultural interpretations of pregnancy threats on health-seeking behavior among pregnant women in urban Accra, Ghana. BMC Pregnancy Childbirth 2013 Nov 19;13(1):211. [CrossRef]
    13. Bjelke M, Martinsson A, Lendahls L, Oscarsson M. Using the Internet as a source of information during pregnancy: a descriptive cross-sectional study in Sweden. Midwifery 2016 Sep;40:187-191. [CrossRef] [Medline]
    14. Seibold C. Young single women's experiences of pregnancy, adjustment, decision-making and ongoing identity construction. Midwifery 2004 Jun;20(2):171-180. [CrossRef] [Medline]
    15. Eysenbach G. The impact of the Internet on cancer outcomes. CA Cancer J Clin 2003;53(6):356-371. [Medline]
    16. Bert F, Gualano MR, Brusaferro S, De Vito E, de Waure C, La Torre G, et al. Pregnancy e-health: a multicenter Italian cross-sectional study on Internet use and decision-making among pregnant women. J Epidemiol Community Health 2013 Dec 1;67(12):1013-1018. [CrossRef] [Medline]
    17. Knoepke CE, Matlock DD. Preliminary development of an informational media use measure for patients with implanted defibrillators: toward a model of social-ecological assessment of patient education and support. Health Soc Work 2017 Nov 01;42(4):199-206 [FREE Full text] [CrossRef] [Medline]
    18. Eller N. Right from the source: trust in health providers, information sources, and vaccine hesitancy. 2017.   URL: https:/​/www.​semanticscholar.org/​paper/​Right-from-the-source%3A-Trust-in-health-providers%2C-Eller/​7f23a06a66b2fd855d6b9ac71582b612d41d0cf6 [accessed 2020-01-07]
    19. Sayakhot P, Carolan-Olah M. Internet use by pregnant women seeking pregnancy-related information: a systematic review. BMC Pregnancy Childbirth 2016;16:65 [FREE Full text] [CrossRef] [Medline]
    20. Slomian J, Bruyère O, Reginster JY, Emonts P. The internet as a source of information used by women after childbirth to meet their need for information: A web-based survey. Midwifery 2017 May;48:46-52. [CrossRef] [Medline]
    21. Lagan BM, Sinclair M, Kernohan WG. Internet use in pregnancy informs women's decision making: a web-based survey. Birth 2010 Jun;37(2):106-115. [CrossRef] [Medline]
    22. Lagan BM, Sinclair M, Kernohan WG. A Web-based survey of midwives' perceptions of women using the Internet in pregnancy: a global phenomenon. Midwifery 2011 Apr;27(2):273-281. [CrossRef] [Medline]
    23. Lim AS, Stewart K, Abramson MJ, Ryan K, George J. Asthma during pregnancy: the experiences, concerns and views of pregnant women with asthma. J Asthma 2012 Jun;49(5):474-479. [CrossRef] [Medline]
    24. Hämeen-Anttila K, Nordeng H, Kokki E, Jyrkkä J, Lupattelli A, Vainio K, et al. Multiple information sources and consequences of conflicting information about medicine use during pregnancy: a multinational Internet-based survey. J Med Internet Res 2014;16(2):e60 [FREE Full text] [CrossRef] [Medline]
    25. Chen Y, Li C, Liang J, Tsai C. Health information obtained from the internet and changes in medical decision making: questionnaire development and cross-sectional survey. J Med Internet Res 2018 Dec 12;20(2):e47 [FREE Full text] [CrossRef] [Medline]
    26. Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Prom Int 2000;15(3):259-267. [CrossRef]
    27. Smith CL, Shochet IM. The impact of mental health literacy on help-seeking intentions: results of a pilot study with first year psychology students. Int J Mental Health Promotion 2011 Jan;13(2):14-20. [CrossRef]
    28. James DC, Harville C. eHealth literacy, online help-seeking behavior, and willingness to participate in mhealth chronic disease research among African Americans, Florida, 2014-2015. Prev Chronic Dis 2016 Dec 17;13:E156 [FREE Full text] [CrossRef] [Medline]
    29. Bernhardt JM, Felter EM. Online pediatric information seeking among mothers of young children: results from a qualitative study using focus groups. J Med Internet Res 2004 Mar 1;6(1):e7 [FREE Full text] [CrossRef] [Medline]
    30. McMullan M. Patients using the Internet to obtain health information: how this affects the patient-health professional relationship. Patient Educ Couns 2006 Oct;63(1-2):24-28. [CrossRef] [Medline]
    31. Larsson M. A descriptive study of the use of the Internet by women seeking pregnancy-related information. Midwifery 2009 Feb;25(1):14-20. [CrossRef] [Medline]
    32. Gao L, Larsson M, Luo S. Internet use by Chinese women seeking pregnancy-related information. Midwifery 2013 Jul;29(7):730-735. [CrossRef] [Medline]
    33. Huberty J, Dinkel D, Beets MW, Coleman J. Describing the use of the internet for health, physical activity, and nutrition information in pregnant women. Matern Child Health J 2013 Oct;17(8):1363-1372. [CrossRef] [Medline]
    34. Song H, Cramer EM, McRoy S, May A. Information needs, seeking behaviors, and support among low-income expectant women. Women Health 2013;53(8):824-842. [CrossRef] [Medline]
    35. Rance S, McCourt C, Rayment J, Mackintosh N, Carter W, Watson K, et al. Women's safety alerts in maternity care: is speaking up enough? BMJ Qual Saf 2013 Apr;22(4):348-355. [CrossRef] [Medline]
    36. Fisher VM. Help-seeking for depression in rural women: a community portrait. 2005.   URL: https://scholarscompass.vcu.edu/etd/681/ [accessed 2020-01-07]
    37. Gholami K, Mohammadi S. Information interaction of pregnant women in Ninisite. Human Information Interaction 2015;1(4):305-318 [FREE Full text]
    38. Renkert S, Nutbeam D. Opportunities to improve maternal health literacy through antenatal education: an exploratory study. Health Promot Int 2001 Dec;16(4):381-388. [Medline]
    39. Cheung W, Ip W, Chan D. Maternal anxiety and feelings of control during labour: a study of Chinese first-time pregnant women. Midwifery 2007 Jun;23(2):123-130. [CrossRef] [Medline]
    40. Melender H. Experiences of fears associated with pregnancy and childbirth: a study of 329 pregnant women. Birth 2002 Jun;29(2):101-111. [CrossRef] [Medline]
    41. Rini CK, Dunkel-Schetter C, Wadhwa PD, Sandman CA. Psychological adaptation and birth outcomes: the role of personal resources, stress, and sociocultural context in pregnancy. Health Psychol 1999 Jul;18(4):333-345. [CrossRef] [Medline]
    42. Marsh H, Hocevar D. Application of confirmatory factor analysis to the study of self-concept: first and higher order factor models and their invariance across groups. Psychol Bull 1985;97(3):562 [FREE Full text] [CrossRef]
    43. Bentler PM. Comparative fit indexes in structural models. Psychol Bull 1990 Mar;107(2):238-246. [Medline]
    44. Bentler PM, Bonett DG. Significance tests and goodness of fit in the analysis of covariance structures. Psychol Bull 1980;88(3):588-606. [CrossRef]
    45. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Structural Equation Modeling Multidiscipl J 1999 Jan;6(1):1-55. [CrossRef]
    46. Browne M, Cudeck R. Alternative ways of assessing model fit. Sociol Methods Res 1993;154:136. [CrossRef]
    47. Griffiths KM, Crisp DA, Barney L, Reid R. Seeking help for depression from family and friends: a qualitative analysis of perceived advantages and disadvantages. BMC Psychiatry 2011 Dec 15;11:196 [FREE Full text] [CrossRef] [Medline]
    48. Romano AM. A changing landscape: implications of pregnant women's internet use for childbirth educators. J Perinat Educ 2007;16(4):18-24 [FREE Full text] [CrossRef] [Medline]
    49. Sharit J, Taha J, Berkowsky RW, Czaja SJ. Seeking and resolving complex online health information: age differences in the role of cognitive abilities. 2016 Sep 15 Presented at: Proceedings of the Human Factors and Ergonomics Society Annual Meeting; 2016; Los Angeles p. 1-5. [CrossRef]

    ‎
    CD: changing their medical decisions based on online health information
    CFI: comparative fit index
    CO: consulting others (health professionals) based on online health information
    CR: critical ratio
    KHIDI: Korea Health Industry Development Institute
    NFMH: nononline formal medical help-seeking
    NIMH: nononline informal medical help-seeking
    OFMH: online formal medical help-seeking
    OIMH: online informal medical help-seeking
    PS: promoting self-efficacy in perinatal health literacy based on online health information
    RMSEA: root mean square error of approximation
    SRMR: standardized root mean square residual
    TLI: Tucker-Lewis index

    Edited by G Eysenbach; submitted 19.04.19; peer-reviewed by KS Lee, A Bhat, C Knoepke, E Neter; comments to author 03.10.19; revised version received 28.11.19; accepted 16.12.19; published 04.03.20

    Copyright

    ©Kyungmi Chung, Hee Young Cho, Young Ran Kim, Kyungun Jhung, Hwa Seon Koo, Jin Young Park. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 04.03.2020.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.