This paper is in the following e-collection/theme issue:

Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (832) Research Instruments, Questionnaires, and Tools (576)

Published on in Vol 17, No 6 (2015): June

Using Web-Based Questionnaires and Obstetric Records to Assess General Health Characteristics Among Pregnant Women: A Validation Study

Using Web-Based Questionnaires and Obstetric Records to Assess General Health Characteristics Among Pregnant Women: A Validation Study

Using Web-Based Questionnaires and Obstetric Records to Assess General Health Characteristics Among Pregnant Women: A Validation Study

Authors of this article:

Marleen MHJ van Gelder1, 2 Author Orcid Image ;   Naomi PE Schouten1 Author Orcid Image ;   Peter JFM Merkus3, 4 Author Orcid Image ;   Chris M Verhaak5 Author Orcid Image ;   Nel Roeleveld1, 6 Author Orcid Image ;   Jolt Roukema3 Author Orcid Image
Article Authors Cited by (16) Tweetations (19) Metrics

    Original Paper

    1Department for Health Evidence, Radboud Institute for Health Sciences, Radboud university medical center, Nijmegen, Netherlands

    2Radboud REshape Innovation Center, Radboud university medical center, Nijmegen, Netherlands

    3Division of Respiratory Medicine, Department of Pediatrics, Radboudumc Amalia Children’s Hospital, Radboud university medical center, Nijmegen, Netherlands

    4Department of Pediatrics, Canisius Wilhelmina Hospital, Nijmegen, Netherlands

    5Department of Medical Psychology, Radboud university medical center, Nijmegen, Netherlands

    6Department of Pediatrics, Radboudumc Amalia Children’s Hospital, Radboud university medical center, Nijmegen, Netherlands

    Corresponding Author:

    Marleen MHJ van Gelder, PhD

    Department for Health Evidence

    Radboud Institute for Health Sciences

    Radboud university medical center

    P.O. Box 9101

    Nijmegen, 6500 HB

    Netherlands

    Phone: 31 24 3666126

    Fax:31 24 3613505

    Email: Marleen.vanGelder@radboudumc.nl


    Background: Self-reported medical history information is included in many studies. However, data on the validity of Web-based questionnaires assessing medical history are scarce. If proven to be valid, Web-based questionnaires may provide researchers with an efficient means to collect data on this parameter in large populations.

    Objective: The aim of this study was to assess the validity of a Web-based questionnaire on chronic medical conditions, allergies, and blood pressure readings against obstetric records and data from general practitioners.

    Methods: Self-reported questionnaire data were compared with obstetric records for 519 pregnant women participating in the Dutch PRegnancy and Infant DEvelopment (PRIDE) Study from July 2011 through November 2012. These women completed Web-based questionnaires around their first prenatal care visit and in gestational weeks 17 and 34. We calculated kappa statistics (κ) and the observed proportions of positive and negative agreement between the baseline questionnaire and obstetric records for chronic conditions and allergies. In case of inconsistencies between these 2 data sources, medical records from the woman’s general practitioner were consulted as the reference standard. For systolic and diastolic blood pressure, intraclass correlation coefficients (ICCs) were calculated for multiple data points.

    Results: Agreement between the baseline questionnaire and the obstetric record was substantial (κ=.61) for any chronic condition and moderate for any allergy (κ=.51). For specific conditions, we found high observed proportions of negative agreement (range 0.88-1.00) and on average moderate observed proportions of positive agreement with a wide range (range 0.19-0.90). Using the reference standard, the sensitivity of the Web-based questionnaire for chronic conditions and allergies was comparable to or even better than the sensitivity of the obstetric records, in particular for migraine (0.90 vs 0.40, P=.02), asthma (0.86 vs 0.61, P=.04), inhalation allergies (0.92 vs 0.74, P=.003), hay fever (0.90 vs 0.64, P=.001), and allergies to animals (0.89 vs 0.53, P=.01). However, some overreporting of allergies was observed in the questionnaire and for some nonsomatic conditions sensitivity of both measurement instruments was low. The ICCs for blood pressure readings ranged between 0.72 and 0.92 with very small mean differences between the 2 methods of data collection.

    Conclusions: Web-based questionnaires can be used to validly collect data on many chronic disorders, allergies, and blood pressure readings among pregnant women.

    J Med Internet Res 2015;17(6):e149

    doi:10.2196/jmir.3847

    Keywords

    questionnaires; medical records; validation studies; pregnancy; chronic disease; allergens; blood pressure; Internet; PRIDE Study 


    Self-reported methods of data collection are often applied in large-scale medical or biomedical studies for efficiency reasons. In these studies, it may not be feasible to conduct clinical measurements on all participants. Therefore, paper-and-pencil questionnaires or telephone interviews were traditionally used to gather information on the study variables. Nowadays, these modes of data collection are increasingly being substituted by Web-based questionnaires. However, knowledge on the validity of data collected with Web-based questionnaires is limited [1], although the quality of the data on a number of traditional epidemiologic risk factors, including body weight [2-4], smoking [5], alcohol consumption [6], and energy and macronutrient intake [7,8], is reported to be high. Medical history is included as an exposure or potential confounding factor in many studies and Web-based questionnaires may be an efficient way to collect these data in large samples of participants, if proven to be valid.

    Most validation studies on medical history collected through self-reported methods has focused on chronic conditions, in particular cardiovascular diseases [9-15], diabetes [10,12-16], cancer [11,17,18], and asthma [10,13,14,19,20]. Agreement between self-reports and medical records differed among these studies and was affected by study methodology, target population, condition of interest, and the statistical analyses. In general, agreement was good for conditions that have clear diagnostic criteria, but it was low to moderate for conditions that are less serious or more complex to diagnose. Accordingly, discordance between questionnaires and biochemical measures or patch testing for allergic conditions or atopy is substantial [21-23]. Data on the validity of self-report on the results of common measurements taken during health care visits, such as blood pressure readings and hemoglobin levels, are very limited.

    To the best of our knowledge, only Landkroon et al [24] compared data on medical history from a Web-based questionnaire with a “reference standard,” but this study was too small (N=106) to produce robust estimates for levels of agreement. Therefore, the aim of this study was to assess the validity of a Web-based questionnaire on chronic conditions, allergies, and blood pressure readings among pregnant women by comparing the questionnaire data to obstetric records and data from general practitioners (GPs).


    Setting

    The Dutch prenatal care system is unique in the Western world. In the Netherlands, midwives are qualified to provide full prenatal care to all women with uncomplicated pregnancies and deliveries. The first prenatal care visit, which may be scheduled without referral of a general practitioner, usually takes place in gestational weeks 8 to 10 and frequent contacts are scheduled throughout pregnancy. Women are referred to a secondary or tertiary midwife or gynecologist in case of risk factors or complications. In 2013, 85% of pregnant women started their prenatal care in a primary care setting [25].

    Study Population

    We used data from the PRegnancy and Infant DEvelopment (PRIDE) Study, an ongoing, prospective cohort study that enrolls Dutch women early in pregnancy. The PRIDE Study started enrollment in July 2011 in the Nijmegen region and aims at including more than 150,000 pregnancies to study a broad range of research questions pertaining to maternal and child health. Details on the study design are described elsewhere [26]. Briefly, pregnant women aged 18 years and older were invited to participate in the PRIDE Study by their midwife or gynecologist just before or during their first prenatal care visit. They were asked to complete Web-based questionnaires at baseline, in gestational weeks 17 (questionnaire 2) and 34 (questionnaire 3), as well as 2 and 6 months after the estimated date of delivery. The baseline questionnaire was completed between weeks 6 and 16 of gestation. Researchers from various medical disciplines selected, modified, and tailored existing, validated paper-based questionnaires or parts thereof to fit our Web-based application. Paper-based questionnaires were available for women who could not or did not want to participate through the Internet (n=1; excluded from this study). Questions were asked on demographic factors, reproductive history, maternal health, lifestyle factors, and occupational exposures. Furthermore, consent was asked for review of medical records to enrich the PRIDE Study database with detailed clinical information.

    Data Collection

    Through the baseline questionnaire, data on medical history were collected. Women were asked gateway questions to assess chronic conditions (“Do you have a chronic or long-term illness that was diagnosed by a medical doctor” followed by some examples of chronic conditions) and allergies (“Do you have an allergy or eczema?”). These questions were followed by multiple-choice questions with blank options to specify the chronic condition or allergy among those who answered positively to the relevant gateway question. Chronic conditions reported in other parts of the baseline questionnaire (eg, as causes for subfertility or as indications for medication use) were included in the analysis as well. In each prenatal questionnaire, we asked for the date of the most recent prenatal care visit, whether blood pressure was measured during this visit, and if so, for the systolic and diastolic blood pressure readings in mm Hg. A screenshot of the relevant parts of the questionnaires is provided in Multimedia Appendix 1.

    A pretested, standardized case report form (CRF) was used to abstract data from the obstetric records of women who gave consent for medical record review. For logistical reasons, obstetric records were only reviewed in participating study centers in the Nijmegen region (7 midwifery practices and 1 academic hospital). Using the CRF, 2 medically trained abstracters collected data from the obstetric records on medical history, including chronic conditions, allergies, and pregnancy history, the pregnancy itself, anthropometrical measures including blood pressure taken during pregnancy, and pregnancy outcome, not all of which were included in this validation study.

    Preexisting medical conditions are self-reported by the pregnant woman during the first prenatal care visit and are usually only recorded in the obstetric record by the prenatal care provider if deemed important for the course of pregnancy or the delivery [27]. As a consequence, obstetric records may not be a suitable reference standard for self-reported chronic conditions and allergies. Therefore, information on the diagnosis of chronic conditions and allergies was obtained from the woman’s GP in case of inconsistencies between the questionnaire and the obstetric record for reasons of efficiency.

    Chronic conditions were classified and coded according to the World Health Organization’s International Classification of Diseases, Tenth Revision [28]. Allergies were ordered into 6 mutually exclusive categories: (1) inhalation allergies (hay fever, allergies to animals, and house dust mite allergy), (2) food allergies, (3) allergic contact dermatitis (allergies to metal, fragrance hypersensitivity, plaster allergy, and latex allergy), (4) insect sting allergy, (5) medication allergies, and (6) other allergies.

    Statistical Analysis

    Only PRIDE Study participants with complete information on chronic conditions, allergies, and blood pressure during the most recent prenatal care visit in the baseline questionnaire who gave consent to review their medical records were included in this validation study. For chronic conditions and allergies with at least 5 cases in either the questionnaire or the obstetrical record, we calculated kappa statistics (κ) to quantify agreement between the baseline questionnaire and the obstetric record regarding chronic conditions and allergies. We also calculated the observed proportions of positive and negative agreement (ppos and pneg, respectively) because kappa is strongly affected by imbalances in marginal totals (ie, a low kappa despite a high level of agreement) [29]. The calculation of ppos and pneg is shown in Figure 1 [30].

    To determine which method of data collection was most valid to collect information on chronic conditions and allergies among pregnant women, sensitivity and specificity were calculated with GP data until the date of completion of the baseline questionnaire as our reference standard. When GP data were unavailable, pharmacy records were screened for diagnoses of chronic conditions or allergies and for medication dispensed that was indicative for chronic conditions or allergies. In addition to the discordant questionnaire–obstetric record pairs, women with positive scores on both the Web-based questionnaire and the obstetric record were included in these calculations as true positives. Likewise, women with negative scores on both methods were included as true negatives. We assessed potential differences in sensitivity and specificity between the questionnaires and the obstetric records using chi-square tests.

    For the validity analyses regarding blood pressure readings, only women with an exact match between the date of the most recent prenatal care visit reported in any of the prenatal questionnaires and a visit date recorded in the obstetric record were included to be certain that both data sources referred to the same measurement. Intraclass correlation coefficients (ICCs) with 95% confidence intervals (CIs) for systolic blood pressure (SBP) and diastolic blood pressure (DBP) were calculated using 2-way mixed effects models (single measure). To assess absolute agreement and potential differences in bias within the SBP and DBP range, we plotted the difference in blood pressure readings between the questionnaire and the obstetric record (y-axis) against the mean of the 2 methods of data collection (x-axis) according to the Bland-Altman technique [31]. In secondary analyses, we included all women who reported the most recent prenatal care visit date in the questionnaire within 5 days of a visit date recorded in the obstetric record. All statistical analyses were performed using IBM SPSS version 20 (IBM Corp, Armonk, NY, USA), except for ppos and pneg, which were calculated in Microsoft Office Excel 2007 (Microsoft Corp, Redmond, WA, USA).

    ‎
    Figure 1. Calculation of positive and negative agreement between two tests.
    View this figure

    Women enrolled in the PRIDE Study between July 2011 through November 2012 were eligible for this study (N=725). The overall participation rate in the PRIDE Study was 42.90% (725/1690) during this time period. Figure 2 shows the flow of participants. Of the 725 women enrolled during the study period, 22 (3.0%) only completed a few sections of the baseline questionnaire, mostly because of technical issues in the first weeks of enrollment. Among those with complete baseline questionnaires, 24.8% (174/703) did not give consent for medical record review. Furthermore, 10 women were excluded because their obstetric records were not available (n=9) or they participated with multiple pregnancies in the PRIDE Study (n=1). Therefore, 519 women were included in this validation study. Compared with the women who did not give consent to obtain medical records, women participating in this validation study were more likely to have a lower level of education (P=.03) and to be obese (P=.06; Table 1). Furthermore, women who did not give consent for medical record review were more likely to have completed the baseline questionnaire before their first prenatal care visit compared to women included in the validation study (P=.02). We did not observe substantial differences in maternal age, country of birth, gravidity, and gestational age at inclusion between these 2 groups. Regarding the blood pressure readings, follow-up information was not available for all participants for several reasons: (1) they did not reach the gestational week for administration of questionnaire 2 or 3 yet at the date of obstetric record review; (2) they had a miscarriage, stillbirth, termination of pregnancy (TOP), or very preterm birth; or (3) they skipped questionnaire 2 or 3, were lost to follow-up, or changed prenatal care provider resulting in incomplete obstetric records.

    ‎
    Figure 2. Flow chart of study participants.
    View this figure
    Table 1. Characteristics of PRIDE Study participants included in this validation study and participants who did not give consent for review of medical records.
    CharacteristicParticipants in validation study, n (%) (N=519)aNo consent for medical record review, n (%) (N=174)aPb
    Maternal age at inclusion (years)

    		.45

    		<2515 (2.9)2 (1.1)

    		25-29172 (33.1)52 (29.9)

    		30-34238 (45.9)88 (50.6)

    		≥3594 (18.1)32 (18.4)
    Maternal country of birth

    		.20

    		Netherlands483 (93.1)154 (88.5)

    		Other31 (6.0)15 (8.6)
    Maternal level of educationc

    		.03

    		Low/intermediate107 (20.6)22 (12.6)

    		High407 (78.4)147 (84.5)
    Gravidity

    		.62

    		0232 (44.7)74 (42.5)

    		≥1287 (55.3)100 (57.5)
    BMI before pregnancyd

    		.06

    		Underweight (<18.5 kg/m2)15 (2.9)2 (1.1)

    		Normal (18.5-24.9 kg/m2)336 (64.7)122 (70.1)

    		Overweight (25.0-29.9 kg/m2)90 (17.3)27 (15.5)

    		Obese (≥30 kg/m2)33 (6.4)3 (1.7)

    		Height/weight unknown45 (8.7)20 (11.5)
    Gestational age at inclusion (weeks)

    		.17

    		<896 (18.5)41 (23.6)

    		8-10178 (34.3)48 (27.6)

    		>10245 (47.2)85 (48.9)
    Timing of baseline questionnaire

    		.02

    		Completed before first prenatal care visit131 (25.2)60 (34.5)

    		Completed after first prenatal care visit388 (74.8)114 (65.5)

    a Numbers may not add up to total group size due to missing values.

    b Difference between the 2 groups using chi-square tests.

    c High level of education: completed higher vocational education or university.

    d Body mass index (BMI) derived from self-reported height and weight.

    Of the 519 participants, 118 (22.7%) women reported having a chronic condition in the baseline questionnaire, whereas chronic conditions were recorded in the obstetric records of 105 (20.2%) women. Overall, agreement between the Web-based questionnaire and the obstetric record was substantial for any chronic condition (κ=.61; Table 2) with a higher pneg (0.92) than ppos (0.69). Level of agreement differed between the specific chronic conditions with relatively high levels of agreement for endocrine, nutritional, and metabolic diseases (κ=.72) and in particular for thyroid disease (κ=.90), epilepsy (κ=.89), and diseases of the genitourinary tract (κ=.72). However, for a number of conditions, including migraine (κ=.30), diseases of the circulatory system (κ=.25), and irritable bowel syndrome (κ=.39), agreement between the questionnaire and the obstetric record was poor. For all specific conditions, the pneg was high (range 0.98-1.00), but the ppos followed a pattern comparable to the kappa statistic.

    Table 2. Agreement between data from the Web-based questionnaire and obstetric record for chronic conditions (n=519).
    Chronic conditionQuestionnaire positive, nQuestionnaire negative, nκpposapnegb

    		Record positiveRecord negativeRecord positiveRecord negative


    Any chronic condition774128373.610.690.92
    Diseases of the blood and blood-forming organs200517———
    Thalassemia100518———
    Immunodeficiency100518———
    Endocrine, nutritional, and metabolic diseases1556493.720.730.99
    Thyroid disease911508.900.901.00
    Polycystic ovarian syndrome535506.550.560.99
    Hypercholesterolemia010518———
    Periodic fever syndrome100518———
    Mental and behavioral disorders764502.570.580.99
    Depression/anxiety554505.520.530.99
    Posttraumatic stress disorder110517———
    ADD/ADHDc110517———
    Diseases of the nervous system12174486.510.530.98
    Multiple sclerosis100518———
    Epilepsy401514.890.891.00
    Migraine4171497.300.310.98
    Tension-type headache101517———
    Chronic fatigue syndrome201516———
    Diseases of the circulatory system115512.250.250.99
    Hypertension003516———
    Cardiac arrhythmia011517———
    Raynaud syndrome101517———
    Diseases of the respiratory system19175478.610.630.98
    Asthma19175478.610.630.98
    Diseases of the digestive system846501.610.620.99
    Crohn disease201516———
    Ulcerative colitis300516———
    Irritable bowel syndrome345507.390.400.99
    Diseases of the skin and subcutaneous tissue471507.490.500.99
    Psoriasis450510.610.621.00
    Rosacea021516———
    Diseases of the musculoskeletal system and connective tissue944502.680.690.99
    Rheumatoid arthritis220515———
    Sjögren syndrome100518———
    Ankylosing spondylitis101517———
    Hypermobility113514———
    Fibromyalgia221514———
    Complex regional pain syndrome100518———
    Diseases of the genitourinary tract412512.720.731.00
    Endometriosis111516———
    Lichen sclerosis301515———

    a Observed proportion of positive agreement.

    b Observed proportion of negative agreement.

    c ADD: attention deficit disorder; ADHD: attention deficit hyperactivity disorder.

    Allergies were reported by 229 of 519 (44.1%) women in the baseline questionnaire and recorded in the obstetric record of 168 (32.4%) women. In Table 3, agreement between the Web-based questionnaire and the obstetric record is shown for the mutually exclusive groups of allergies and selected specific allergies. For any allergy, agreement between the questionnaire and the obstetric record was moderate (κ=.51) with a ppos and pneg of 0.70 and 0.81, respectively. The kappa values for the groups of allergies ranged between 0.21 (insect sting allergy) and 0.66 (drug allergies) and between 0.33 (fragrance hypersensitivity) and 0.73 (latex allergy) for the specific types of allergies. House dust mite allergy, latex allergy, and drug allergies were more often reported in the obstetric record than in the questionnaire. Again, the pneg (range 0.81-1.00) was higher than the ppos (range 0.19-0.73) for all groups of allergies or specific allergies included.

    Table 3. Agreement between data from a Web-based questionnaire and obstetric record for allergies (n=519).
    AllergyQuestionnaire positive, nQuestionnaire negative, nκpposapnegb

    		Record positiveRecord negativeRecord positiveRecord negative


    Any allergy1389130260.510.700.81
    Inhalation allergies808310346.530.630.88
    Hay fever53533410.600.650.94
    Allergies to animals21457446.400.450.94
    House dust mite allergy151522467.410.450.96
    Food allergies15391464.400.430.96
    Allergic contact dermatitis337311402.360.440.91
    Allergies to metal11373468.330.350.96
    Fragrance hypersensitivity6510462.170.190.95
    Plaster allergy9167487.420.440.98
    Latex allergy403512.730.731.00
    Insect sting allergy3201495.210.220.98
    Drug allergies26915469.660.680.98
    Other allergies230514.570.571.00

    a Observed proportion of positive agreement.

    b Observed proportion of negative agreement.

    Regarding the 254 women with an inconsistency between the Web-based questionnaire and the obstetric record for chronic conditions or allergies, complete GP data were obtained for 194 (76.4%) women; the GP was unknown for 12 women, 21 women were not registered with the GP whose name was provided, the GP did not respond to our multiple data requests for 25 women, and GP records were incomplete for 2 women. For 7 women lacking GP data, the diagnosis of a chronic disorder was ascertained from their pharmacy records. Generally, sensitivity was better for the Web-based questionnaire than for the obstetric record when compared to GP data (Table 4), specifically for migraine (0.90 vs 0.40, P=.02), asthma (0.86 vs 0.61, P=.04), any allergy (0.96 vs 0.85, P=.007), inhalation allergies (0.92 vs 0.74, P=.003), hay fever (0.90 vs 0.64, P=.001), and allergies to animals (0.89 vs 0.53, P=.01). For a number of chronic conditions, including mental and behavioral disorders, depression/anxiety, and irritable bowel syndrome, sensitivity of both measurement instruments was low. Overall, specificity of the Web-based questionnaire and the obstetric record was high. However, specificity of the questionnaire was slightly lower than specificity of the obstetric record for a number of (groups of) allergies, including any allergy (0.74 vs 0.85, P=.009), inhalation allergies (0.83 vs 0.93, P<.001), hay fever (0.91 vs 0.96, P=.001), allergies to animals (0.91 vs 0.97, P<.001), food allergies (0.92 vs 0.98, P<.001), allergic contact dermatitis (0.84 vs 0.94, P<.001), allergies to metal (0.94 vs 0.99, P<.001), and fragrance hypersensitivity (0.91 vs 0.99, P<.001).

    Table 4. Validity comparisons of chronic conditions and allergies among pregnant women: Web-based questionnaires and obstetric records compared to GP records.
    Condition or allergynSensitivitySpecificity


    		QuestionnaireRecordPaQuestionnaireRecordPa
    Any chronic condition4960.830.74.140.930.92.79
    Endocrine, nutritional, and metabolic diseases5150.760.70.731.000.99.41
    Thyroid disease5180.770.69.661.001.00>.99
    Polycystic ovarian syndrome5160.670.83.521.000.99.41
    Mental and behavioral disorders5160.450.27.390.990.99.78
    Depression/anxiety5160.500.30.370.990.99.48
    Diseases of the nervous system5130.850.65.150.991.00.09
    Epilepsy5190.801.00.321.001.00>.99
    Migraine5140.900.40.020.991.00.03
    Diseases of the respiratory system5110.860.61.040.990.99.76
    Asthma5110.860.61.040.990.99.76
    Diseases of the digestive system5170.580.67.680.990.99.74
    Irritable bowel syndrome5170.330.33>.990.990.99.74
    Diseases of the skin and subcutaneous tissue5190.690.38.121.001.00.16
    Psoriasis5190.800.40.081.001.00.32
    Diseases of the musculoskeletal system and connective tissue5151.000.82.151.001.00.16
    Diseases of the genitourinary tract5180.801.00.321.001.00.32
    Any allergy4940.960.85.0070.740.85.009
    Inhalation allergies4960.920.74.0030.830.93<.001
    Hay fever5080.900.64.0010.910.96.001
    Allergies to animals5090.890.53.010.910.97<.001
    House dust mite allergy5120.580.65.580.980.97.44
    Food allergies5111.000.86.320.920.98<.001
    Allergic contact dermatitis5021.001.00>.990.840.94<.001
    Allergies to metal5081.001.00>.990.940.99<.001
    Fragrance hypersensitivity5071.001.00>.990.910.99<.001
    Plaster allergy5111.001.00>.990.960.97.36
    Drug allergies5150.840.95.300.970.96.32

    a Difference between the 2 modes of data collection using chi-square tests.

    For 4 chronic conditions, additional self-reports were identified in the questions about causes of subfertility that preceded the chronic condition question (polycystic ovarian syndrome [n=7] and endometriosis [n=2]) and through medication use (rosacea [n=2] and lichen sclerosis [n=1]). When these women were considered as not having reported these chronic conditions, agreement between the Web-based questionnaire and the obstetric record decreased, except for skin diseases. Furthermore, it decreased the sensitivity of the questionnaire, especially for endocrine diseases (0.67), polycystic ovarian syndrome (no true positive subjects), and diseases of the genitourinary tract (0.67).

    Analyses on the validity of the Web-based questionnaires for blood pressure readings could not be conducted on the complete study sample. At baseline, 123 of 519 (23.7%) women did not have a prenatal care visit yet and, therefore, no valid blood pressure measurement (Table 5). Among women with a prenatal care visit, no match on visit date was established for 91 of 396 (23.0%), 65 of 423 (15.4%), and 32 of 295 (10.8%) women for the baseline questionnaire, questionnaire 2, and questionnaire 3, respectively. Furthermore, a substantial proportion of women whose blood pressure was measured could not remember the blood pressure readings (baseline questionnaire: 27.9%, 76/272; questionnaire 2: 28.4%, 93/328; questionnaire 3: 19.1%, 50/262). Of the women included at baseline and eligible for the reliability analyses of the follow-up questionnaires, 78.6% (121/154) and 84.6% (88/104) were included for questionnaires 2 and 3, respectively. Out of the 142 women included for questionnaire 2 and eligible for the analysis of questionnaire 3, 128 (90.1%) were included for questionnaire 3.

    Table 5. Validity analyses comparing Web-based questionnaires and obstetric records for systolic and diastolic blood pressure readings: sample description and intraclass correlation coefficients.
    CharacteristicBaseline questionnaire (n=519)Questionnaire 2 (n=423)Questionnaire 3 (n=295)
    Did not have prenatal care visit yet, n12300
    No match date prenatal care visit, n916532
    Questionnaire: blood pressure not measured, n33301
    Questionnaire: blood pressure unknown, n769350
    Obstetric record: blood pressure not recorded, n11121
    Included in validity analyses, n185223211
    Blood pressure, ICC (95% CI)



    		Systolic0.72 (0.65-0.79)0.92 (0.89-0.94)0.90 (0.88-0.93)

    		Diastolic0.79 (0.73-0.84)0.91 (0.88-0.93)0.89 (0.86-0.91)

    At baseline, the ICCs for SBP and DBP were 0.72 (95% CI 0.65-0.79) and 0.79 (95% CI 0.73-0.84), respectively. In the follow-up questionnaires, ICCs were substantially higher, ranging between 0.89 (95% CI 0.86-0.91; DBP in questionnaire 3) and 0.92 (95% CI 0.89-0.94; SBP in questionnaire 2). The Bland-Altman plots (Figure 3) also showed good agreement between the 2 methods of data collection with very small mean differences, ranging between 1.26 mm Hg (SD 7.72) for SBP in the baseline questionnaire and –0.04 (SD 4.09) for DBP in questionnaire 3. No trends in bias within the SBP and DBP ranges were observed. The secondary analyses, in which the date of the prenatal care visit was allowed to differ up to 5 days between the questionnaire and the obstetric record, yielded similar results (data not shown).

    ‎
    Figure 3. Bland-Altman plots showing the differences in reported systolic blood pressure (SBP) and diastolic blood pressure (DBP) between the 3 Web-based questionnaires and the obstetric record plotted against the mean of the 2 methods of data collection. Each data point shows one participant. The short dashed line shows the mean difference. The long dashed lines show the 95% limits of agreement (mean difference ±2 SD).
    View this figure

    Principal Findings

    Web-based questionnaires are increasingly being used as a method of data collection in medical research. The results from the present study show that data on many chronic conditions and allergies can be validly collected among pregnant women using Web-based questionnaires with sensitivities comparable to or even higher than obstetric records. However, some overreporting of allergies was observed and absence of disease was more accurately reported than presence of disease. In addition, pregnant women were able to reliably recall blood pressure readings from the most recent prenatal care visit, especially in the follow-up questionnaires, but a substantial proportion of women could not remember their blood pressure readings at all.

    Strengths and Limitations

    In addition to the relatively large sample size, the use of GP records as a reference standard to validate the Web-based questionnaire and obstetric records for chronic conditions and allergies is a major strength of this study. In the Netherlands, inhabitants are obligatory listed with one GP, who coordinates access to specialized care and always receives all relevant medical information about the patient [32]. Therefore, GP records should contain the most complete information, although inaccuracies in registration of diagnoses cannot be excluded. Other strengths of this validation study include the high consent rate (75.2%) to review medical records, the high retrieval rate of obstetric and GP records (98.3% and 76.4%, respectively), and the high willingness of PRIDE Study participants to complete questionnaires through the Internet despite the study’s mixed-mode design.

    Women participating in the PRIDE Study represent a highly educated population, potentially limiting the generalizability of our results. However, women included in the validation study had a lower level of education compared to women who did not give consent for review of medical records. Previous studies on the association between maternal level of education and recall sensitivity of pregnancy-related events showed inconsistent results [33-36], indicating that imbalances in this baseline characteristic may or may not be a major threat to external validity.

    Validity could not be determined reliably for a number of specific chronic conditions due to their low prevalence rates in our study population or in strata based on baseline characteristics. However, it was not feasible to increase the size of the study population because medical record abstraction is a labor-intensive process. Moreover, during the time frame of this study, only one secondary/tertiary care facility participated in the PRIDE Study. Women with certain medical conditions, including preexisting hypertension or diabetes and rheumatoid arthritis, are often referred to these facilities for prenatal care in the Netherlands. Reassuringly, only a small proportion of pregnant women (15%) start prenatal care in a secondary or tertiary care setting, mainly because of complications in a previous pregnancy [25].

    Comparison With Prior Work

    For many chronic conditions that were included in our analyses, data on the validity of self-report are scarce due to differences in study populations between this study among pregnant women and previous studies, which often selected an older population with higher prevalences of cardiovascular diseases, diabetes, and cancer. However, the general pattern of a better agreement for chronic conditions that have clear diagnostic criteria than for conditions that are less well-defined observed previously [11,37,38] was also visible in our study. We observed high sensitivities and specificities for somatic diseases, but low levels of agreement for a number of nonsomatic diseases, including mental and behavioral disorders and irritable bowel syndrome. This was not only the case for data from the Web-based questionnaire, but also for data from the obstetric records. Possible causes for this variability include poor communication between the patient and the health care provider, limited health literacy of the patient, or self-diagnosis in the absence of a satisfactory medical explanation for the symptoms [39].

    Surprisingly, sensitivity of the Web-based questionnaire was substantially higher for asthma (0.86) and migraine (0.90) compared to the obstetric record, whereas the specificities were comparable. The traditional self-reported modes of data collection have a sensitivity ranging between 0.55 and 0.95 (median 0.72) for asthma [10,13,14,19,40] and between 0.35 and 0.67 (median 0.51) for migraine [41-43], suggesting that Web-based questionnaires might be more suitable for detecting subjects with these conditions in epidemiologic studies than paper-based questionnaires, interviews, and obstetric records. However, future studies should confirm these findings, also taking into account the manner in which the questions about these conditions are posed.

    With regard to allergies, the Web-based questionnaire also seemed to be more sensitive than the obstetric record, but at the expense of its specificity indicating that overreporting occurs with the use of the Web-based questionnaire and underreporting is present when using obstetric records. However, participants with allergic symptoms who manage their symptoms with over-the-counter medication may not be registered as allergic in GP records, resulting in a lower specificity (increased number of false positives). Therefore, skin-prick tests or serum-specific immunoglobulin E levels may be a more appropriate reference standard. In comparison with previous studies in different populations [20-23], allergies were somewhat more accurately reported in our Web-based questionnaire compared to the other self-reported modes of data collection.

    Research interests in changes in blood pressure over time in relation to disease outcomes is growing (eg, [44,45]), but obtaining data on individual blood pressure readings may be challenging. Alonso et al [46] observed a low correlation between self-reported and directly observed information on SBP and DBP among 127 university graduates with an ICC of 0.35 (95% CI 0.09-0.55 and 95% CI 0.16-0.51, respectively). We are not aware of other studies reporting on the validity of self-reported blood pressure readings. In our longitudinal study, we observed a learning effect; the ICC for SBP and DBP was higher for the follow-up questionnaires than for the baseline questionnaire. Once women reported a blood pressure reading, they were very likely to report blood pressure readings in follow-up questionnaires as well. In addition, the proportion of women who could not remember their blood pressure readings decreased. As a future alternative to self-reports of blood pressure measurements conducted in health care settings, home blood pressure telemonitoring may be used to collect data on blood pressure changes over time. In addition, dedicated applications may be developed in which pregnant women could record their blood pressure readings directly after every prenatal care visit.

    Conclusions

    We showed that Web-based questionnaires can validly collect data on many chronic disorders, including asthma, migraine, and thyroid disease, and also allergies among pregnant women with equal or better data quality compared to obstetric records. Although a substantial proportion of women could not remember their blood pressure readings, pregnant women who did recall the readings, recalled them well. This indicates that accurate data on general health characteristics may be collected using Web-based questionnaires in this population.

    Acknowledgments

    This work was supported by a grant from the Lung Foundation Netherlands and grants to the PRIDE Study from the Netherlands Organisation for Health Research and Development (ZonMw).

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Screenshots of the Web-based questionnaire.

    PDF File (Adobe PDF File), 289KB
    1. Van Gelder MMHJ, Pijpe A. E-epidemiology: a comprehensive update. OA Epidemiology 2013 Jun;1(1):5. [CrossRef]
    2. Bonn SE, Trolle Lagerros Y, Bälter K. How valid are Web-based self-reports of weight? J Med Internet Res 2013 Apr;15(4):e52 [FREE Full text] [CrossRef] [Medline]
    3. Lassale C, Péneau S, Touvier M, Julia C, Galan P, Hercberg S, et al. Validity of web-based self-reported weight and height: results of the Nutrinet-Santé study. J Med Internet Res 2013 Aug;15(8):e152 [FREE Full text] [CrossRef] [Medline]
    4. West R, Gilsenan A, Coste F, Zhou X, Brouard R, Nonnemaker J, et al. The ATTEMPT cohort: a multi-national longitudinal study of predictors, patterns and consequences of smoking cessation; introduction and evaluation of internet recruitment and data collection methods. Addiction 2006 Sep;101(9):1352-1361. [CrossRef] [Medline]
    5. Brigham J, Lessov-Schlaggar CN, Javitz HS, Krasnow RE, McElroy M, Swan GE. Test-retest reliability of web-based retrospective self-report of tobacco exposure and risk. J Med Internet Res 2009 Mar;11(3):e35 [FREE Full text] [CrossRef] [Medline]
    6. Graham AL, Papandonatos GD, Bock BC, Cobb NK, Baskin-Sommers A, Niaura R, et al. Internet- vs. telephone-administered questionnaires in a randomized trial of smoking cessation. Nicotine Tob Res 2006 Dec;8 Suppl 1:S49-S57 [FREE Full text] [Medline]
    7. Christensen SE, Möller E, Bonn SE, Ploner A, Wright A, Sjölander A, et al. Two new meal- and web-based interactive food frequency questionnaires: validation of energy and macronutrient intake. J Med Internet Res 2013 Jun;15(6):e109 [FREE Full text] [CrossRef] [Medline]
    8. Labonté MÈ, Cyr A, Baril-Gravel L, Royer MM, Lamarche B. Validity and reproducibility of a web-based, self-administered food frequency questionnaire. Eur J Clin Nutr 2012 Feb;66(2):166-173. [CrossRef] [Medline]
    9. St Sauver JL, Hagen PT, Cha SS, Bagniewski SM, Mandrekar JN, Curoe AM, et al. Agreement between patient reports of cardiovascular disease and patient medical records. Mayo Clin Proc 2005 Feb;80(2):203-210. [Medline]
    10. Heliövaara M, Aromaa A, Klaukka T, Knekt P, Joukamaa M, Impivaara O. Reliability and validity of interview data on chronic diseases. The Mini-Finland Health Survey. J Clin Epidemiol 1993 Feb;46(2):181-191. [Medline]
    11. Colditz GA, Martin P, Stampfer MJ, Willett WC, Sampson L, Rosner B, et al. Validation of questionnaire information on risk factors and disease outcomes in a prospective cohort study of women. Am J Epidemiol 1986 May;123(5):894-900. [Medline]
    12. Alves E, Lunet N, Correia S, Morais V, Azevedo A, Barros H. Medical record review to recover missing data in a Portuguese birth cohort: agreement with self-reported data collected by questionnaire and inter-rater variability. Gac Sanit 2011 Jun;25(3):211-219 [FREE Full text] [CrossRef] [Medline]
    13. Muggah E, Graves E, Bennett C, Manuel DG. Ascertainment of chronic diseases using population health data: a comparison of health administrative data and patient self-report. BMC Public Health 2013 Jan;13:16 [FREE Full text] [CrossRef] [Medline]
    14. Oksanen T, Kivimäki M, Pentti J, Virtanen M, Klaukka T, Vahtera J. Self-report as an indicator of incident disease. Ann Epidemiol 2010 Jul;20(7):547-554. [CrossRef] [Medline]
    15. Okura Y, Urban LH, Mahoney DW, Jacobsen SJ, Rodeheffer RJ. Agreement between self-report questionnaires and medical record data was substantial for diabetes, hypertension, myocardial infarction and stroke but not for heart failure. J Clin Epidemiol 2004 Oct;57(10):1096-1103. [CrossRef] [Medline]
    16. Ramadhani TA, Canfield MA, Waller DK, Case AP. Medical records vs. interview responses: a comparative analysis of selected variables for linked birth defect cases. Birth Defects Res A Clin Mol Teratol 2004 Sep;70(9):592-596. [CrossRef] [Medline]
    17. Bergmann MM, Calle EE, Mervis CA, Miracle-McMahill HL, Thun MJ, Heath CW. Validity of self-reported cancers in a prospective cohort study in comparison with data from state cancer registries. Am J Epidemiol 1998 Mar;147(6):556-562 [FREE Full text] [Medline]
    18. Desai MM, Bruce ML, Desai RA, Druss BG. Validity of self-reported cancer history: a comparison of health interview data and cancer registry records. Am J Epidemiol 2001 Feb;153(3):299-306 [FREE Full text] [Medline]
    19. Weakley J, Webber MP, Ye F, Zeig-Owens R, Cohen HW, Hall CB, et al. Agreement between obstructive airways disease diagnoses from self-report questionnaires and medical records. Prev Med 2013 Jul;57(1):38-42. [CrossRef] [Medline]
    20. Linet MS, Harlow SD, McLaughlin JK, McCaffrey LD. A comparison of interview data and medical records for previous medical conditions and surgery. J Clin Epidemiol 1989;42(12):1207-1213. [Medline]
    21. Delclos GL, Arif AA, Aday L, Carson A, Lai D, Lusk C, et al. Validation of an asthma questionnaire for use in healthcare workers. Occup Environ Med 2006 Mar;63(3):173-179 [FREE Full text] [CrossRef] [Medline]
    22. Josefson A, Färm G, Meding B. Validity of self-reported nickel allergy. Contact Dermatitis 2010 May;62(5):289-293. [CrossRef] [Medline]
    23. Hoppin JA, Jaramillo R, Salo P, Sandler DP, London SJ, Zeldin DC. Questionnaire predictors of atopy in a US population sample: findings from the National Health and Nutrition Examination Survey, 2005-2006. Am J Epidemiol 2011 Mar;173(5):544-552 [FREE Full text] [CrossRef] [Medline]
    24. Landkroon AP, de Weerd S, van Vliet-Lachotzki E, Steegers EAP. Validation of an internet questionnaire for risk assessment in preconception care. Public Health Genomics 2010;13(2):89-94. [CrossRef] [Medline]
    25. The Netherlands Perinatal Registry. Perinatal care in the Netherlands, 2013. Utrecht: The Netherlands Perinatal Registry; 2014.   URL: http://www.perinatreg.nl/uploads/150/153/PRN_jaarboek_2013_09122014.pdf [accessed 2015-03-23] [WebCite Cache]
    26. Van Gelder MMHJ, Bretveld RW, Roukema J, Steenhoek M, van Drongelen J, Spaanderman MEA, et al. Rationale and design of the PRegnancy and Infant DEvelopment (PRIDE) Study. Paediatr Perinat Epidemiol 2013 Jan;27(1):34-43 [FREE Full text] [CrossRef] [Medline]
    27. Ravelli ACJ, Tromp M, van Huis M, Steegers EAP, Tamminga P, Eskes M, et al. Decreasing perinatal mortality in The Netherlands, 2000-2006: a record linkage study. J Epidemiol Community Health 2009 Sep;63(9):761-765. [CrossRef] [Medline]
    28. World Health Organization. International Classification of Diseases. 2013.   URL: http://www.who.int/classifications/icd/en/ [accessed 2013-10-18] [WebCite Cache]
    29. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol 1990;43(6):543-549. [Medline]
    30. Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol 1990;43(6):551-558. [Medline]
    31. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986 Feb;1(8476):307-310. [Medline]
    32. Van Weel C, Schers H, Timmermans A. Health care in the Netherlands. J Am Board Fam Med 2012 Mar;25 Suppl 1:S12-S17 [FREE Full text] [CrossRef] [Medline]
    33. Tate AR, Dezateux C, Cole TJ, Davidson L, Millennium Cohort Study Child Health Group. Factors affecting a mother's recall of her baby's birth weight. Int J Epidemiol 2005 Jun;34(3):688-695 [FREE Full text] [CrossRef] [Medline]
    34. Coolman M, de Groot CJM, Jaddoe VW, Hofman A, Raat H, Steegers EAP. Medical record validation of maternally reported history of preeclampsia. J Clin Epidemiol 2010 Aug;63(8):932-937. [CrossRef] [Medline]
    35. Bat-Erdene U, Metcalfe A, McDonald SW, Tough SC. Validation of Canadian mothers' recall of events in labour and delivery with electronic health records. BMC Pregnancy Childbirth 2013;13 Suppl 1:S3 [FREE Full text] [CrossRef] [Medline]
    36. Van Gelder MMHJ, van Rooij IALM, de Walle HEK, Roeleveld N, Bakker MK. Maternal recall of prescription medication use during pregnancy using a paper-based questionnaire: a validation study in the Netherlands. Drug Saf 2013 Jan;36(1):43-54. [CrossRef] [Medline]
    37. Wada K, Yatsuya H, Ouyang P, Otsuka R, Mitsuhashi H, Takefuji S, et al. Self-reported medical history was generally accurate among Japanese workplace population. J Clin Epidemiol 2009 Mar;62(3):306-313. [CrossRef] [Medline]
    38. Bergmann MM, Jacobs EJ, Hoffmann K, Boeing H. Agreement of self-reported medical history: comparison of an in-person interview with a self-administered questionnaire. Eur J Epidemiol 2004;19(5):411-416. [Medline]
    39. Smith B, Chu LK, Smith TC, Amoroso PJ, Boyko EJ, Hooper TI, Millennium Cohort Study Team. Challenges of self-reported medical conditions and electronic medical records among members of a large military cohort. BMC Med Res Methodol 2008 Jun;8:37 [FREE Full text] [CrossRef] [Medline]
    40. Baumeister H, Kriston L, Bengel J, Härter M. High agreement of self-report and physician-diagnosed somatic conditions yields limited bias in examining mental-physical comorbidity. J Clin Epidemiol 2010 May;63(5):558-565. [CrossRef] [Medline]
    41. Rasmussen BK, Jensen R, Olesen J. Questionnaire versus clinical interview in the diagnosis of headache. Headache 1991 May;31(5):290-295. [Medline]
    42. Valentinis L, Valent F, Mucchiut M, Barbone F, Bergonzi P, Zanchin G. Migraine in adolescents: validation of a screening questionnaire. Headache 2009 Feb;49(2):202-211. [CrossRef] [Medline]
    43. Hagen K, Zwart JA, Aamodt AH, Nilsen KB, Bråthen G, Helde G, et al. The validity of questionnaire-based diagnoses: the third Nord-Trøndelag Health Study 2006-2008. J Headache Pain 2010 Feb;11(1):67-73 [FREE Full text] [CrossRef] [Medline]
    44. Wills AK, Lawlor DA, Muniz-Terrera G, Matthews F, Cooper R, Ghosh AK, FALCon Study Team. Population heterogeneity in trajectories of midlife blood pressure. Epidemiology 2012 Mar;23(2):203-211 [FREE Full text] [CrossRef] [Medline]
    45. Allen NB, Siddique J, Wilkins JT, Shay C, Lewis CE, Goff DC, et al. Blood pressure trajectories in early adulthood and subclinical atherosclerosis in middle age. JAMA 2014 Feb;311(5):490-497 [FREE Full text] [CrossRef] [Medline]
    46. Alonso A, Beunza JJ, Delgado-Rodríguez M, Martínez-González MA. Validation of self reported diagnosis of hypertension in a cohort of university graduates in Spain. BMC Public Health 2005;5:94 [FREE Full text] [CrossRef] [Medline]

    ‎
    BMI: body mass index
    CRF: case report form
    DBP: diastolic blood pressure
    GP: general practitioner
    ICC: intraclass correlation coefficient
    pneg: observed proportion of negative agreement
    ppos: observed proportion of positive agreement
    PRIDE: PRegnancy and Infant DEvelopment
    SBP: systolic blood pressure
    TOP: termination of pregnancy

    Edited by G Eysenbach; submitted 05.09.14; peer-reviewed by S Farag, Y Jayasinghe; comments to author 19.02.15; revised version received 02.04.15; accepted 18.04.15; published 16.06.15

    Copyright

    ©Marleen MHJ van Gelder, Naomi PE Schouten, Peter JFM Merkus, Chris M Verhaak, Nel Roeleveld, Jolt Roukema. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 16.06.2015.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.