Published on in Vol 27 (2025)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/52043, first published .
Parental Mental Health and Child Maltreatment in the COVID-19 Pandemic: Importance of Sampling in a Quantitative Statistical Study

Parental Mental Health and Child Maltreatment in the COVID-19 Pandemic: Importance of Sampling in a Quantitative Statistical Study

Parental Mental Health and Child Maltreatment in the COVID-19 Pandemic: Importance of Sampling in a Quantitative Statistical Study

Original Paper

1Clinical Psychology and Psychotherapy, Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany

2Clinical Child and Adolescent Psychology and Psychotherapy, Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany

3German Center for Mental Health (DZPG), Berlin, Germany

4Division Methods and Evaluation, Department of Education and Psychology, Freie Universität Berlin, Berlin, Germany

5Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Charité - Universitätsmedizin Berlin, Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Berlin, Germany

*these authors contributed equally

Corresponding Author:

Lara Engelke, BSc, MSc

Clinical Psychology and Psychotherapy

Department of Education and Psychology

Freie Universität Berlin

Habelschwerdter Allee 45

Berlin, 14195

Germany

Phone: 49 30 838 76522

Email: engelke.l@fu-berlin.de


Background: Results on parental burden during the COVID-19 pandemic are predominantly available from nonrepresentative samples. Although sample selection can significantly influence results, the effects of sampling strategies have been largely underexplored.

Objective: This study aimed to investigate how sampling strategy may impact study results. Specifically, we aimed to (1) investigate if outcomes on parental health and child maltreatment during the COVID-19 pandemic from a convenience sample differ from those of a specific representative sample and (2) investigate reasons for differences in the results.

Methods: In 2020, we simultaneously conducted 2 studies: (1) a web-based survey using a convenience sample of 4967 parents of underage children, primarily recruited via social media, and (2) a study using a quota sample representative of the German adult population with underage children (N=1024), recruited through a combination of telephone interviews and computer-assisted web interviews. In both studies, the same questionnaire was used. To evaluate the impact of sampling, we compared the results on outcomes (parental stress, subjective health, parental mental health, general stress, pandemic-related stress, and the occurrence of child maltreatment) between the 2 samples. To explain differences in the results between the 2 studies, we controlled for sociodemographic data, parent-related risk factors, and COVID-19–related experiences.

Results: Compared to parents from the quota sample, parents from the convenience sample reported significantly more parental stress (η2=0.024); decreased subjective health (η2=0.016); more anxiety and depression symptoms (η2=0.055); more general stress (η2=0.044); more occurrences of verbal emotional abuse (VEA; φ=0.12), witnessing domestic violence (WDV; φ=0.13), nonverbal emotional abuse (NEA; φ=0.03), physical abuse (φ=0.10), and emotional neglect (φ=0.06); and an increase of child maltreatment (VEA: exp(B)=2.95; WDV: exp(B)=3.19; NEA: exp(B)=1.65). Sociodemographic data, parent-related risk factors, and COVID-19–related experiences explained the differences in parental stress (remaining difference between samples after controlling for covariates: η2=0.002) and subjective health (remaining difference between samples after controlling for covariates: η2=0.004) and partially explained differences in parental mental health (remaining: η2=0.016), general stress (remaining: η2=0.014), and child maltreatment (remaining: VEA: exp(B)=2.05 and WDV: exp(B)=2.02) between the 2 samples. The covariates could not explain the difference in NEA (exp(B)=1.70). We discuss further factors that may explain the unexplained differences.

Conclusions: Results of studies can be heavily impacted by the sampling strategy. Scientists are advised to collect relevant explaining variables (covariates) that are possibly related to sample selection and the outcome under investigation. This approach enables us to identify the individuals to whom the results apply and to combine findings from different studies. Furthermore, if data on the distribution of these explanatory variables in the population are available, it becomes possible to adjust for sample selection bias.

J Med Internet Res 2025;27:e52043

doi:10.2196/52043

Keywords



Background

The COVID-19 pandemic and the resulting consequences (eg, restrictions, such as school and daycare closings, social distancing, and stay-at-home orders) posed various challenges, especially for families. Parents felt stressed [1-4] and were especially burdened during the COVID-19 pandemic [1,5,6]. Studies showed worsening or significant impairments of parental mental health (depression, anxiety, and stress) [2,6-16], a significant increase in parental stress [3,14,17-20], and an increase in the occurrence of child maltreatment during the COVID-19 pandemic [17,21,22].

The existing trend toward convenience sampling in psychological research [23-25] increased during the COVID-19 pandemic [26]. As is the case for most studies on mental burden and well-being of families during the COVID-19 pandemic, findings on parental outcomes are primarily generated through web-based studies with convenience sampling (eg, [4,7,10,13-17,19,22,27,28]). In a web-based study, convenience sampling is often conducted by recruiting participants through easily accessible methods, such as social media posts, previous study participants, or email lists. Consequently, they are often not representative of the population of interest. Only a few studies evaluating the impact of the COVID-19 pandemic on families recruited a representative sample (eg, [1-3,11]). Therefore, many studies are not representative of the general population or the specific population of interest. In other words, in web-based convenience samples, the general population is often not adequately represented. Differences in sample characteristics may not be a problem, if the sample characteristics that differ are irrelevant for the outcome one wants to assess. However, if the samples differ in variables that are relevant for the outcomes (eg, single parents are less likely to take part in a web-based convenience study on parental burden because they may not have the time; at the same time, single parents will more likely be affected by COVID-19 restrictions), the results will be biased and cannot be generalized to the overall population.

Studies have shown substantial differences in sociodemographic composition from web-based convenience samples in comparison with representative samples [29-32]. In their systematic literature review on recruitment via Facebook for health-related studies, Whitaker et al [33] found that some studies showed an overrepresentation of White, female, younger individuals with higher education and higher income. The overrepresentation of these sample characteristics in web-based convenience samples on parenting and family research is also known [34-39]. In various web-based convenience samples assessing the impact of the COVID-19 pandemic on parents, participants are more likely to be female [4,9,14-17,19,22,27,40-45], in a relationship [4,15,19,22,40,41,43], and have higher education [4,14,16,17,27,41-43,45-47]. Differences in sociodemographic variables (eg, sex, child age, parental age, and socioeconomic status) compared to the general population further influence the outcomes, as some of these factors are associated with poorer parental outcomes and greater family burden during the pandemic [8,11,16,40-42,48-60]. Thus, these study results may not necessarily generalize to the overall population during the COVID-19 pandemic [26,61-66].

The generalizability of results from nonprobability studies with web-based convenience samples was already discussed before the COVID-19 pandemic [29,31,67-69]. In their literature review, Cornesse et al [31] concluded that the generalizability of probability sample surveys (eg, representative samples) is higher than that of nonprobability sample surveys (eg, web-based convenience samples). Similar results were found by other researchers [25,30,32,70-76]. Even before the COVID-19 pandemic, studies assessing mental health indicated that individuals with higher burden were more likely to choose to participate in web-based convenience sample studies, resulting in discrepancies in burden scores compared to representative samples [25,77,78]. This selection effect is also evident in studies assessing the impact of the COVID-19 pandemic, resulting in an overestimation and underestimation of specific outcomes [65,79].

Studies on the generalizability of web-based convenience sampling in parenting and family research are scarce. One Australian study found that mothers surveyed on the web via Facebook, compared to a population-representative sample, showed sociodemographic differences (eg, overrepresentation of younger individuals, those in a relationship, with higher education, and who spoke only English at home) as well as poorer mental health and self-rated health [39]. Similarly, Bennetts et al [80] reported higher psychological distress in their parent sample recruited through Facebook compared to a representative sample.

Objective

To the best of our knowledge, no published studies have investigated the generalizability of results from web-based convenience samples on parental outcomes during the COVID-19 pandemic, leaving the potential biasing effect unknown. The following research questions were addressed: (1) Are results from a convenience sample during the COVID-19 pandemic comparable to results from a specific representative sample regarding parental outcomes (ie, parental stress, parental health, and the occurrence of child maltreatment)? and (2) To what extent can we explain differences in outcomes between the studies by sample characteristics?


Study Design

This study is based on the project “Parenting in Pandemic Times.” The target population of the project was parents with underage children in the household in Germany (national) with sufficient German language skills. Data presented in this paper refer to 2 separate studies conducted simultaneously. First, a quota sample was collected by the market research company INFO Marktforschungsinstitut between August 3 and August 11, 2020. Computer-assisted telephone interviews (CATIs; n=402) were combined with a computer-assisted web interview (CAWI; n=622) to provide data that are as representative as possible. The CAWI allowed respondents to complete the questionnaire on the web at home on a computer, tablet, cell phone, or other smart device. A dual-frame design was used for the CATI sample to reach respondents via landline and cell phone. Participants in the CAWI sample were recruited from an active web-based access panel of the market research institute. They received incentives for their participation. In the first step, specific representativity was assured by recruiting participants based on quota obtained from the latest microcensus data regarding parental sex, parental age, number of children, federal state, school educational level (with vs without A-level), and income. In the second step, to adjust for the remaining disproportion in these variables due to unit nonresponse, INFO Marktforschungsinstitut additionally used poststratification weighting. While educational level was assessed in 4 categories, representativeness (eg, quota and weighting) was only ensured for the dichotomous variable of having an A-level (high education) versus not having an A-level. For the dichotomous variable on education, specific representativity could be assured. The weighting took place retrospectively to the actual quota sampling. (For more information, refer to the study by Calvano et al [2]; in this study, weighting only slightly altered the strata-variable distribution in the specific representative study. Calvano et al [2] determined that using unweighted as compared to weighted data does not alter the findings or conclusions.) Second, a convenience sample was collected between August 2 and September 16, 2020. The recruitment of this sample (N=4967) took place through requests and emails via an web link through social media (eg, Facebook groups, Instagram, and Twitter; 2561/4967, 51.56%; refer to Multimedia Appendix 1, picture 1), friends and family (769/4967, 15.48%), internet (658/4967, 13.25%), schools (551/4967, 11.09%), daycare facilities (263/4967, 5.29%), work (84/4967, 1.69%), clubs (65/4967, 1.31%), clinics for child and adolescent psychiatry (34/4967, 0.68%), child and adolescent therapists and pediatricians (18/4967, 0.36%), newspapers (32/4967, 0.64%), and child protection outpatient clinics (16/4967, 0.32%). As the research team was based in Berlin, Germany, recruitment primarily took place in Berlin through schools, daycare facilities, child and adolescent therapists and physicians, clubs, and newspapers. Social media requests were placed in groups with German-speaking parents. A total of 42.5% (2111/4967) of the convenience sample came from Berlin. Participants in the convenience sample were able to take part in a raffle for vouchers and iPads at the end of the survey. In both samples, few data were excluded due to an unrealistic short completion time (convenience: 53/5020, 1.06%; specific representative: 66/1090, 6.06%). Noteworthily, the surveys were identical in content, that is, all measures and instructions applied were identical.

Ethical Considerations

Data collection, analysis, evaluation, and reporting were conducted in accordance with the current ethical guidelines and data protection laws. All data are anonymized. The Ethics Committee of Charité – Universitätsmedizin Berlin (EA2/128/20; July 27, 2020) approved the study. Participants received information on the purpose of the research, the scope and duration of the diagnostic measures, possible risks and benefits, the voluntary nature of participation, and all aspects of data protection. For participation in the study, participants agreed to an informed consent declaration. Participants were provided with contact details for support services. Participants in the convenience sample could take part in a prize raffle for cash vouchers. Participants in CAWI received incentives from the market research company.

Measures

We distinguished between explaining variables (sociodemographic data, parent-related risk factors, and pandemic-related experience) and parental outcomes (parental stress, subjective health, parental mental health, general stress, and child maltreatment).

Explaining Variables
Sociodemographic Data

Data on parental sex and age, parental status (biological parent, stepparent, and other), nationality, number of children in the household, children’s age and sex, marital status, federal state, parental school and professional education, and parental current employment status were assessed. We calculated the Winkler Index [81] with reference data from the German population to classify the socioeconomic status as low, medium, or high.

Parent-Related Risk Factors

The alcohol abuse module of the Patient Health Questionnaire, German version [82,83] was applied to assess parents’ risk of alcohol abuse during the COVID-19 pandemic. Furthermore, the study assessed parents’ experiences of sexual or physical violence in adulthood, their own history of childhood sexual or physical abuse, and the presence of mental disorders. In addition, parents were asked to report the presence of a severe or chronic physical condition and if they belonged to the risk group for COVID-19.

Pandemic-Related Experience

To measure pandemic-related experiences and stress related to COVID-19, the Pandemic Stress Scale (Winter, S, Corona Belastungsskala. Charité - Universitätsklinikum Berlin, Klinik für Psychiatrie, Psychosomatik und Psychotherapie des Kindes- und Jugendalter, Sektion Traumafolgen und Kinderschutz, unpublished data, January 2020) was developed. Questions were answered with respect to household members, the family, or the parents themselves. At first, COVID-19–related personal experiences were assessed, that is, contact with persons with a COVID-19 infection, hospital admission, or death. Afterward, the month or months with the subjectively highest burden (January 2020 to August 2020) were assessed.

Parental Outcomes
Parental Stress

To assess negative and positive perceptions of parenthood, the Parental Stress Scale [84] was implemented for (1) the time with the subjectively highest burden (total sample [convenience and specific representative sample combined]: Cronbach α=0.88) and (2) before the COVID-19 pandemic, that is, January 2020 in retrospect (total sample: Cronbach α=0.86). Parents rated their perception on the 18 items on a 5-point frequency scale. Higher scores expressed higher parental stress.

Subjective Health

To measure parents’ perception of their own health (1) at the time of the highest burden and (2) retrospectively in January 2020, a well-established single-item measure [85] with a 10-point scale was implemented. Higher scores indicated better self-rated health.

Parent Mental Health

To assess depression and anxiety symptoms, the Patient Health Questionnaire 4 [86] was applied (total sample: generalized anxiety, Cronbach α=0.70; depression, Cronbach α=0.68; and total score, Cronbach α=0.78). Higher scores on the 4-point scale indicated poorer mental health.

Pandemic-Related Stress

The Pandemic Stress Scale (Winter, S, Corona Belastungsskala. Charité - Universitätsklinikum Berlin, Klinik für Psychiatrie, Psychosomatik und Psychotherapie des Kindes- und Jugendalter, Sektion Traumafolgen und Kinderschutz, unpublished, January 2020) consists of 14 items about specific restrictions (eg, daycare closings and social distancing), which parents rated regarding their subjective burden on a 5-point scale (ranging from not at all burdening to very burdening). The answering format also included a “not applicable” option. The Pandemic Stress Scale refers to the time point with the subjectively highest burden during the COVID-19 pandemic so far, from the beginning to August 2020. The scale had good internal consistency (total sample: Cronbach α=0.85). Higher scores expressed a higher subjective burden.

General Stress

To measure general parental stress at the time of the subjectively highest burden, we implemented the stress module of the Patient Health Questionnaire, German version [83] (total sample: Cronbach α=0.71). It consists of 10 items with several psychosocial stressors on a 3-point scale, and higher scores indicated higher general stress.

Child Maltreatment

Parents provided information about child maltreatment regarding the children in their household. The Pediatric Maltreatment and Abuse Chronology of Exposure interview [87-89] was adjusted to collect data on household dysfunction, child neglect, and abuse. First, parents were asked if the children ever faced severe stressful life experiences (eg, abuse, violence, and neglect). Afterward, 10 items on distinct subtypes of child maltreatment experiences were asked: 5 items on different types of child abuse (nonverbal emotional abuse [NEA], verbal emotional abuse [VEA], physical abuse, witnessing domestic violence [WDV], and sexual abuse), 3 items on different types of neglect (supervisory neglect, physical neglect, and emotional neglect), and 2 items on different types of household dysfunction (problems regarding mental illness or alcohol or substance use in the household). Only the first item explicitly referred to severe levels of maltreatment; the other items on subtypes mirrored comparatively lower severity levels in our surveys (according to the maltreatment classification system [90,91]). First, a dichotomous yes or no item was presented for each subtype asking parents if the children ever experienced a specific subtype of child maltreatment. If the answer was yes, a follow-up question was posed, asking about the change in frequency since the beginning of the COVID-19 pandemic, with response options on a 5-point scale ranging from “significantly more often” to “significantly less often.” For more details about the measure, refer to the study by Calvano et al [2]. To simplify the presentation of the results, we dichotomized the responses on child maltreatment during the COVID-19 pandemic into increase (increase or significant increase) versus no increase (staying the same, decrease, or significant decrease). To provide meaningful results, we decided not to further compare subtypes with case numbers smaller <30.

Data Analysis

For investigating the research questions, we aimed to answer the following: To what extent do the 2 studies differ with respect to the parental outcomes and specific sample characteristics? How well can we predict sample membership by these sample characteristics? How well can we explain differences in parental outcomes between the 2 samples by sample characteristics?

Investigating Differences in Explaining Variables Between Samples

We calculated effect sizes describing the differences in explaining variables/covariates (sociodemographic data, parent-related risk factors, and COVID-19–related experiences) between the 2 studies. The significance of differences between the samples was investigated using 2-tailed t tests or Welch tests for interval-scaled variables (eg, age, rooms in the household, parental mental health, and parental stress) and chi-square tests for nominal and ordinal variables (eg, sex, federal state, nationality, garden, belonging to a risk group for COVID-19, and school education). For metric variables, we report Cohen d, and for nominal and ordinal-scaled variables, we report the phi coefficient.

After examining differences between studies for each individual variable, the next step was to consider all covariates simultaneously. As such, we used logistic regression to predict sample membership (0=specific representative sample and 1=convenience sample) by the covariates, that is, sociodemographic data, parent-related risk factors, and COVID-19–related experience. Due to estimation requirements, we only reported on the prediction of study membership for categories of categorical outcomes with an absolute frequency >30.

Investigating Differences in Parental Outcomes Between Samples

We calculated effect sizes describing the differences in outcomes between the 2 studies, using ANOVA. We report partial η2 for effect size. Chi-square tests were used for nominal and ordinal outcomes, with the phi coefficient reported for effect size.

Explaining Differences in Parental Outcomes Between Samples

We conducted analyses of covariance (ANCOVAs) using sample type and all covariates as predictors for the respective outcome variable. We then compared the adjusted effect estimates for sample type from the ANCOVA with the unadjusted effect estimates from an ANOVA that did not include any covariates. When in the ANCOVA the equality of error variances assumption was violated, we calculated parameter estimates with robust standard errors for the ANCOVAs. We could not find any considerable differences in the results, and therefore, we decided to present the results of the common and well-known standard methods. The size of the adjusted difference in outcomes between types of samples (adjusted effect estimate) indicates how much the covariates describe the difference in study results between the 2 samples. If effect estimates of the type of sample on the outcome variable decrease or even disappear after controlling for covariates, the considered covariates explain partly (in case of just decrease) or fully (in case of disappearance) the differences between samples for the respective outcome. Herein, we compared the effect sizes (η2) of the sample indicator using ANOVA, with the covariate-adjusted effect sizes (η2) of the sample indicator from the ANCOVA.

For an increase in child maltreatment (0=no increase and 1=increase), binary logistic regressions for predicting each subtype of child maltreatment from study type (and covariates) were used. Odds (exp(B)) of the type of sample adjusted for all covariates were compared to unadjusted odds (exp(B)) with study type being the only predictor. If the odds of sample type influencing the outcome variable decrease or approach 1 after controlling for covariates, it indicates that the covariates partially (if the odds decrease) or fully (if the odds approach 1) explain the differences between the samples for the respective outcome. Due to estimation requirements, we only reported on covariate-adjusted outcome differences between studies for categories of categorical outcomes with an absolute frequency >30. All analyses were performed using SPSS (version 29; IBM Corp).


In the following, the results of the convenience sample are compared with the results of the specific representative sample. Note that results are hardly impacted by the weighting [2].

Investigating Differences in Sample Characteristics

We evaluated differences in sociodemographic data, parent-related risk factors, and COVID-19–related experiences.

Sociodemographic Data

Tables 1 and 2 depict the comparison of sociodemographic characteristics of the 2 samples (convenience vs specific representative). In the convenience sample, 4967 people, primarily women (n=4349, 87.56%), participated with a mean age of 39.11 (SD 6.25; range 20-70) years. The average number of children in the household was 1.82 (SD 0.78; range 1-11). These children were on average aged 6.46 (SD 4.25; range 0-17) years. In 42.5% (2111/4967) of the convenience sample, the families lived in Berlin. The distribution of the participants among the federal states and a comparison between the samples is shown in Table S1 in Multimedia Appendix 1. In most cases, participants worked either part time (2051/4967, 41.29%) or full time (1778/4967, 35.8%). Most participants were highly educated with a high school education (4305/4967, 86.67%) or a university degree (3390/4967, 68.25%), and they had a high socioeconomic status (3165/4967, 63.72%).

The specific representative sample comprised 1024 participants (n=534, 52.14% women) with a mean age of 40.89 (SD 8.17; range 18-73) years. On average, 1.69 (SD 0.77; range 1-6) children aged <18 years lived in the household. The mean age of the children was 8.92 (SD 4.92; range 0-17) years. In total, 4.3% (44/1024) of the participants lived in Berlin. In 55.66% (570/1024) of the cases, the parent worked full time, and in 27.64% (283/1024) of the cases, they worked part time. Most participants had a middle school education (480/1024, 46.88%), had completed an apprenticeship (586/1024, 57.23%), and had a middle socioeconomic status (585/1024, 57.13%).

Table 1. Sociodemographic characteristics of the convenience sample and the specific representative sample—parent data.

Convenience sample 2020, (N=4967), n (%)Specific representative sample 2020a, (N=1024), n (%)Comparison, test statisticsP valueEffect size
Parental sex (female), n (%)4349 (87.6)534 (52.1)χ21=706.2<.001φ=0.34
Biological parent, n (%)4913 (97.8)979 (95.6)χ21=57.1<.001φ=0.10
Single parent, n (%)504 (10.1)123 (12)χ21=3.2.08φ=0.02
Nationality, n (%)

German4803 (94.7)1001 (97.8)χ21=3.1.08φ=0.02

Other270 (5.3)23 (2.2)χ21=16.3<.001φ=0.05
Age (years), mean (SD)39.11 (6.25)40.89 (8.17)t1218.10=6.58<.001d=0.27
Marital status: in a relationship, n (%)4568 (92.9)917 (89.6)χ21=13.3<.001φ=0.05
Federal State of Germany: city-state, n (%)2263 (45.6)75 (7.3)χ21=521.6<.001φ=0.30
School education, n (%)

Low (up to 9 years of schooling)58 (1.2)109 (10.7)χ21=281.4<.001φ=–0.22

Middle (10 years of schooling)564 (11.3)480 (42.5)χ21=744.4<.001φ=–0.35

High (up to 13 years of schooling)4305 (86.7)426 (41.6)χ21=1038.4<.001φ=0.42

No or other education40 (0.7)9 (5.2)χ21=0.1.81φ=–0.003
Professional education, n (%)

Apprenticeship1041 (21.0)586 (57.2)χ21=564.5<.001φ=–0.31

Technical school308 (6.2)119 (11.6)χ21=37.7<.001φ=–0.08

University3390 (68.3)247 (24.1)χ21=693.1<.001φ=0.34

No or other education228 (4.6)72 (7)χ21=10.6<.001φ=–0.04
Current employment status: working, n (%)3983 (80.2)863 (84.3)χ21=9.2<.001φ=–0.04
Socioeconomic status indexb, n (%)t1268.97=–4.05<.001d=–.17

Low116 (2.3)100 (9.8)



Middle1686 (33.9)585 (57.5)



High3165 (63.7)338 (33)


Household

Balcony, n (%)3125 (62.9)598 (58.4)χ21=7.4.007φ=–0.04

Garden, n (%)2969 (59.8)768 (75.0)χ21=83.9<.001φ=–0.12

Rooms, mean (SD)4.31 (3.13)4.87 (4.42)t1242.91=3.84<.001d=0.16

Other persons in the household, n (%)4357 (87.7)914 (89.3)χ21=1.9.17φ=–.02

aWeighted data.

bIndex calculated according to the Winkler Index [81].

Table 2. Sociodemographic characteristics of the convenience sample and the specific representative sample—child data.

Convenience sample 2020 (N=4967), n (%)Specific representative sample 2020a (N=1024), n (%)Comparison, test statisticsP valueEffect size
Number of childrent1492.82=–4.90<.001d=–0.17

11813 (36.65)474 (46.3)



22417 (48.7)427 (41.7)



≥3737 (14.8)122 (12)


Child’s age (y)t1329.24=18.06<.001d=0.70

0-21953 (39.3)230 (22.3)



3-52380 (48)283 (27.4)



6-123634 (73.1)713 (69.6)



13-171046 (21.1)506 (49.5)


Child’s sex (female)4387 (48.7)862 (49.9)b
Children living in 2 households155 (3.1)27 (2.6)χ21=0.7.41φ=–.01
Previous contact with social and family services1197 (24.1)249 (24.3)χ21=0.2.88φ=–.01

aWeighted data.

bNot applicable.

The 2 samples differed substantially in relevant characteristics. The convenience sample had a significantly higher number of women, the children were younger, participants were more frequently from a city-state (ie, Berlin), and had a higher school and professional education than the specific representative sample (all differences with medium effect sizes). Compared to the specific representative sample, in the convenience sample, we observed small differences in the following variables: participants were significantly younger, had more children aged <18 years in the household, had a higher socioeconomic status, were of another nationality than German, were more frequently biological parents and in a relationship, had fewer rooms in the household, and had less frequently a garden. Samples were comparable regarding being a single parent, child sex, current employment status, having a balcony, children living in 2 households (eg, in case of separated parents), and having others in the household or previous contact with social family services.

Parent-Related Risk Factors

Table 3 shows parent-related risk factors. Significantly fewer parents from the convenience sample reported belonging to the risk group for severe COVID-19. A significantly higher share of parents from the convenience sample reported a current or previous mental disorder and a history of own child abuse or neglect. Furthermore, a significantly higher share of parents from the convenience sample reported March and April 2020 as the most stressful months. In contrast, a significantly higher share of parents from the specific representative sample reported that “all months were equally stressful” and “no month was especially stressful.” The effect-sizes were small.

Table 3. Parent-related risk factors: descriptive data and comparison between the samples.

Convenience sample 2020 (N=4967), n (%)Specific representative sample 2020a (N=1024), n (%)Comparison



χ2 (df)P valueφ
Parental risk of alcohol abuseb317 (6.38)56 (5.47)1.2 (1).270.01
Parental mental disorder864 (17.39)107 (10.45)30.2 (1)<.0010.07
Chronic severe health condition604 (12.16)146 (14.26)3.4 (1).07–.02
Parental history of child abuse or neglect1562 (31.45)238 (23.24)27.0 (1)<.0010.07
Parental experience of violence in adulthood601 (12.10)114 (11.13)0.8 (1).39–.01

aWeighted data.

bn=3430 (convenience sample) and n=700 (specific representative sample) parents indicated that they regularly drink alcohol. n=317 (convenience sample) and n=56 (specific representative sample) were at risk for alcohol abuse according to Patient Health Questionnaire-D [83].

COVID-19–Related Experiences

Table 4 shows the COVID-19–related experiences. In both samples, nearly a quarter of the participants reported reduced working hours (convenience sample: 1197/4967, 24.1% and specific representative sample: 277/1024, 27.05%), and one-fifth of the participants reported a significant financial loss due to COVID-19 (convenience sample: 1111/4967, 22.37% and specific representative sample: 221/1024, 21.58%). In addition, April 2020 (convenience sample: 3508/4967, 70.62% and specific representative sample: 487/1024, 47.56%) and May 2020 (convenience sample: 2975/4967, 59.89% and specific representative sample:430/1024, 42%) were the most stressful months until August 2020 in both samples.

Table 4. COVID-19–related experiences: descriptive data and comparison.

Convenience sample 2020 (N=4967), n (%)Specific representative sample 2020a (N=1024), n (%)Comparison



χ2(df)P valueφ
Effects of the COVID-19 pandemic on the health situation

Family or household member infected with COVID-1993 (1.87)22 (2.15)0.3 (1).56–0.01

A family or household member was hospitalized due to COVID-1915 (0.30)8 (0.78)5.0 (1).02–0.03

Family or household member passed away due to COVID-1913 (0.26)4 (0.39)0.5 (1).480.01
Parent belongs to the risk group for severe COVID-19328 (6.60)103 (10.06)15.3 (1)<.0010.05
Effects of the COVID-19 pandemic on job situation

Reduced working hours1197 (24.10)277 (27.05)4.0 (1).0460.03

Job loss235 (4.73)55 (5.37)0.8 (1).390.01

Significant financial loss1111 (22.37)221 (21.58)0.3 (1).58–0.01
Most stressful months

February113 (2.28)21 (2.05)0.2 (1).660.01

March2207 (44.43)268 (26.17)116.8 (1)<.0010.14

April3508 (70.63)487 (47.56)203.3 (1)<.0010.18

May2975 (59.90)430 (41.99)110.9 (1)<.0010.14

June1634 (32.90)263 (25.68)20.4 (1)<.0010.06

July582 (11.72)117 (11.43)0.1 (1).790.003

August307 (6.18)46 (4.49)4.4 (1).040.03

All months were equally stressful288 (5.80)125 (12.21)54.3 (1)<.001–0.10

No month was especially stressful381 (7.67)186 (18.16)109.1 (1)<.001–0.14

aWeighted data.

All 34 covariates were simultaneously considered in a binomial logistic regression for the prediction of sample membership (0=specific representative sample and 1=convenience sample). Explained variance in outcomes by the predictors was large (Nagelkerke R2=0.56) and statistically significant from 0 (χ236=2411.4; P<.001). The overall accuracy percentage in classification was 89.6%, with a sensitivity of 96.3% and a specificity of 57.2%. The 5 most important predictors for convenience sample membership were as follows: living in a federal state in Germany (Berlin, Bremen, or Hamburg; odds ratio [OR] 14.75, 95% CI 10.93-19.91), parental sex female (OR 11.13, 95% CI 8.89-13.94), parental mental disorder (OR 2.18, 95% CI 1.63-2.92), being a biological parent (OR 2.05, 95% CI 1.11-3.79), and higher education (professional education [university]: OR 2.02, 95% CI 1.33-3.08 and school education [high school]: OR 2.01, 95% CI 0.77-5.23). All model coefficients and odds ratios can be found in Table S2 in Multimedia Appendix 1.

Investigating Differences in Parental Outcomes Between Samples

Differences in parental outcomes between the convenience sample and the specific representative sample are displayed in Table 5 for parental outcomes and Tables 6 and 7 for child maltreatment outcomes.

Table 5. Descriptive data: investigating and explaining differences between the convenience and the specific representative samples in parental outcomes.

Convenience sample (N=4967), mean (SD)aSpecific representative sampleb (N=1024 [weighted]), mean (SD)aComparison between the samples, ANOVAs (unadjusted)ANCOVAsc (adjusted)



F test (df)P valueη2F test (df)P valueη2
Parental stress: January 202036.07 (8.67)34.72 (10.63)18.35 (1)<.0010.0030.33d.56<0.001
Parental stress: time of the highest burden41.37 (10.50)36.93 (10.45)149.97 (1)<.0010.02414.92e<.0010.002
Subjective health: January 20207.36 (1.76)7.34 (2.04)0.03 (1).87<0.0010.06f.81<0.001
Subjective health: time of the highest burden6.06 (2.27)6.80 (2.21)101.13 (1)<.0010.01621.69g<.0010.004
Patient Health Questionnaire-4 (anxiety and depression symptoms)4.37 (3.0)2.52 (2.63)354.94 (1)<.0010.05597.98h<.0010.016
General stress7.40 (3.73)5.28 (4.13)4023.2 (1)<.0010.04484.21i<.0010.014
Pandemic-related stress31.80 (8.03)31.97 (10.96)0.56 (1).45<0.0016.75j.0090.001

aUnadjusted significant covariates.

bWeighted data.

cANCOVA: analysis of covariance.

dBiological parent, federal state, current employment status, previous contact with social and family services, parental mental disorder, parental history of child abuse or neglect, children’s age, parental age, socioeconomic status, number of children in the household, and parental risk of alcohol abuse.

eParental sex, biological parent, federal state, current employment status, previous contact with social and family services, parental mental disorder, parental history of child abuse or neglect, children’s age, parental age, socioeconomic status, and number of children in the household.

fNationality: German, previous contact with social and family services, family or household member had a COVID-19 infection, family or household member was hospitalized due to COVID-19, parental chronic severe health condition, parental mental disorder, parent belongs to the risk group for severe COVID-19, parental history of child abuse or neglect, socioeconomic status, and parental risk of alcohol abuse.

gParental sex, federal state, previous contact with social and family services, significant financial loss, parental chronic severe health condition, parental mental disorder, parental history of child abuse or neglect, children’s age, and parental risk of alcohol abuse.

hParental sex, federal state, single parent, previous contact with social and family services, the availability of a garden, family or household member admitted to the hospital with COVID-19, significant financial loss, parental mental disorder, parental history of child abuse or neglect, parental experience of violence in adulthood, children’s age, parental age, number of children in the household, parental risk of alcohol abuse, and number of persons in the household.

iParental sex, federal state, current employment status, previous contact with social and family services, significant financial loss, reduced working hours, job loss, parental mental disorder, parent belongs to the risk group for severe COVID-19, parental history of child abuse or neglect, parental experience of violence in adulthood, children’s age, socioeconomic status, child having another main place of residence, and parental risk of alcohol abuse.

jParental sex, current employment status, previous contact with social and family services, significant financial loss, reduced working hours, job loss, parental mental disorder, parent belongs to the risk group for severe COVID-19, parental experience of violence in adulthood, children’s age, number of children in the household, and number of persons in the household.

Table 6. Comparison of the occurrence and frequency of adverse childhood experiences between the convenience sample and the specific representative sample.

OccurrenceIncrease during the COVID-19 pandemic

Values, n (%)χ2 (df)P valueφn (%)χ2(df)P valueφ
Severe stressful living conditions0.1 (1).750.004
0.4 (1).520.01

Web-based sample307 (6.2)


129 (2.6)



Representative sample66 (6)


23 (2)


Physical abuse55.8 (1)<.0010.10
38.5 (1)<.0010.08

Web-based sample696 (14.1)


329 (6.6)



Representative sample57 (5.6)


17 (2)


Nonverbal emotional abuse7.0 (1).030.03
10.5 (1).0010.04

Web-based sample470 (10)


356 (7.1)



Representative sample82 (8)


45 (4)


Supervisory neglect3.1 (1).08–0.02
11.3 (1)<.0010.04

Web-based sample559 (11.3)


326 (6.6)



Representative sample96 (9.4)


39 (4)


Emotional neglect24.7 (1)<.0010.06
0.8(1).37–0.01

Web-based sample497 (10)


284 (5.7)



Representative sample157 (15.3)


66 (6)


Verbal emotional abuse91.5 (1)<.0010.12
137.83<.0010.15

Web-based sample2423 (48.82)


1585 (31.91)



Representative sample332 (32.4)


140 (13.7)


Witness of domestic violence107.9 (1)<.0010.13
116.91 (1)<.0010.14

Web-based sample2488 (50.22)


1236 (24.91)



Representative sample332 (32.4)


97 (10)


Sexual abuse15.2 (1)<.0010.05
a

Web-based sample19 (0.4)


2 (0.04)



Representative sample14 (1.4)


3 (0.3)


Physical neglect8.4 (1).020.04

Web-based sample19 (0.4)


8 (0.2)



Representative sample11 (1)


3 (0.3)


aInference statistics are only reported for cell counts ≥5 in all cells.

Table 7. Logistic regression analysis: investigating and explaining differences in the increase of child maltreatment outcomes between the convenience and the specific representative samples.

UnadjustedAdjusted

B (SE)P valueORa (95% CI)B (SE)P valueOR (95% CI)
Verbal emotional abuse1.08b (0.10)<.0012.95 (2.44-3.56)0.72c (0.12)<.0012.05d (1.63-2.57)
Witnessing domestic violence1.16e (0.11)<.0013.19 (2.56-3.97)0.70f (0.13)<.0012.02g (1.55-2.63)
Nonverbal emotional abuse0.50h (0.16).0021.65 (1.20-2.27)0.53i (0.20).011.70j (1.15-2.52)
Emotional neglect–0.14k (0.14).330.87 (0.66-1.148)–0.10l0.19 (0.59)0.90m (0.62-1.30)

aOR: odds ratio.

bModel fit: χ21=153.5; P<.001; Nagelkerke R2=0.04.

cModel fit: χ237=589.6; P<.001; Nagelkerke R2=0.14; Hosmer and Lemeshow χ28=10.3; P=.25.

dCovariates with a significant partial regression coefficient: Sample membership, parental sex, number of children in the household, parental age, children’s age, federal state, current employment status, previous contact with social and family services, the availability of a balcony, reduced working hours, job loss, parental mental disorder, parental history of child abuse or neglect, school and professional education, and child’s sex.

eModel fit: χ21=137.1; P<.001; Nagelkerke R2=0.04.

fModel fit: χ237=522.7; P<.001; Nagelkerke R2=0.13; Hosmer and Lemeshow χ28=6.9; P=.54.

gCovariates with a significant partial regression coefficient: Sample membership, parental sex, number of children in the household, child’s age, single parent, material status, current employment status, previous contact with social and family services, significant financial loss, parental mental disorder, parental history of child abuse or neglect, and professional education: university.

hModel fit: χ21=10.8; P=.001; Nagelkerke R2=0.01.

iModel fit: χ237=248.6; P<.001; Nagelkerke R2=0.11; Hosmer and Lemeshow χ28=11.1; P=.20.

jCovariates with a significant partial regression coefficient: Number of children in the household, children’s age, federal state, current employment status, previous contact with social and family services, the availability of a garden, significant financial loss, parental mental disorder, parental history of child abuse or neglect, parent belongs to the risk group for severe COVID-19, and professional education: apprenticeship.

kModel fit: χ21=0.9; P=.33; Nagelkerke R2≤0.001.

lModel fit: χ237=96.9; P<.001; Nagelkerke R2=0.05; Hosmer and Lemeshow χ28=5.2; P=.74. Covariates with a significant partial regression coefficient.

mCovariates with a significant partial regression coefficient: Previous contact with social and family services, significant financial loss, and parental mental disorder.

Parental Stress

In January 2020, parents from the convenience sample reported significant but negligibly more parental stress (η2=0.003). However, at the time of the highest burden, parents from the convenience sample reported significantly higher parental stress than parents from the specific representative sample, with a small effect (η2=0.024).

Subjective Health

In January 2020, there was no significant difference (η2<0.001) between parents of the 2 samples regarding subjective health (η2<0.001). Again, at the time of the highest burden, parents from the convenience sample reported significantly poorer subjective health, with a small effect (η2=0.016).

Mental Health and General Stress

Parents in the convenience sample reported higher overall anxiety and depression symptoms (Patient Health Questionnaire-4) than parents in the specific representative sample at times of the highest burden, with a medium effect size (η2=0.055). The same pattern applied to general stress, with a medium effect (η2=0.044).

Pandemic-Related Stress

Samples did not differ in their overall pandemic-related stress (η2<0.001).

Child Maltreatment

In both samples, the lifetime occurrence of WDV (convenience sample: 2488/4966, 50.2% and specific representative sample: 332/1024, 32.42%) and VEA (convenience sample: 2423 4965, 48.82% and specific representative sample: 3321023, 32.4%) were the most frequent subtypes (Table 6). In the convenience sample, a higher share of parents reported the lifetime occurrence of VEA (φ=0.12), WDV (φ=0.13), NEA (φ=0.03), physical abuse (φ=0.10), and emotional neglect (φ=0.06). No differences were found for severe stressful living conditions, NEA, or supervisory neglect. The odds of a child experiencing an increase in one of the following subtypes of child maltreatment were higher in the convenience sample: for VEA (convenience sample: 1585/4966, 31.91% and specific representative sample: 140/1024, 13.7%), it was 2.95 times higher; for WDV (convenience sample: 1236/4961, 24.91% and specific representative sample: 97/1024, 9.47%), it was 3.19 times higher; and for NEA (convenience sample: 356/4966, 7.17% and specific representative sample: 45/1024, 4.39%), it was 1.65 times higher during the COVID-19 pandemic (Table 7). The odds (0.87) of an increase in the frequency of emotional neglect were not significantly different between the samples (convenience sample: 284/4967, 5.72% and specific representative sample: 66/1024, 6.45%).

Explaining Differences in Parental Outcomes Between the 2 Samples

To explain differences in parental outcomes, we calculated ANCOVAs with sociodemographic data, parent-related risk factors, and COVID-19–related experiences as covariates. Only significant covariates are listed in Tables 5 and 7.

Parental Stress

At the time of the highest burden, the difference in parental stress between the samples remained significant but was negligible in size (η2=0.002) after controlling for sociodemographic data, parent-related risk factors, and COVID-19–related experiences. The covariates explained 92% of the difference in the outcome between the studies. The difference in parental stress between the samples in January was still not significant, and the effect size was negligible (η2<0.001). As there were no differences between samples, the covariates cannot explain any of them. It must be noted that the covariates also do not induce any relevant differences.

Subjective Health

The difference between the samples at the time of the highest burden was significant but negligible in size (η2=0.004) after controlling for sociodemographic data, parent-related risk factors, and COVID-19–related experiences. The covariates explained 75% (at the time of the highest burden) of the difference in outcome between the samples. The difference between the samples in January regarding subjective health was still not significant, and the effect size was negligible (η2<0.001). As there were no differences between samples, the covariates cannot explain any of them. It must be noted that the covariates also do not induce any relevant differences.

Mental Health and General Stress

The difference between the samples regarding mental health was small (η2=0.016) but significant after controlling for sociodemographic data, parent-related risk factors, and COVID-19–related experiences. The covariates explained 71% of the difference in the outcome between the samples. The same pattern applied to general stress (η2=0.014). In total, 68% of the difference in the outcome between the samples was explained by the covariates.

Pandemic-Related Stress

The difference between the samples regarding pandemic-related stress was significant, and the effect size slightly grew but was still negligible (η2=0.001) after controlling for sociodemographic data, parent-related risk factors, and COVID-19–related experiences. As there are no differences between studies, the covariates cannot explain any of them.

Child Maltreatment

There are large and significant differences in child maltreatment between the 2 samples in all subcategories except emotional neglect. The odds of experiencing VEA, WDV, and NEA were between 1.65 and 3.19 times higher in the convenience sample as compared to the specific representative sample. The difference in VEA and WDV between the samples could partly be explained by the covariates. After controlling for sociodemographic data, parent-related risk factors, and COVID-19–related experiences, the odds ratio of a child experiencing an increase in VEA and WDV in the convenience sample compared to the specific representative sample were 2.05 (as compared to 2.95 before adjustment) and 2.02 (as compared to 3.19 before adjustment), respectively. The differences in NEA between the samples could not be explained by the covariates. As there were no differences in emotional neglect between samples, there were no effect differences to be explained by the covariates. It must be noted that the covariates also do not induce any relevant differences.


Principal Findings

So far, research has paid little attention to the representativeness of samples. However, this is crucial for drawing valid inferences on the population of interest. This is especially important in areas in which far-reaching decisions are made based on results of a study. This is specifically prevalent in studies on psychological consequences of the COVID-19 pandemic, which made immediate decisions necessary. In this paper, we showed that sampling strategy can have a great impact on the results of a study. We relied on data investigating parental outcomes in the COVID-19 pandemic of 2 web-based studies that differ in their sampling scheme, one being a typical convenience sample (N=4967) and the other a typical specific representative sample drawn by a survey institute (N=1024).

Our first aim was to investigate the impact of sampling strategy on the results. For this, we examined differences in results between the 2 samples. Our second aim was to investigate reasons for these differences. For this, we assessed (1) to what extent the studies differ in sample characteristics and to what extent the sample characteristics may explain group membership and (2) to what extent differences in parental outcomes between the 2 samples can be explained by sample characteristics (sociodemographic data, parent-related risk factors, and COVID-19–related experiences).

Discussion of Results

Differences in Sample Characteristics

We found that the convenience sample differed from the specific representative sample in sociodemographic composition, parent-related risk factors, and COVID-19–related experiences. Sociodemographic data, parent-related risk factors, and COVID-19–related experiences predicted sample membership well. Thus, this study identified a set of variables that partly explain differences in study results and explain differences in the sample selection.

As expected, the sociodemographic composition of the 2 study samples (convenience and specific representative sample) differed in several ways (eg, overrepresentation of the female sex, younger parental and child age, a higher number of children, higher socioeconomic status, and higher education level in the convenience sample). The differences in the sociodemographic composition of the convenience sample were in line with previous findings [29-32,34-39]. Whitaker et al [33] also reported an overrepresentation of White, female individuals, and younger individuals with higher income and educational status compared to the general population in some studies in their literature review on Facebook recruitment. The overrepresentation of women in convenience samples is likely to be especially pronounced in studies related to mental and physical health [33,69] as well as parenting and family [35,38,80,92-94]. In various convenience samples evaluating parenthood during the COVID-19 pandemic, mothers constituted the majority [4,9,14-17,19,22,27,40-42,44,45]. However, the effects of this overrepresentation have not yet been empirically examined. Furthermore, in many of these convenience samples, most participants reported higher education [4,14,16,17,27,41,42,45-47]. One reason could be that women [95,96], younger people with younger children, and people with higher educational qualifications may more readily participate in web-based research and are thus more likely to be reached for study purposes [80,95,97]. Individuals with high educational attainment may be more familiar and more interested in research. In addition, the use of personal emails and invitations for recruitment may have resulted in a participant pool primarily from academic circles. The overrepresentation of women, along with younger parents and those with more children, could also be attributed to thematic self-selection. This group of parents, who were particularly stressed during the pandemic [4,14,59], may have been more motivated to share their experiences of hardship [30]. In summary, the sample composition of our convenience sample is comparable to many other samples recruited during the COVID-19 pandemic.

Differences in Parental Outcomes

Results on parental outcomes during the COVID-19 pandemic differed notably between the 2 samples. Overall, parents from the convenience sample were more burdened during the COVID-19 pandemic than parents from the specific representative sample. Compared with parents from the specific representative sample, parents from the convenience sample reported significantly more parental stress at the time of the highest burden, less subjective health, more anxiety and depression symptoms overall (Patient Health Questionnaire-4), more general stress, more often the occurrence of child maltreatment (VEA, WDV, physical abuse, NEA, and emotional neglect), and an increase of child maltreatment during the COVID-19 pandemic (VEA, WDV, and NEA). These results corroborate the claim that sampling has considerable impact on the results [31].

Our results are in line with findings from previous studies. Studies show that people with higher burdens are more likely to participate in web-based surveys than those with lower burdens [25,39,77,78]. The reasons may be a higher motivation to share one’s burden and a thematic awareness of the topic due to one’s own burden [30]. A higher amount of mental health burden in web-based studies with convenience sampling compared to representative sampling has also been observed in other studies with samples of parents [39,80] before the COVID-19 pandemic. Lawson et al [98] showed that the recruitment of web-based convenience samples influenced the prevalence of child abuse. Joyal-Desmarais et al [79] also found differences between samples in >70% of their variables when comparing their web-recruited convenience samples with representative surveys regarding preventive behaviors.

Explaining Differences in Parental Outcomes Between Studies

Differences in parental outcomes between the 2 samples could (partially) be explained by differences in sociodemographic data, parent-related risk factors, and COVID-19–related experiences between the samples. Consequently, at least for this study, we seem to have identified relevant confounders for the difference in effects between the 2 samples.

On the basis of our results, it is initially not possible to say which covariates are explicitly responsible for the differences. However, we can identify covariates that differed between the 2 samples: parental sex and age, children’s age, federal state (ie, city-state), school and professional education, number of children in the household, socioeconomic status, nationality, relationship status, biological parents, rooms and garden in the household, risk group of severe COVID-19, current or previous mental disorder, and history of own child abuse or neglect. Other studies identified sociodemographic variables as correlates for poorer mental health outcome (eg, female sex, younger parental and children age, parental psychological or physical health conditions before the COVID-19 pandemic, and lower financial background and income or financial stress) [8,11,16,40-42,48-60]. Furthermore, sociodemographic variables were also related to higher parental stress (eg, more children in the household, younger age of the children, and being a single parent) [4,14,17,20,41,46,59], and the occurrence of child maltreatment and parent-child conflicts (eg, younger parental and children age, job and financial loss, living in an urban area, and having low income or financial concerns) [1,2,43,46,47,59,98-100] for parents during the COVID-19 pandemic. Our convenience sample showed a higher presence of individuals with sociodemographic risks (such as more female individuals, younger age, and more and younger children), potentially contributing to greater disparities in parental outcomes due to an overrepresentation of these identified risk factors. Thus, these sociodemographic risk factors may explain differences in parental outcomes.

Strengths

Investigation of the Impact of Sampling

So far, research has largely overlooked the impact of sampling on the informativeness of the results. With this paper, we present one of the few studies that highlight and evaluate the impact of sampling on the results of studies in clinical psychology.

Availability of 2 Studies With the Same Measurements

We were able to use 2 samples in which not only the same outcome measures but also the same explaining variables were measured in exactly the same way. This is a rather rare case, but it was essential for answering the research question. This also allowed us to identify variables explaining these differences. These factors could be assessed in future research on parental outcomes during pandemics to (1) understand which population groups a study provides information for and (2) for understanding differences in study results. Note that while we compared 2 typical sampling approaches—a convenience sample and a specific representative sample, as commonly used by survey institutes—this comparison provided insights into how these approaches lead to different results. However, this type of design and analysis is not limited to these specific sampling methods. The results of any sampling approaches, including 2 different convenience samples, can be compared in this way. The key is that the same variables are assessed in both studies.

Explaining Differences in Results

We not only investigated the differences in results between studies due to different sampling approaches but also aimed to understand why studies yield different results. The collection of a comprehensive set of explaining variables in both studies allowed us to explain most of the differences in study results. This is remarkable, as usually this is often not the case (eg, [31]). In many studies, only a few sociodemographic variables are available that are similarly assessed in the different studies. These usually cannot explain differences in results very well [101]. With sociodemographic data, parent-related risk factors, and COVID-19–related experiences, we had powerful explanatory variables for the differences in study results. If, as in many other studies, we had only limited sociodemographic data, we likely would have explained much less of the differences in the results.

Strategy for Investigating Differences in Results Across Studies

Our paper also presents a strategy for investigating reasons for differences in results across different studies. This strategy can also be applied to compare different sampling strategies in other fields. Thus, researchers may at the same time conduct studies with different types of sampling strategies. This may also be 2 different convenience samples. Relevant explaining variables need to be collected in the same way in both studies to analyze to what extent these can explain differences in outcomes between the 2 studies.

Limitations and Outlook

Limitations of the Comparison of the Studies

Our aim was to investigate the impact of the sampling strategy on the study results on parental outcomes. While we could make use of the rare case of having 2 studies with different sampling strategy that were almost identical in their design and setup, not all design aspects were identical. First, while participants in the convenience sample were interviewed via web surveys (CAWI), parents in the specific representative sample were interviewed either via CAWI or CATIs. This could have led to differences in response behavior, especially with regard to sensitive topics (eg, adverse childhood experiences and parental mental disorder) due to social desirability. Second, the 2 studies partly differed with respect to the incentives they received. Parents in the convenience sample and parents in the specific representative study that were interviewed via CAWI may have been particularly motivated by the prospect of incentives (money, vouchers, and Ipad). In contrast, parents in the specific representative study that were interviewed via telephone did not receive any incentives. This could have also influenced response behavior, leading to differences in study results. In summary, in addition to differences in the sample composition, there are further explanations for differences in the results between the samples, such as different modes of data collection in the representative sample (CATI and CAWI) versus the convenience sample (only CAWI) and different motivations of participants (no incentives for participants in CATI in the quota sample vs incentives for participants in CAWI and the convenience sample). These were not controlled for and must be considered in the final evaluation of the study results. While we could explain a large part of the differences in results between the 2 studies, some unexplained differences remained. These may be due to unobserved explanatory variables, other factors that could account for both sample differences and variations in parental outcomes, or differences in the study designs.

From our study, we cannot necessarily conclude that the specific representative sample results in more valid conclusions on the underlying population than the convenience sample. If selection bias is stronger on relevant variables not controlled for in the specific representative sample as compared to the convenience sample, bias could even be larger in the specific representative study. What we surely can conclude is that the sampling strategy does have a large impact on the study results. For the specific representative study, we can rule out that sample selection due to strata variables biases the results in this study.

Limitations of Single Studies

While both studies were conducted rigorously with the aim of accurately inferring parental outcomes in the population, each study has its own limitations. First, while the 2 studies aimed at drawing inferences on parents with underaged children in Germany, due to practical reasons, they had to restrict the sample to German-speaking parents, which excludes a part of the population [102]. Second, although anonymity was assured by different measures, we cannot rule out that participants answered in a socially desirable way, refraining from reporting, for example, child maltreatment. Though we tried to reduce this, it needs to be kept in mind when interpreting the results. All data were collected and evaluated anonymously. The researchers did not have the contact details of the participants available at any time. This was also explained to the participants in the informed consent form. Overall, our lifetime prevalence rates of child maltreatment are in line with the prevalence rates found in the literature [103]. Third, neither of the 2 samples is globally representative of the general population and is, thus, limited in generalizing to the overall population. While specific representativeness for some variables can be assured for the specific representative study, selection bias may still exist for all other variables. Note that in this specific setting, it would have been possible to weight the convenience sample using census data, thereby achieving representativeness for this sample as well. As compared to the quota sample, specific representativity in the convenience sample would be solely achieved by poststratification. As our aim was to investigate the impact of sampling on study results, we did not consider this here.

Even if their results cannot necessarily be generalized to the underlying population, both the specific representative as well as the convenience samples may provide valuable insights if the sample is well described with respect to variables relevant to the construct under investigation. Adding up information from many studies (that may all be convenience samples) does help to build theory and to describe for which type of persons the theory has been tested for. For this, it is important that relevant person characteristics of the sample are comprehensively described. These should go beyond just some demographic variables and should consider the variables that are relevant to the phenomena of interest. Further, if we know the distribution of these variables in the population, we can infer population-level results from a convenience sample (eg, by weighting the observations in the convenience sample). This approach is similar to how election results are predicted [104].

Generalizability of Our Results

As with any research study, it is not possible to generalize findings from a single investigation to all other studies. Therefore, the results presented in this paper may not necessarily apply to other studies in clinical psychology or those focused on parental outcomes. As in other research, our study provides another piece on the topic that together with other studies adds up to a broader picture. We presume that the results of our study also hold for studies that use similar outcomes to those of our study and recruitment or sampling strategies that do not differ massively from ours. The results of this study may not necessarily be the same for other research topics (specifically other outcomes) and other sample selection mechanisms. In a different study, the selection mechanism of participants in the sample may be different, which leads to different results and possibly also different explaining variables that explain the differences. Different research questions within a study lead to different outcome measures. These are most likely impacted by different explaining variables (covariates). While it is not possible to generalize the findings of this study to other studies, this study shows (1) that sampling strategy can have a large impact on the results and (2) how one can investigate reasons for differences in the results. Relevant covariates that explain differences in study results will most probably be those that are strongly related to the outcomes (researchers usually have a good theory about that from the literature in their field) as well as are relevant for participating in a study (research on this is usually a bit scarce, but pilot studies may help). Assessing these variables as covariates is a promising strategy as it leaves less room for unexplained variation in the outcome [101]. Note that this is not tied to having a quota (ie, a specific representative) sample; it can be done with any 2 studies. The requirement is that the same covariates are assessed in both studies. Future studies may want to incorporate this into their study planning.

Conclusions

To allow readers to judge the representativity of a study, detailed information on sampling and recruitment as well as on generalizability should be provided (eg, constraints on generality [105]).

Our results show that the results of studies differ depending on the sampling strategy. We were able to investigate this. Scientists need to be aware of the fact that from a single study, one cannot directly generalize to the population of interest. Full (global) representative studies are the gold standard in research. However, these are often not feasible for various reasons. Therefore, scientists should collect as many relevant explaining variables (covariates) as possible to describe the sample selection. Such variables are usually those that may have an impact on the results. Collecting these variables makes it possible to understand for which subgroups in the population conclusions can be drawn. As our study shows, if the same explaining variables are assessed in different studies, it is also possible to explain differences in the study results by differences in the group characteristics.

Even if researchers just perform a single study with a specific sampling strategy, we recommend collecting data on potentially explaining variables. This (1) allows to judge for which part of the population the results hold and (2) may allow future studies that assess the same explaining variables to explain differences in their results. If researchers even have information on the distribution of these variables in the general population, they may adjust for sampling bias using sampling weights [31] (eg, [12,13,106]). The choice of explanatory variables is crucial for this.

If possible, scientists may want to consider assessing explaining variables and covariates that are (1) relevant to the outcome under consideration and (2) possibly related to the selection into the study. These will help to identify which piece of the puzzle is actually added to the overall theory by this study. It also helps to integrate results over multiple studies in the future. If they have information on the distribution of these variables in the target population, they may even draw a quota sample or weight their convenience sample to assure specific representativeness. In most cases, it will not be possible to assess all relevant explaining variables. However, considering some relevant explanatory variables can already enhance the robustness of the results.

Acknowledgments

This study was funded by a grant by the Berlin University Alliance to SMW and BR. The funding source Berlin University Alliance had no role in the study design, data collection, analysis, or interpretation of data; the writing of the article; or decision to submit the article for publication.

Data Availability

The datasets generated and analyzed during this study are available from the corresponding author upon reasonable request.

Authors' Contributions

BR and SMW conceived and designed the study. CC, SMW, BR, and LE contributed to the compilation of measures. LE and CC compiled the literature review. LE and SP analyzed the data. LE wrote the first draft of the paper. All authors critically reviewed the manuscript and contributed to data interpretation and discussion. All authors approved the final manuscript for submission.

Conflicts of Interest

BR and SMW have received funding from the Berlin University Alliance for the study. All other authors declare no other conflicts of interest.

Multimedia Appendix 1

The flyer on the recruitment of the convenience sample, a table about the sample composition regarding the Federal State of Germany of the web-based convenience sample and the representative sample, and a detailed table about the logistic regression analysis to predict sample membership through our sociodemographic data, parent-related risk factors, and COVID-19–related experiences.

DOCX File , 321 KB

  1. Gadermann AC, Thomson KC, Richardson CG, Gagné M, McAuliffe C, Hirani S, et al. Examining the impacts of the COVID-19 pandemic on family mental health in Canada: findings from a national cross-sectional study. BMJ Open. Jan 12, 2021;11(1):e042871. [FREE Full text] [CrossRef] [Medline]
  2. Calvano C, Engelke L, Di Bella J, Kindermann J, Renneberg B, Winter SM. Families in the COVID-19 pandemic: parental stress, parent mental health and the occurrence of adverse childhood experiences-results of a representative survey in Germany. Eur Child Adolesc Psychiatry. Jul 2022;31(7):1-13. [FREE Full text] [CrossRef] [Medline]
  3. A L van Tilburg M, Edlynn E, Maddaloni M, van Kempen K, Díaz-González de Ferris M, Thomas J. High levels of stress due to the SARS-CoV-2 pandemic among parents of children with and without chronic conditions across the USA. Children (Basel). Oct 21, 2020;7(10):193. [FREE Full text] [CrossRef] [Medline]
  4. Marchetti D, Fontanesi L, Mazza C, Di Giandomenico S, Roma P, Verrocchio MC. Parenting-related exhaustion during the Italian COVID-19 lockdown. J Pediatr Psychol. Nov 01, 2020;45(10):1114-1123. [FREE Full text] [CrossRef] [Medline]
  5. Rabe JE, Schillok H, Merkel C, Voss S, Coenen M, De Bock F, et al. [Burden in parents of school-aged children during different phases of the COVID-19 pandemic in Germany: an analysis within the COVID-19 snapshot monitoring (COSMO) study]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. Dec 2021;64(12):1500-1511. [FREE Full text] [CrossRef] [Medline]
  6. Alonzo D, Popescu M, Zubaroglu Ioannides P. Mental health impact of the Covid-19 pandemic on parents in high-risk, low income communities. Int J Soc Psychiatry. May 2022;68(3):575-581. [FREE Full text] [CrossRef] [Medline]
  7. Achterberg M, Dobbelaar S, Boer OD, Crone EA. Perceived stress as mediator for longitudinal effects of the COVID-19 lockdown on wellbeing of parents and children. Sci Rep. Feb 03, 2021;11(1):2971. [FREE Full text] [CrossRef] [Medline]
  8. C Fong V, Iarocci G. Child and family outcomes following pandemics: a systematic review and recommendations on COVID-19 policies. J Pediatr Psychol. Nov 01, 2020;45(10):1124-1143. [FREE Full text] [CrossRef] [Medline]
  9. Hamadani JD, Hasan MI, Baldi AJ, Hossain SJ, Shiraji S, Bhuiyan MS, et al. Immediate impact of stay-at-home orders to control COVID-19 transmission on socioeconomic conditions, food insecurity, mental health, and intimate partner violence in Bangladeshi women and their families: an interrupted time series. Lancet Global Health. Nov 2020;8(11):e1380-e1389. [CrossRef]
  10. Racine N, Hetherington E, McArthur BA, McDonald S, Edwards S, Tough S, et al. Maternal depressive and anxiety symptoms before and during the COVID-19 pandemic in Canada: a longitudinal analysis. Lancet Psychiatry. May 2021;8(5):405-415. [FREE Full text] [CrossRef] [Medline]
  11. Paschke K, Arnaud N, Austermann MI, Thomasius R. Risk factors for prospective increase in psychological stress during COVID-19 lockdown in a representative sample of adolescents and their parents. BJPsych Open. May 03, 2021;7(3):e94. [FREE Full text] [CrossRef] [Medline]
  12. Johnson MS, Skjerdingstad N, Ebrahimi OV, Hoffart A, Johnson SU. Parenting in a pandemic: parental stress, anxiety and depression among parents during the government-initiated physical distancing measures following the first wave of COVID-19. Stress Health. Oct 2022;38(4):637-652. [CrossRef] [Medline]
  13. Westrupp EM, Bennett C, Berkowitz T, Youssef GJ, Toumbourou JW, Tucker R, et al. Child, parent, and family mental health and functioning in Australia during COVID-19: comparison to pre-pandemic data. Eur Child Adolesc Psychiatry. Feb 2023;32(2):317-330. [FREE Full text] [CrossRef] [Medline]
  14. Li J, Bünning M, Kaiser T, Hipp L. Who suffered most? Parental stress and mental health during the COVID-19 pandemic in Germany. J Fam Res. Apr 01, 2022;34(1):281-309. [CrossRef]
  15. Kerr ML, Rasmussen HF, Fanning KA, Braaten SM. Parenting during COVID-19: a study of parents' experiences across gender and income levels. Fam Relat. Dec 2021;70(5):1327-1342. [FREE Full text] [CrossRef] [Medline]
  16. Malkawi SH, Almhdawi K, Jaber AF, Alqatarneh NS. COVID-19 quarantine-related mental health symptoms and their correlates among mothers: a cross sectional study. Matern Child Health J. May 2021;25(5):695-705. [FREE Full text] [CrossRef] [Medline]
  17. Tchimtchoua Tamo AR. An analysis of mother stress before and during COVID-19 pandemic: the case of China. Health Care Women Int. 2020;41(11-12):1349-1362. [CrossRef] [Medline]
  18. Kurata S, Hiraoka D, Ahmad Adlan AS, Jayanath S, Hamzah N, Ahmad-Fauzi A, et al. Influence of the COVID-19 pandemic on parenting stress across Asian countries: a cross-national study. Front Psychol. 2021;12:782298. [FREE Full text] [CrossRef] [Medline]
  19. Adams EL, Smith D, Caccavale LJ, Bean MK. Parents are stressed! Patterns of parent stress across COVID-19. Front Psychiatry. Apr 8, 2021;12:626456. [FREE Full text] [CrossRef] [Medline]
  20. Miller JJ, Cooley ME, Mihalec-Adkins BP. Examining the impact of COVID-19 on parental stress: a study of foster parents. Child Adolesc Social Work J. Nov 20, 2022;39(2):147-156. [FREE Full text] [CrossRef] [Medline]
  21. Karbasi Z, Safdari R, Eslami P. The silent crisis of child abuse in the COVID-19 pandemic: a scoping review. Health Sci Rep. Sep 2022;5(5):e790. [FREE Full text] [CrossRef] [Medline]
  22. Rodriguez CM, Lee SJ, Ward KP, Pu DF. The perfect storm: hidden risk of child maltreatment during the Covid-19 pandemic. Child Maltreat. May 23, 2021;26(2):139-151. [FREE Full text] [CrossRef] [Medline]
  23. Callegaro M, Baker R, Bethlehem J, Göritz AS, Krosnick JA, Lavrakas PJ. Online panel research. In: Callegaro M, Baker R, Bethlehem J, Göritz AS, Krosnick JA, Lavrakas PJ, editors. Online Panel Research: A Data Quality Perspective. Hoboken, NJ. John Wiley & Sons; 2014.
  24. Zhao K. Sample representation in the social sciences. Synthese. Mar 13, 2020;198:9097-9115. [CrossRef]
  25. Lehdonvirta V, Oksanen A, Räsänen P, Blank G. Social media, web, and panel surveys: using non‐probability samples in social and policy research. Policy Internet. Apr 29, 2020;13(1):134-155. [CrossRef]
  26. Hlatshwako TG, Shah SJ, Kosana P, Adebayo E, Hendriks J, Larsson EC, et al. Online health survey research during COVID-19. Lancet Digit Health. Feb 2021;3(2):e76-e77. [CrossRef]
  27. Spinelli M, Lionetti F, Setti A, Fasolo M. Parenting stress during the COVID-19 outbreak: socioeconomic and environmental risk factors and implications for children emotion regulation. Fam Process. Jun 2021;60(2):639-653. [CrossRef] [Medline]
  28. Johnson MS, Skjerdingstad N, Ebrahimi OV, Hoffart A, Urnes Johnson S. Mechanisms of parental distress during and after the first COVID-19 lockdown phase: a two-wave longitudinal study. PLoS One. Jun 24, 2021;16(6):e0253087. [FREE Full text] [CrossRef] [Medline]
  29. Yeager DS, Krosnick JA, Chang L, Javitz HS, Levendusky MS, Simpser A, et al. Comparing the accuracy of RDD telephone surveys and internet surveys conducted with probability and non-probability samples. Public Opin Q. Oct 05, 2011;75(4):709-747. [CrossRef]
  30. Chang L, Krosnick JA. National surveys via RDD telephone interviewing versus the internet: comparing sample representativeness and response quality. Public Opin Q. Dec 01, 2009;73(4):641-678. [CrossRef]
  31. Cornesse C, Blom AG, Dutwin D, Krosnick JA, De Leeuw ED, Legleye S, et al. A review of conceptual approaches and empirical evidence on probability and nonprobability sample survey research. J Surv Stat Methodol. Jan 09, 2020;8(1):4-36. [CrossRef]
  32. Blasius J, Brandt M. Repräsentativität in online-befragungen. In: Weichbold M, Bacher J, Wolf C, editors. Umfrageforschung. Freiburg im Breisgau, Germany. VS Verlag für Sozialwissenschaften; 2009.
  33. Whitaker C, Stevelink S, Fear N. The use of Facebook in recruiting participants for health research purposes: a systematic review. J Med Internet Res. Aug 28, 2017;19(8):e290. [FREE Full text] [CrossRef] [Medline]
  34. Johnson LA, Dias N, Clarkson G, Schreier AM. Social media as a recruitment method to reach a diverse sample of bereaved parents. Appl Nurs Res. Dec 2019;50:151201. [CrossRef] [Medline]
  35. Oesterle S, Epstein M, Haggerty KP, Moreno MA. Using Facebook to recruit parents to participate in a family program to prevent teen drug use. Prev Sci. May 2018;19(4):559-569. [FREE Full text] [CrossRef] [Medline]
  36. Skeens MA, Sutherland-Foggio M, Damman C, Gerhardt CA, Akard TF. Facebook recruitment for research of children and parents during the COVID-19 pandemic. Appl Nurs Res. Jun 2022;65:151574. [FREE Full text] [CrossRef] [Medline]
  37. Yaremych HE, Persky S. Recruiting fathers for parenting research: an evaluation of eight recruitment methods and an exploration of fathers' motivations for participation. Parent Sci Pract. 2023;23(1):1-32. [FREE Full text] [CrossRef] [Medline]
  38. Dworkin J, Hessel H, Gliske K, Rudi JH. A comparison of three online recruitment strategies for engaging parents. Fam Relat. Oct 2016;65(4):550-561. [FREE Full text] [CrossRef] [Medline]
  39. Leach LS, Butterworth P, Poyser C, Batterham PJ, Farrer LM. Online recruitment: feasibility, cost, and representativeness in a study of postpartum women. J Med Internet Res. Mar 08, 2017;19(3):e61. [FREE Full text] [CrossRef] [Medline]
  40. Crescentini C, Feruglio S, Matiz A, Paschetto A, Vidal E, Cogo P, et al. Stuck outside and inside: an exploratory study on the effects of the COVID-19 outbreak on Italian parents and children's internalizing symptoms. Front Psychol. Oct 22, 2020;11:586074. [FREE Full text] [CrossRef] [Medline]
  41. Cusinato M, Iannattone S, Spoto A, Poli M, Moretti C, Gatta M, et al. Stress, resilience, and well-being in Italian children and their parents during the COVID-19 pandemic. Int J Environ Res Public Health. Nov 10, 2020;17(22):8297. [FREE Full text] [CrossRef] [Medline]
  42. Mousavi SF. Psychological well-being, marital satisfaction, and parental burnout in Iranian parents: the effect of home quarantine during COVID-19 outbreaks. Front Psychol. Dec 3, 2020;11:553880. [FREE Full text] [CrossRef] [Medline]
  43. Wolf JP, Freisthler B, Chadwick C. Stress, alcohol use, and punitive parenting during the COVID-19 pandemic. Child Abuse Negl. Jul 2021;117:105090. [FREE Full text] [CrossRef] [Medline]
  44. Yamaoka Y, Hosozawa M, Sampei M, Sawada N, Okubo Y, Tanaka K, et al. Abusive and positive parenting behavior in Japan during the COVID-19 pandemic under the state of emergency. Child Abuse Negl. Oct 2021;120:105212. [FREE Full text] [CrossRef] [Medline]
  45. Thorell LB, Skoglund C, de la Peña AG, Baeyens D, Fuermaier AB, Groom MJ, et al. Parental experiences of homeschooling during the COVID-19 pandemic: differences between seven European countries and between children with and without mental health conditions. Eur Child Adolesc Psychiatry. Apr 2022;31(4):649-661. [FREE Full text] [CrossRef] [Medline]
  46. Brown SM, Doom JR, Lechuga-Peña S, Watamura SE, Koppels T. Stress and parenting during the global COVID-19 pandemic. Child Abuse Negl. Dec 2020;110(Pt 2):104699. [FREE Full text] [CrossRef] [Medline]
  47. Lee SJ, Ward KP, Lee JY, Rodriguez CM. Parental social isolation and child maltreatment risk during the COVID-19 pandemic. J Fam Violence. Jan 14, 2021;37(5):813-824. [FREE Full text] [CrossRef] [Medline]
  48. Fancourt D, Steptoe A, Bu F. Trajectories of depression and anxiety during enforced isolation due to COVID-19: longitudinal analyses of 59,318 adults in the UK with and without diagnosed mental illness. medRxiv. Preprint posted online on November 3, 2020. [CrossRef]
  49. Fitzpatrick KM, Drawve G, Harris C. Facing new fears during the COVID-19 pandemic: the State of America's mental health. J Anxiety Disord. Oct 2020;75:102291. [FREE Full text] [CrossRef] [Medline]
  50. Malesza M, Kaczmarek MC. Predictors of anxiety during the COVID-19 pandemic in Poland. Pers Individ Dif. Feb 15, 2021;170:110419. [FREE Full text] [CrossRef] [Medline]
  51. Benassi E, Vallone M, Camia M, Scorza M. Women during the Covid-19 lockdown: more anxiety symptoms in women with children than without children and role of the resilience. Mediterr J Clin Psychol. 2020;8(3). [CrossRef]
  52. Zamarro G, Prados MJ. Gender differences in couples' division of childcare, work and mental health during COVID-19. Rev Econ Househ. 2021;19(1):11-40. [FREE Full text] [CrossRef] [Medline]
  53. Cheng Z, Mendolia S, Paloyo AR, Savage DA, Tani M. Working parents, financial insecurity, and childcare: mental health in the time of COVID-19 in the UK. Rev Econ Househ. 2021;19(1):123-144. [FREE Full text] [CrossRef] [Medline]
  54. Huebener M, Waights S, Spiess CK, Siegel NA, Wagner GG. Parental well-being in times of Covid-19 in Germany. Rev Econ Househ. 2021;19(1):91-122. [FREE Full text] [CrossRef] [Medline]
  55. Mazza C, Ricci E, Marchetti D, Fontanesi L, Di Giandomenico S, Verrocchio MC, et al. How personality relates to distress in parents during the Covid-19 lockdown: the mediating role of child's emotional and behavioral difficulties and the moderating effect of living with other people. Int J Environ Res Public Health. Aug 27, 2020;17(17):6236. [FREE Full text] [CrossRef] [Medline]
  56. Patrick SW, Henkhaus LE, Zickafoose JS, Lovell K, Halvorson A, Loch S, et al. Well-being of parents and children during the COVID-19 pandemic: a national survey. Pediatrics. Oct 2020;146(4):e2020016824. [CrossRef] [Medline]
  57. Yue J, Zang X, Le Y, An Y. Anxiety, depression and PTSD among children and their parent during 2019 novel coronavirus disease (COVID-19) outbreak in China. Curr Psychol. 2022;41(8):5723-5730. [FREE Full text] [CrossRef] [Medline]
  58. Ren J, Li X, Chen S, Chen S, Nie Y. The influence of factors such as parenting stress and social support on the state anxiety in parents of special needs children during the COVID-19 epidemic. Front Psychol. Dec 10, 2020;11:565393. [FREE Full text] [CrossRef] [Medline]
  59. Russell BS, Hutchison M, Tambling R, Tomkunas AJ, Horton AL. Initial challenges of caregiving during COVID-19: caregiver burden, mental health, and the parent-child relationship. Child Psychiatry Hum Dev. Oct 2020;51(5):671-682. [FREE Full text] [CrossRef] [Medline]
  60. Mazza M, Marano G, Lai C, Janiri L, Sani G. Danger in danger: interpersonal violence during COVID-19 quarantine. Psychiatry Res. Jul 2020;289:113046. [FREE Full text] [CrossRef] [Medline]
  61. Bramstedt KA. The carnage of substandard research during the COVID-19 pandemic: a call for quality. J Med Ethics. Dec 2020;46(12):803-807. [CrossRef] [Medline]
  62. Holmes EA, O'Connor RC, Perry VH, Tracey I, Wessely S, Arseneault L, et al. Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry. Jun 2020;7(6):547-560. [FREE Full text] [CrossRef] [Medline]
  63. Kennedy EB, Jensen EA, Jensen AM. Methodological considerations for survey-based research during emergencies and public health crises: improving the quality of evidence and communication. Front Commun. Feb 15, 2022;6:1-7. [CrossRef]
  64. Singh S, Sagar R. A critical look at online survey or questionnaire-based research studies during COVID-19. Asian J Psychiatr. Nov 2021;65:102850. [FREE Full text] [CrossRef] [Medline]
  65. Zhao Q, Ju N, Bacallado S, Shah RD. BETS: the dangers of selection bias in early analyses of the coronavirus disease (COVID-19) pandemic. Ann Appl Stat. Mar 1, 2021;15(1):363-390. [CrossRef]
  66. Pierce M, McManus S, Jessop C, John A, Hotopf M, Ford T, et al. Says who? The significance of sampling in mental health surveys during COVID-19. Lancet Psychiatry. Jul 2020;7(7):567-568. [CrossRef]
  67. Dutwin D, Buskirk TD. Apples to oranges or gala versus golden delicious? Comparing data quality of nonprobability internet samples to low response rate probability samples. Public Opin Q. 2017;81(S1):213-239. [CrossRef]
  68. Szolnoki G, Hoffmann D. Online, face-to-face and telephone surveys—comparing different sampling methods in wine consumer research. Wine Econ Policy. Dec 2013;2(2):57-66. [CrossRef]
  69. Thornton L, Batterham PJ, Fassnacht DB, Kay-Lambkin F, Calear AL, Hunt S. Recruiting for health, medical or psychosocial research using Facebook: systematic review. Internet Interv. Apr 27, 2016;4:72-81. [FREE Full text] [CrossRef] [Medline]
  70. Duda MD, Nobile JL. The fallacy of online surveys: no data are better than bad data. Hum Dimens Wildl. Feb 08, 2010;15(1):55-64. [CrossRef]
  71. Newman A, Bavik YL, Mount M, Shao B. Data collection via online platforms: challenges and recommendations for future research. Appl Psychol. Jul 2021;70(3):1380-1402. [CrossRef]
  72. Kohler U, Kreuter F, Stuart EA. Nonprobability sampling and causal analysis. Annu Rev Stat Appl. 2019;6:149-172. [CrossRef]
  73. Groves RM. Nonresponse rates and nonresponse bias in household surveys. Public Opin Q. Jan 01, 2006;70(5):646-675. [CrossRef]
  74. Martin E, Winters F. Money and motive: effects of incentives on panel attrition in the survey of income and program participation. J Off Stat. Jun 2001;17(2):267-284.
  75. Zhang C, Lonn S, Teasley SD. Understanding the impact of lottery incentives on web survey participation and response quality: a leverage-salience theory perspective. Field Methods. 2017;29(1):42-60. [CrossRef]
  76. Bethlehem J. Selection bias in web surveys. Int Stat Rev. Aug 03, 2010;78(2):161-188. [CrossRef]
  77. Choi I, Milne DN, Glozier N, Peters D, Harvey SB, Calvo RA. Using different Facebook advertisements to recruit men for an online mental health study: engagement and selection bias. Internet Interv. Jun 2017;8:27-34. [FREE Full text] [CrossRef] [Medline]
  78. Batterham PJ. Recruitment of mental health survey participants using internet advertising: content, characteristics and cost effectiveness. Int J Methods Psychiatr Res. Jun 2014;23(2):184-191. [FREE Full text] [CrossRef] [Medline]
  79. Joyal-Desmarais K, Stojanovic J, Kennedy EB, Enticott JC, Boucher VG, Vo H, et al. How well do covariates perform when adjusting for sampling bias in online COVID-19 research? Insights from multiverse analyses. Eur J Epidemiol. Dec 2022;37(12):1233-1250. [FREE Full text] [CrossRef] [Medline]
  80. Bennetts SK, Hokke S, Crawford S, Hackworth NJ, Leach LS, Nguyen C, et al. Using paid and free Facebook methods to recruit Australian parents to an online survey: an evaluation. J Med Internet Res. Mar 06, 2019;21(3):e11206. [FREE Full text] [CrossRef] [Medline]
  81. Lampert T, Müters S, Stolzenberg H, Kroll LE, KiGGS Study Group. [Measurement of socioeconomic status in the KiGGS study: first follow-up (KiGGS Wave 1)]. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. Jul 2014;57(7):762-770. [CrossRef] [Medline]
  82. Finlay I, Gilmore I. Covid-19 and alcohol-a dangerous cocktail. BMJ. May 20, 2020;369:m1987. [CrossRef] [Medline]
  83. Löwe B, Spitzer RL, Zipfel S, Herzog W. Gesundheitsfragebogen für Patienten (PHQ-D). Medizinische Universitätsklinik. 2002. URL: https:/​/www.​klinikum.uni-heidelberg.de/​fileadmin/​medizinische_klinik/​Abteilung_2/​pdf/​Komplett_PHQ_Fragebogen.​pdf [accessed 2024-12-19]
  84. Berry JO, Jones WH. The parental stress scale: initial psychometric evidence. J Soc Pers Relatsh. Aug 01, 1995;12(3):463-472. [CrossRef]
  85. Benyamini Y, Leventhal EA, Leventhal H. Self-assessments of health: what do people know that predicts their mortality? Res Aging. May 01, 1999;21(3):477-500. [CrossRef]
  86. Löwe B, Wahl I, Rose M, Spitzer C, Glaesmer H, Wingenfeld K, et al. A 4-item measure of depression and anxiety: validation and standardization of the Patient Health Questionnaire-4 (PHQ-4) in the general population. J Affect Disord. Apr 2010;122(1-2):86-95. [CrossRef] [Medline]
  87. Teicher MH, Parigger A. The 'Maltreatment and Abuse Chronology of Exposure' (MACE) scale for the retrospective assessment of abuse and neglect during development. PLoS One. Feb 25, 2015;10(2):e0117423. [FREE Full text] [CrossRef] [Medline]
  88. Hecker T, Boettcher VS, Landolt MA, Hermenau K. Child neglect and its relation to emotional and behavioral problems: a cross-sectional study of primary school-aged children in Tanzania. Dev Psychopathol. Mar 26, 2018;31(1):325-339. [CrossRef]
  89. Isele D, Teicher MH, Ruf-Leuschner M, Elbert T, Kolassa IT, Schury K, et al. KERF–ein instrument zur umfassenden ermittlung belastender kindheitserfahrungen. Zeitschrift Klinische Psychologie Psychotherapie. Apr 2014;43(2):121-130. [CrossRef]
  90. Runyan DK, Cox CE, Dubowitz H, Newton RR, Upadhyaya M, Kotch JB, et al. Describing maltreatment: do child protective service reports and research definitions agree? Child Abuse Negl. May 2005;29(5):461-477. [CrossRef] [Medline]
  91. Manly JT, Cicchetti D, Barnett D. The impact of subtype, frequency, chronicity, and severity of child maltreatment on social competence and behavior problems. Dev Psychopathol. Oct 31, 2008;6(1):121-143. [CrossRef]
  92. Cabrera NJ, Volling BL, Barr R. Fathers are parents, too! Widening the lens on parenting for children's development. Child Dev Perspectives. Jan 08, 2018;12(3):152-157. [CrossRef]
  93. Costigan CL, Cox MJ. Fathers' participation in family research: is there a self-selection bias? J Fam Psychol. 2001;15(4):706-720. [CrossRef]
  94. Davison KK, Charles JN, Khandpur N, Nelson TJ. Fathers' perceived reasons for their underrepresentation in child health research and strategies to increase their involvement. Matern Child Health J. Feb 2017;21(2):267-274. [FREE Full text] [CrossRef] [Medline]
  95. Bartholomew MK, Schoppe-Sullivan SJ, Glassman M, Kamp Dush CM, Sullivan JM. New parents' Facebook use at the transition to parenthood. Fam Relat. Jul 2012;61(3):455-469. [FREE Full text] [CrossRef] [Medline]
  96. Duggan M, Lenhart A, Lampe C, Ellison NB. Parents and social media. Pew Research Center. Jul 16, 2015. URL: https://www.pewresearch.org/internet/2015/07/16/parents-and-social-media/ [accessed 2024-12-19]
  97. Leach LS, Bennetts SK, Giallo R, Cooklin AR. Recruiting fathers for parenting research using online advertising campaigns: evidence from an Australian study. Child Care Health Dev. Nov 22, 2019;45(6):871-876. [CrossRef] [Medline]
  98. Lawson M, Piel MH, Simon M. Child maltreatment during the COVID-19 pandemic: consequences of parental job loss on psychological and physical abuse towards children. Child Abuse Negl. Dec 2020;110(Pt 2):104709. [FREE Full text] [CrossRef] [Medline]
  99. Beland LP, Brodeur A, Haddad J, Mikola D. Covid-19, family stress and domestic violence: remote work, isolation and bargaining power. IZA Institute of Labor Economics. Jun 5, 2020. URL: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3627031 [accessed 2024-12-19]
  100. Machlin L, Gruhn MA, Miller AB, Milojevich HM, Motton S, Findley AM, et al. Predictors of family violence in North Carolina following initial COVID-19 stay-at-home orders. Child Abuse Negl. Aug 2022;130(Pt 1):105376. [FREE Full text] [CrossRef] [Medline]
  101. Cook TD, Steiner PM, Pohl S. How bias reduction is affected by covariate choice, unreliability, and mode of data analysis: results from two types of within-study comparisons. Multivariate Behav Res. Nov 30, 2009;44(6):828-847. [CrossRef] [Medline]
  102. Gelebte vielfalt: familien mit migrationshintergrund in Deutschland. Bundesministerium für Familie, Senioren, Frauen und Jugend. 2017. URL: https:/​/www.​bmfsfj.de/​resource/​blob/​116880/​83c02ec19dbea15014d7868048f697f2/​gelebte-vielfalt--familien-mit-migrationshintergrund-in-deutschland-data.​pdf [accessed 2024-12-19]
  103. Witt A, Brown RC, Plener PL, Brähler E, Fegert JM. Child maltreatment in Germany: prevalence rates in the general population. Child Adolesc Psychiatry Ment Health. Sep 29, 2017;11(1):47. [FREE Full text] [CrossRef] [Medline]
  104. Wang W, Rothschild D, Goel S, Gelman A. Forecasting elections with non-representative polls. Int J Forecast. Jul 2015;31(3):980-991. [FREE Full text] [CrossRef]
  105. Simons DJ, Shoda Y, Lindsay DS. Constraints on generality (COG): a proposed addition to all empirical papers. Perspect Psychol Sci. Nov 2017;12(6):1123-1128. [CrossRef] [Medline]
  106. Neuhauser H, Rosario AS, Butschalowsky H, Haller S, Hoebel J, Michel J, et al. Nationally representative results on SARS-CoV-2 seroprevalence and testing in Germany at the end of 2020. Sci Rep. Nov 14, 2022;12(1):19492. [FREE Full text] [CrossRef] [Medline]


ANCOVA: analyses of covariance
CATI: computer-assisted telephone interview
CAWI: computer-assisted web interview
NEA: nonverbal emotional abuse
VEA: verbal emotional abuse
WDV: witnessing domestic violence


Edited by A Coristine; submitted 24.08.23; peer-reviewed by S Gordon, B Rodgers; comments to author 11.01.24; revised version received 03.07.24; accepted 20.09.24; published 24.01.25.

Copyright

©Lara Engelke, Claudia Calvano, Steffi Pohl, Sibylle Maria Winter, Babette Renneberg. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 24.01.2025.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.