This paper is in the following e-collection/theme issue:

Quality/Credibility of eHealth Information and Trust Issues (492) Perinatal Education for Parents (210) Information Quality in Digital Media (318) Information Seeking, Information Needs (426) Information Retrieval (271)

Published on in Vol 24, No 10 (2022): October

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/38641, first published .
Interactivity, Quality, and Content of Websites Promoting Health Behaviors During Infancy: 6-Year Update of the Systematic Assessment

Interactivity, Quality, and Content of Websites Promoting Health Behaviors During Infancy: 6-Year Update of the Systematic Assessment

Interactivity, Quality, and Content of Websites Promoting Health Behaviors During Infancy: 6-Year Update of the Systematic Assessment

Authors of this article:

Danielle Jawad1, 2, 3 Author Orcid Image ;   Heilok Cheng3, 4, 5 Author Orcid Image ;   Li Ming Wen1, 2, 3, 5 Author Orcid Image ;   Chris Rissel1, 6 Author Orcid Image ;   Louise Baur1, 3, 7 Author Orcid Image ;   Seema Mihrshahi8 Author Orcid Image ;   Sarah Taki1, 2, 3, 5 Author Orcid Image
Article Authors Cited by (6) Tweetations (19) Metrics

    Original Paper

    1Sydney School of Public Health, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia

    2Health Promotion Unit, Population Health Research & Evaluation Hub, Sydney Local Health District, Sydney, Australia

    3National Health and Medical Research Council Centre of Research Excellence in the Early Prevention of Obesity in Childhood - Translate, The University of Sydney, Sydney, Australia

    4Susan Wakil School of Nursing and Midwifery, Faculty of Medicine and Health, The University of Sydney, Sydney, Australia

    5Sydney Institute for Women, Children and their Families, Sydney Local Health District, Sydney, Australia

    6College of Medicine and Public Health, Rural and Remote Health South Australia and Northern Territory, Flinders University, Darwin, Australia

    7Specialty of Child and Adolescent Health, Sydney Medical School, The University of Sydney, Sydney, Australia

    8Department of Health Sciences, Faculty of Medicine, Health and Human Sciences, Macquarie University, Sydney, Australia

    Corresponding Author:

    Danielle Jawad, BSc, MPH

    Sydney School of Public Health

    Faculty of Medicine and Health

    The University of Sydney

    A27 Fisher road, Camperdown

    Sydney, 2006

    Australia

    Phone: 61 2 9515 9895

    Email: danielle.jawad@sydney.edu.au


    Background: As of 2021, 89% of the Australian population are active internet users. Although the internet is widely used, there are concerns about the quality, accuracy, and credibility of health-related websites. A 2015 systematic assessment of infant feeding websites and apps available in Australia found that 61% of websites were of poor quality and readability, with minimal coverage of infant feeding topics and lack of author credibility.

    Objective: We aimed to systematically assess the quality, interactivity, readability, and comprehensibility of information targeting infant health behaviors on websites globally and provide an update of the 2015 systematic assessment.

    Methods: Keywords related to infant milk feeding behaviors, solid feeding behaviors, active play, screen time, and sleep were used to identify websites targeting infant health behaviors on the Google search engine on Safari. The websites were assessed by a subset of the authors using predetermined criteria between July 2021 and February 2022 and assessed for information content based on the Australian Infant Feeding Guidelines and National Physical Activity Recommendations. The Suitability Assessment of Materials, Quality Component Scoring System, the Health-Related Website Evaluation Form, and the adherence to the Health on the Net code were used to evaluate the suitability and quality of information. Readability was assessed using 3 web-based readability tools.

    Results: Of the 450 websites screened, 66 were included based on the selection criteria and evaluated. Overall, the quality of websites was mostly adequate. Media-related sources, nongovernmental organizations, hospitals, and privately owned websites had the highest median quality scores, whereas university websites received the lowest median score (35%). The information covered within the websites was predominantly poor: 91% (60/66) of the websites received an overall score of ≤74% (mean 53%, SD 18%). The suitability of health information was mostly rated adequate for literacy demand, layout, and learning and motivation of readers. The median readability score for the websites was grade 8.5, which is higher than the government recommendations (<grade 8). Overall, 74% (49/66) of the websites obtained a poor rating for interactivity, measuring active control, 2-way communication, and synchronicity. The most common features found on websites were social media links (61/66, 92%), frequently asked questions (48/66, 73%), and videos (44/66, 67%). Only 14% (9/66) of websites presented culturally responsive information.

    Conclusions: Quality, content, readability, and interactivity of websites promoting health behaviors during infancy ranged between poor and adequate. Since the 2015 systematic assessment, there was a slight improvement in the quality of websites but no difference in the Suitability Assessment of Materials rating and readability of information. There is a need for researchers and health care providers to leverage innovative web-based platforms to provide culturally competent evidence-based information based on government guidelines that are accessible to those with limited English proficiency.

    J Med Internet Res 2022;24(10):e38641

    doi:10.2196/38641

    Keywords

    breastfeeding; bottle feeding; websites; web-based platform; infant food; readability; accuracy; consumer; health information; interactivity; solid food; quality; grading; comprehensibility; infant; baby; babies; feeding; food; eating; nutrition; health behavior; web-based information; health website; sleep; screen time; rating 


    Background

    With technological advances and developments, internet access continues to increase [1]. Globally, approximately 4.53 billion people have access to web portals [2,3], with more than half using mobile devices and 38.5% using desktop computers to access the internet worldwide [4]. Increasingly, internet users are making use of the availability of web-based resources, with approximately 4.5% of all internet searches looking for health-related information [5-7]. Recently, nationwide lockdowns, social distancing, and restrictions due to the COVID-19 pandemic have led to an inevitable surge in internet use among individuals, including parents of young children, to seek health information on the web [8,9] and by health care practitioners to assist in service delivery [10,11]. Given that the internet offers considerable opportunities for immediate and easy access to web-based resources, it has become a significant medium for the dissemination of health-related information.

    A universal growing demand for, and use of, web-based resources related to child health information is evident [12-15]. The consolidated behavior of using web-based resources for health information is particularly common among new and expecting parents, where they most frequently search the internet for information related to infant nutrition, development, social support, and health symptoms [16-22]. A 2016 Australian survey found that more than 73% of parents with children aged <5 years used websites and web-based forums to access child health–related information [23]. Interestingly, 30% of those parents reported not trusting the information sources [23]. Another recent study in Switzerland showed that 91% of parents with at least one child aged <2 years used the internet to search for information related to their child’s health and development [24]. The most frequently used sources reported were search engines (55%) and websites for parents (47%) [24]. Although the majority used the internet to search for health-related information, a large percentage of parents were skeptical about the trustworthiness of the web-based resources and their ability to correctly interpret the reliability of the health information they found [24]. This highlights the need for, and importance of, accessible websites to present health information accurately while ensuring it can be easily understood by their intended users.

    This study updates and expands on a 2015 systematic assessment of infant feeding websites and mobile apps available in Australia [25]. The update of apps has been recently conducted by Cheng et al [26] in 2020. Hence, this study focused on updating and expanding the assessment of websites globally. The 2015 assessment found that 61% of Australian websites were of poor quality, with minimal coverage of infant feeding topics, lack of author credibility, and abstruse readability of content [25]. Since the publication of the 2015 systematic assessment, several other website assessments reviewing information related to infant health behaviors have identified similar findings [27,28]. A Korean study reported that websites were scored poorly when evaluated for availability, quality, and reliability of infant health information on the web [27]. Moreover, a recent analysis of 197 websites addressing preterm infants’ health information also found that the overall quality of websites was low to moderate in terms of reliability and content [28]. Provision of inadequate or incomplete infant health information on the web could result in parental confusion, apprehension, and poorer care for infants when parents are unable to evaluate the accuracy and credibility of the web-based information.

    Over the past 10 years, websites have evolved from static read only to a more interactive and fully immersive experience [29,30]. Since the 2015 assessment of websites [25], there has been a marked growth in bandwidth levels enabling the design of more sophisticated websites to offer consumer-oriented health information in various interactive ways, such as videos, parent forums, podcasts, and multilingual options that have provided a context to support culturally diverse people across the world [31-38]. In addition, the emergence of artificial intelligence and machine learning since 2016 has given rise to chatbot technology that simulates human-like conversations to provide consumers with support and relevant information [39]. This has been tested and proven to be successful among parents of young infants searching for information related to infant sleep and feeding practices [40].

    Objective

    With more parents resorting to web-based sources to seek infant health information, it is imperative for resources on the internet to reflect the latest infant and child health guidelines. Therefore, the aim of this study was to update the 2015 systematic assessment of websites [25] by evaluating the content, suitability, readability, comprehensibility, and quality of information targeting infant nutrition, active play, screen time, and sleep behaviors on websites globally. In addition, this review expanded the 2015 systematic assessment [25] by examining interactivity, features, and cultural considerations of the websites.


    Study Design

    A systematic search and assessment were conducted to identify and evaluate websites targeting infant feeding, active play, screen time, and sleep behaviors between July 2021 and February 2022. As shown in Table 1, a range of validated tools was used to assess the selected websites. Details of the evaluation tools are described in Multimedia Appendix 1, and the details of the methods are given in Multimedia Appendix 2.

    Table 1. Comparison of the systematic assessment between the 2015 assessment and this study.
    CriteriaSystematic assessment

    		20152021
    Website selection

    		Australian websites only✓

    		Global websites including Australian websites
    		✓
    Topic areas

    		Milk feeding practices (breastfeeding and formula)✓✓

    		Solid feeding behaviors✓✓

    		Infant active play
    		✓

    		Infant screen time
    		✓

    		Infant sleep
    		✓
    Scope, accuracy, and depth of information

    		Excel spreadsheet built with an assessment criterion of 8 topics and 22 subtopics✓

    		Comprehensive REDCapa tool built with an assessment criterion of 9 topics and 65 subtopics
    		✓
    Quality assessment

    		Quality Component Scoring System✓✓

    		Health-Related Website Evaluation Form✓✓

    		Adherence to the Health on the Net code✓✓
    Suitability of information

    		The Suitability Assessment of Material✓✓
    Readability

    		Flesch-Kincaid✓✓

    		Simple Measure of Gobbledygook✓✓

    		Consensus based on 7 readability formulas
    		✓
    Website interactivity and features

    		The interactivity scale (15 items)
    		✓

    		Interactive features on websites
    		✓

    		Addresses culture
    		✓

    aREDCap: Research Electronic Data Capture.

    Stage 1: Website Selection

    Overview

    Websites were identified using the Chrome browser and Google search engine. All cookies and search history were erased from the web browser to ensure no previous web-based activities influenced the search results. The search terms were Infant feeding, Baby food, Breast feeding, Infant feeding to appetite, Infant formula feeding, Introducing solid foods to baby, Good foods to start baby with no teeth, Best puree for babies, Solids and fussy babies, Solids and milk feeding, Infant active play, Tummy time, Screen time, Infant sleep, Baby co-sleep. These key terms were identified from questions asked in Facebook groups that consisted of parents with infants and from “related searches” on Google, which was used as a cross-reference to ensure the representativeness of the keywords.

    Evidence shows that users concentrate their exploration of websites on the first 10 search results retrieved from a search engine and rarely go beyond the first 2 pages [41]. Hence, the first 30 websites generated from every search term were screened.

    Inclusion Criteria

    We included global websites that used English as a primary language or language option, were free of charge, targeted at parents of infants, and contained information on at least one of the following topics: milk feeding behaviors (breastfeeding, formula feeding, expressing breast milk, feeding to appetite, frequency or timing of feeding, and correct preparation of infant formula, storage of milk, quantity of milk, and transport of milk), solid food feeding behaviors (age of introduction, types of food introduced, and food allergies), infant activity (“tummy time,” infant play, and movement), and infant screen time or infant sleep (bedtime routine, recommended hours of sleep, and cosleep) regardless of whether the websites addressed other content or age groups.

    Exclusion Criteria

    We excluded websites that had no information on one of the topics of interest listed in the inclusion criteria; were newspapers; were electronic books; required a password; had a payment fee; or had a link that redirected user to a scientific article, podcast, or downloadable Microsoft Word document and PDF document.

    The first author screened all the websites for eligibility using predefined inclusion and exclusion criteria. Any uncertainties or disagreements regarding the inclusion of websites in the study were cross-checked by researchers ST, LMW, LB, and CR in a group meeting and discussed until consensus was reached.

    Stage 2: Website Evaluation

    Scope, Accuracy, and Depth of Information

    Scope, accuracy, and depth of information were evaluated using a newly built tool on a password-protected database (REDCap [Research Electronic Data Capture]; Vanderbilt University) that was based on the Australian government’s guidelines on infant feeding [42], physical activity [43], and sleep [44]. The tool consists of 11 broad topics with 65 subtopics on encouraging and supporting breastfeeding; initiating, establishing, and maintaining breastfeeding; management of common breastfeeding problems; expressing and storing breast milk; breastfeeding in specific situations; infant formula; solid food introduction; encouraging infant active play; screen time; and infant sleep behaviors. Each subtopic was scored as correct (+1), incorrect (−1), not addressed (0), or not applicable (which was not counted in the denominator of the overall score). For subtopics that were partially addressed, a partially complete (+0.5) score was given. A summary section score was automatically calculated for each topic, and a final overall score was generated after the assessment of all the content for the 11 topic areas. Overall scores were summarized as excellent (≥90%), adequate (75%-89%), or poor (≤74%) using the criteria from the Health-Related Website Evaluation Form (HRWEF) [45] similar to the updated assessment of apps [26].

    Website Quality

    Website quality was evaluated using the same validated tools as the 2015 assessment: the Quality Component Scoring System (QCSS) [46,47], the HRWEF [45], and the adherence to the Health on the Net Foundation Code of Conduct (HONcode) [48]. The QCSS is an instrument designed to offer scores on ownership, authorship, author qualification, purpose, attribution (references provided for requiring statements), interactivity, and currency of posting and revision. The sum of scores generates a final score summarized as excellent (80%-100%), very good (70%-79%), good (60%-69%), fair (50%-59%), or (poor 0%-50%). The HRWEF tool can be used by health professionals and patients to assess the appropriateness of websites. It consists of 30 items where each criterion is rated on a 3-point scale, scored as not applicable (score=0), disagree (score=1), or agree (score=2). It is divided into 7 main sections assessing the content, accuracy, author, currency, audience, navigation, and external links. An overall score was designated as excellent (90%-100%), adequate (75%-89%), or poor (0%-75%). Moreover, the HONcode certification validates and certifies the quality of the medical information provided on the internet.

    Suitability of Information

    The Suitability Assessment of Materials (SAM) tool [49] was used to assess the appropriateness of health information materials by considering characteristics such as content, graphics, literacy level, layout, typography, and cultural appropriateness of the websites. Each of the 22 items was rated as superior (rating +2), adequate (rating +1), not suitable (rating 0), or not applicable. Scores were summed to yield an overall percentage for the website reported as superior (70%-100%), adequate (40%-69%), or not suitable (0%-39%).

    Readability

    Readability tools were used to assess the difficulty of reading the written texts on the websites. The Flesh Kincaid test (F-K) [50], Simple Measure of Gobbledygook (SMOG) [51], and readability consensus based on 7 readability formulas (Flesch Reading Ease score, Gunning Fog, F-K, SMOG, the Coleman-Liau Index, Automated Readability Index, and Linsear Write Formula) [52] were used. The reviewers assessed the readability by selecting multiple written sections from each website and inserting it into a web-based readability calculator [52] that calculated F-K, SMOG, and readability consensus scores. In addition, as an item of the SAM tool, readability was also assessed using SMOG and rated as superior (grade 5 or lower), adequate (6th-8th grade level), or not suitable (grade 9 or higher). The Australian government recommends aiming for a lower than grade 8 reading level for health information [53-55], whereas the American Medical Association recommends education materials to be written at grade 6 reading level or lower [56].

    Website Interactivity and Features

    A validated interactivity scale was used as an individual consumers’ perceptual assessment of websites in a previous study, which asked undergraduate business students ranging from age 19 to 40 years were asked to browse and rate websites using 15 items based on their personal experience [57]. The 15 items measuring active control (control over what users can do and see on the websites), two-way communication (ease of communication and offering feedback on the website), and synchronicity (website responsiveness to input and obtaining instantaneous information) were adopted for the purpose of this study.

    A 3-point Likert scale was created to score each item as follows: agree (score=2), partially agree (score=1), or disagree (score=0), and an average score for all components was calculated. Interactivity scale was summarized as excellent (≥90%), adequate (70%-89%), or poor (≤69%).

    Interactive aspects and features were also assessed by looking at whether the website was functional on a smartphone screen, had an associated app, addressed ethnicity, and included language options, paid features, search functions, games, videos, podcasts, chatbot, question and answer forum, quizzes, animation, a feedback form, slide shows, ratings, frequently asked questions section, recipes, read out loud options, navigation menu, social media links, acceptable page speed, webinars, or other.

    Statistical Analysis

    Interrater Reliability

    Authors DJ and HC undertook interrater reliability (IRR) checking. A random 10% sample of all websites (n=7) were selected—the coding by DJ and HC were compared, and an IRR score was generated. Discrepancies were discussed until reviewers reached a consensus on their final ratings. Any disagreements were resolved by a third reviewer (ST).

    IRR was calculated for the readability scores, quality of content scores, SAM, and the evaluation of information content using intraclass correlation coefficients (ICCs), with a high ICC value (maximum 1.0 indicating no variance in the scoring between different assessors, whereas ≥0.5 was moderate, ≥0.70 was good, and ≥0.80 indicated excellent reliability).

    Software Used

    Data were transferred from REDCap to SPSS for MacBook (version 27.0; IBM Corp), where statistical analyses were performed. The ICC values calculated for content, HRWEF, QCSS, interactivity, and SAM were 0.5, 0.6, 0.6, 0.7, and 0.7, respectively, indicating a moderate to good level of consistency for the rating measurements. As the readability grades were calculated using computerized software, interrater consistency was not measured. The reviews discussed discrepancies by re-evaluating the websites together to ensure scoring consensus.


    Screening Process

    As shown in Figure 1, a total of 450 global websites were reviewed between August 2021 and February 2022. The removal of 218 duplicate websites left a total of 232 unique websites. Of these, 66 websites met the inclusion criteria and were eligible to be evaluated. The remaining 166 websites were excluded, as 50 were not relevant to infant health behaviors; 38 were articles, PDF documents, or downloadable documents; 42 were a web-based shops; 23 had insufficient content; 4 were government guidelines; 7 offered web-based consultations, and 2 were infant-related apps on Google Play and App Store.

    ‎
    Figure 1. Diagram of website selection process.
    View this figure

    Scope, Accuracy, and Depth of Information

    Scope and Depth of Subtopics

    The scope and depth of the information covered in the subtopics were predominantly poor, with 91% (60/66) of websites obtaining an overall score of ≤74%, whereas only 9% (6/66) of websites were rated as adequate.

    The overall mean rating of all websites was poor (53%, SD 18%; IQR 40%-67%; Table 2). Expressing, feeding, and storage of expressed breast milk, preparing and feeding infant formula practices, and monitoring infant’s progress topics had the lowest mean scores of 33%, 43%, and 49%, respectively. For information on infant sleep recommendations and bedtime practices, active play, screen time, and breastfeeding recommendations, correct advice was mostly reported as reflected by their mean scores of 73%, 70%, and 66%, respectively. Only 17% (10/58) of websites fully addressed infant feeding to appetite by encouraging responsiveness to infant hunger and satiety cues, not pressuring the baby to finish the bottle, feeding to appetite or baby-led feeding, avoiding bottle propping and bottle use in bed, and benefits of allowing infants to self-regulate their own appetite. Infant feeding to appetite was partially addressed by 59% (34/58) of the websites by highlighting a few of the above-mentioned points (Table 2 and Multimedia Appendix 3).

    Table 2. The quantitative scope and depth of information based on Australian infant feeding and physical activity guidelines on all websites (N=66).
    Topics addressed and websitesaValues, mean (SD)Values, median
    Breastfeeding

    		Breastfeeding recommendations (n=58)66 (21)71

    		Physiology of breast milk and breastfeeding (n=55)62 (21)62.5

    		Monitoring infant’s progress (n=56)49 (26)50
    Breastfeeding, common problems, and their management

    		Maternal factors affecting breastfeeding (n=54)52 (26)54

    		Infant factors affecting breastfeeding (n=54)50 (26)50
    Expressing and storing breast milk

    		Expressing, feeding, and storage of expressed breast milk (n=50)33 (32)30
    Infant formula

    		Preparing and feeding infant formula practices (n=58)43 (19)46
    Introducing solids

    		Solid introduction and foods and beverages not suitable for infants (n=58)50 (23)50
    Infant activity

    		Active play and screen time (n=52)70 (30)77.5
    Infant sleep

    		Cosleep recommendations (n=53)61 (21)62.5

    		Sleep recommendations and bedtime practices (n=54)73 (29)75
    Overall content

    		Overall scope and depth of information (n=66)53 (18)55

    aNot all websites included information on all subtopics.

    Subtopics Addressed

    Subtopics that were most frequently correctly addressed on websites were recommendation to exclusively breastfeed till 6 months of age and continue breastfeeding with appropriate complementary food till 12 months of age and beyond (45/58, 77% of websites); natural patterns of breastfeeding 8 to 12 times over 24 hours (42/55, 76% of websites); postnatal breastfeeding advice to seek support from lactation consultants, midwives, or doctors (40/58, 68% of websites); tummy time recommendations (40/52, 76% of websites); and cosleeping in a separate cot but the same room as parents for the first 6- to 12-month recommendation (39/53, 73% of websites).

    Subtopics Not Addressed

    Subtopics that were most often not addressed on websites were supplemental requirements for infants on a vegan diet (30/58, 51% of websites); sterilization and proper use of hand pumps (30/50, 60% of websites); factors affecting initiation of lactation after birth (30/54, 55% of websites); importance and awareness of baby-friendly hospital initiative (27/58, 46% of websites); and correct water temperature for preparing infant formula and risk of infection from Cronobacter sakazakii bacteria (26/58, 44% of websites).

    Subtopics Incorrectly Addressed

    Subtopics that were most frequently incorrectly addressed were storage of freshly expressed, thawed, or used breast milk (22/50, 44% of websites); correct selection of infants’ first foods (6/58, 10% of websites); and correct preparation of infant formula (5/58, 8% of websites).

    Assessment of Website Quality

    Using HRWEF

    A majority of the websites attained an adequate rating (49/66, 74%) for the quality of the websites using the HRWEF tool. Although 20% (13/66) of the websites were rated as excellent, the remaining 3% (4/66) of websites received a poor scoring, 3 of which were commercial.

    The overall HRWEF mean percentage score was 85% (SD 5.98%). The questions with the highest scores addressed the organization of the site, navigation, internal link, and the type of audience the author targeted. Conversely, the questions with the lowest scores were related to dates of publication and revision of content (Multimedia Appendix 3).

    Using the QCSS

    From the quality evaluation conducted using the QCSS tool, 8% (5/66) were rated as excellent, 21% (14/66) as very good, 32% (21/66) as good, 11% (7/66) as fair, and 29% (19/66) were rated as poor. The overall mean QCSS score was 60 (SD 18; Multimedia Appendix 3).

    In comparison with websites that scored excellent, poorly rated websites failed to provide references, author qualifications, and currency of content. A total of 32 websites stated that the author was a health care professional, whereas 30 websites clearly listed the name of the person supplying the information and author qualification. In addition, 31 websites had failed to display references for requiring statements. Only 3 websites presented the dates of original posting and revision.

    A total of 9 websites stated they had acquired the HONcode certification demonstrating the intent of offering quality health information to meet ethical standards.

    Quality (QCSS) by Organization

    Figure 2 shows the quality of the websites by the type of organization as measured by QCSS. This ranged from poor to very good. Media-related sources received the highest median score of 77% (very good), followed by nongovernmental organizations, hospital websites, and privately owned and government websites, which received a score corresponding to good. Commercial websites had a mean score of 46% (fair), and university websites received the lowest median score of 35% (poor).

    ‎
    Figure 2. Quality Component Scoring System (QCSS) by type of organization. NGO: nongovernmental organizations.
    View this figure

    Assessment of Suitability of Website Information by SAM

    Overall, 3% (2/66 assessments) of websites were rated superior for suitability of health information, 82% (54/66 assessments) of websites as adequate, and 15% (10/66 assessments) of websites not suitable, as shown in Table 3 (Multimedia Appendix 3).

    Very few websites were rated superior on literacy demand, such as writing style, context, and vocabulary used. Overall, 15% of websites provided culturally appropriate visual aids based on the consumers they were targeting. There were variations in the type of images used in the resources. Some images depicted different sex, race, color, religion, and age, whereas others targeted specific cultural groups such as Indian and Aboriginal and Torres Strait Islander. Only one of the sites addressed the cultural specificity of information relating to experience, language, or provision of examples to patients from diverse sociodemographic backgrounds.

    Overall, 21% (14/66) of websites had the option to be translated into a language other than English, such as Arabic, Spanish, Hindi, or Bengali. Only 14% (9/66) presented information that addressed culture in texts or images. The culturally appropriate information varied between fasting tips and breastfeeding in Islam, Christianity or Judaism, fasting and pregnancy tips, multiracial graphics, and recipes.

    Many websites provided a clear layout of information and easily understandable cover graphics that clearly portrayed the purpose of the material. Most topics were subdivided to improve readers’ self-efficacy, which was rated as adequate; for instance, infant sleep was subdivided into quiet playtime, bedtime routine, safety sleep practices, and infant sleep recommendations.

    Table 3. Website scores based on the Suitability Assessment of Materials (SAM) criteria.

    		SAM scores (evaluations), n (%)

    		Not suitableAdequateSuperiorNot applicable
    Content

    		Purpose is evident1 (1)30 (45)35 (53)—a

    		Content about behaviors2 (3)59 (89)5 (8)—

    		Limited to essential information4 (6)40 (61)22 (33)—

    		Summary and review46 (70)16 (24)3 (5)1 (1)
    Literacy demand

    		Reading grade level31 (47)31 (47)3 (5)1 (1)

    		Writing style with active voice5 (8)55 (83)6 (9)—

    		Vocabulary uses common words5 (8)56 (84)5 (8)—

    		Context given first11 (17)47 (71)8 (12)—

    		Headers or topic captions1 (1)21 (32)44 (67)—
    Graphics

    		Purposeful cover graphic9 (14)26 (39)30 (46)1 (1)

    		Appropriate type of illustrations9 (14)22 (33)6 (9)29 (44)

    		Relevance of illustrations11 (17)19 (29)8 (12)28 (42)

    		Lists, tables, graphs, and charts explained7 (11)45 (68)5 (8)9 (14)

    		Captions used for graphics9 (14)41 (62)11 (17)5 (8)
    Layout and typography

    		Layout factors6 (10)58 (88)2 (3)—

    		Typography2 (3)59 (89)5 (8)—

    		Subheadings used0 (0)42 (64)24 (36)—
    Learning, stimulation, and motivation

    		Interaction with readers used22 (33)44 (67)0 (0)—

    		Modeled and specific behaviors16 (24)45 (68)5 (8)—

    		Self-efficacious tasks and behaviors5 (8)57 (86)4 (6)—
    Cultural appropriateness

    		Cultural match0 (0)1 (1)0 (0)65 (99)

    		Cultural image and examples0 (0)8 (12)2 (3)56 (85)

    aNot included in the overall score.

    Assessment of Website Readability

    Very few websites met the Australian Federal government’s recommended level for written health information of lower than a grade 8 reading level: 29% (19/66) of websites (SMOG), 20% (13/66) of websites (F-K web-based tool), and 12% (8/66) of websites (consensus tool). In 2 of the websites assessed for interreliability, the readability scores ranged from 8 to 11 and 7 to 14 between the researchers depending on the varying content selected for assessment (Multimedia Appendix 4).

    The median readability grades were 8.5 (IQR 7-10), 9 (IQR 8-11), and 10 (IQR 8-11) using the SMOG formula, web-based F-K calculator, and consensus calculator, respectively. There was a good correlation among reading grade scores across the readability measures (P<.001; 2-tailed).

    Assessment of Website Interactivity and Features

    Table 4 presents the results of the websites interactivity scores in terms of active control, two-way communication, and synchronicity.

    Table 4. Interactivity scores of websites (N=66).

    		Agree, n (%)
    Active control

    		I felt that I had a lot of control over my visiting experiences at this website17 (25)

    		While I was on the website, I could choose freely what I wanted to see23 (34)

    		While surfing the website, I had full control over what I can do on the site21 (31)

    		While surfing the website, my actions decided the kind of experiences I got24 (36)
    Two-way communication

    		The website is effective in gathering visitors’ feedback6 (9)

    		This website facilitates two-way communication between the visitors and the site7 (10)

    		It is easy to offer feedback to the website13 (19)

    		The website makes me feel it wants to listen to its visitors9 (13)

    		The website encourages visitors to talk back8 (12)

    		The website gives visitors the opportunity to talk back18 (27)
    Synchronicity

    		The website processed my input very quickly16 (24)

    		Getting information from the website is very fast15 (22)

    		I was able to obtain the information I want without any delay10 (15)

    		When I clicked on the links, I felt I was getting instantaneous information26 (39)

    		The website was very fast in responding to my requests18 (27)

    The overall interactivity of websites was predominantly poor (49/66, 74%). The remaining 26% (17/66) of websites were marginally adequate with no websites obtaining an excellent rating. More than half the websites acquired an incomplete score for active control resulting from slow loading web pages and the inability of site search engines to return relevant results effectively. Very few websites encouraged visitors to talk back or facilitated 2-way communication between the visitors and the site. As for synchronicity of the sites, approximately one-fourth of websites received a full score (agree) for their ability to process input and respond to requests promptly (Multimedia Appendix 4).

    The most common features found on websites were social media links (61/66, 92%), frequently asked questions (48/66, 73%), videos (44/66, 67%), and recipes (35/66, 53%), whereas language options, webinars, question and answer forums, chatbots, read out loud function, slide shows, animation, and games were less common. Moreover, 80% (53/66) of websites had additional features such as text font size options, tools (eg, ovulation calculator, pregnancy calculator, and parenting tools), download and print page content option, and YouTube accounts. Overall, 47% (31/66) of websites had associated apps on Google Play and Apple Store. Log-in options for personalized health information were presented on 40% (26/66) of the websites.


    Principal Findings

    In this review, we systematically assessed 66 websites that reported health information related to infant nutrition, active play, screen time, or sleep behaviors. This review extends on the existing 2015 assessment by providing 2 main conceptual contributions. First, it covers the quality, content, suitability, readability, and comprehensibility of web-based infant health information at a global level. Second, it assesses the interactivity, features, and cultural considerations of webpages. In this section, we discuss the principal findings, comparison to prior work, implications for future practice, and then outline the strengths and limitations of this review.

    This study found that the information content of the websites was overall poor in terms of scope and depth of information, which was similar to the findings of the previous 2015 assessment. Approximately one-third of the websites reported different advice on storage of expressed breast milk; for instance, “freshly expressed breast milk can be stored safely in the refrigerator for up to five days.” This information was contrary to the Australian guidelines on infant feeding that stated storage of expressed breast milk should not exceed 72 hours in the fridge [42]. This was due to the development of some websites in other countries, such as America or Europe, where guidelines differ from that of Australia [58,59].

    The quality of websites was generally adequate when evaluated by HRWEF and mostly ranged from poor to good when rated by the QCSS tool. These findings are consistent with those from previous studies [60-63], which evaluated a range of health information available on the web using similar tools. This study also highlights that the quality of websites in terms of ownership, authorship, author qualification, purpose, referencing statements, and currency of information was the highest among media, nongovernmental organizations, and hospital websites and the lowest among university websites. This is an important finding, given that parents view university sites as a high-quality, reliable, and credible source of information [64]. It is vital that the health information on websites is continuously updated to meet the latest guidelines with relevant currency, authorship, qualification, and supporting attribution statements. This in turn will provide readers with the clarity they need to assess the quality of web-based health information and identify reputable websites. Moreover, websites with outdated information and no supporting statements can mislead readers, resulting in adverse health consequences [65-67].

    Using the SAM tool, we found that website information on infant health behaviors was generally adequate. This finding is consistent with that of other studies on the SAM [68-70]. Despite the overall adequate suitability ratings of information on the selected websites, there were marked limitations in terms of cultural appropriateness, literacy demand, and illustrations. This highlights the issue that web-based infant health information rarely considered the needs of people from non–English-speaking ethnic groups and how they may interpret or apply the health information. This is unfortunate, given the ubiquitous nature of the internet. Given the increase in cultural diversity within Australia and abroad, it is important to consider cultural appropriateness of information and provide culturally and ethnically diverse consumers the capacity to access, understand, and use health information to make well-informed health decisions [71-73].

    In regard to readability, this study highlighted that most websites were at readability levels beyond the ideal level of lower than grade 8. This finding was also reflected in previous studies [74-76]. Notably, a difference of 4 to 6 grades was observed between the interreliability readability grades of 2 websites scored by the 2 researchers DJ and HC. This reflected the inconsistent readability levels across various webpages within a website. Readability and health literacy play an integral part in information accessibility and usability [77]. The readability formulas are based on the number of words, sentence length, and number of syllables per word. Therefore, using simpler words, shorter sentences, pictures, videos, and co-design methods as per the Australian Commission for Safety and Quality in health care are important considerations for writing health information for consumers [78].

    Furthermore, readability levels of web-based health information should be tested for consistency and presented in an easy-to-read format providing access to people with low health literacy.

    The overall interactivity of website functions was poor. In addition, interactive features were mostly common among media and commercial websites and least common among government websites. According to the World Health Organization, various provision methods of providing health information are important to increase accessibility and achieve positive health outcomes [79]. Multiple patient-focused interventions have reported that various health information formats, such as videos, audios, and infographics, have contributed to the improvement in parental knowledge, satisfaction, and health outcomes [80-82]. Hence, it is crucial for credible websites such as government owned to make the wealth of information available on the web interactive and accessible to increase consumer engagement and use of reliable sources.

    Comparison With Prior Work

    In comparison with the 2015 review, 55 new websites were assessed in this review, whereas 11 were common across both studies. It is important to note that the first 30 websites generated from every search term used resulted in a range of global websites for consumers to access. Hence, it is vital for education to be provided to parents of young children and health professionals that will enable them to determine the quality and credibility of web-based health information as accuracy is critical, especially in the first 1000 days of life [83,84].

    Furthermore, several websites from the 2015 review were excluded for reasons such as their web-based content had been removed or they no longer exist. A potential reason may be due to the cost and maintenance of websites that were developed with limited funding. A systematic review of factors that influence eHealth reported that ongoing maintenance costs were barriers for several studies [85]. Another reason may be due to the evolution of websites over time and the inability of static websites to enable and host new features [30].

    To replicate and compare results from the 2015 assessment, the same validated tools were used to assess the quality, suitability, and readability of infant health websites. Interestingly, the results of the new eligible websites did not greatly differ from the results of the 2015 systematic assessment [25]. There was a slight improvement in the quality of websites from poor to adequate or good measured by HRWEF and QCSS since 2015. However, the suitability rating of websites in both assessments measured by SAM was the same. Similarly, the readability of written health information in the majority of the websites did not meet government recommendation in both assessments as well. Although websites have evolved since 2015, minimal to no improvements have been identified in this review. One reason could be due to the lack of user involvement in the website design. A recent systematic review reported that a considerable number of studies raised concerns that involving users in technology design can be fairly demanding and requires time and effort [86]. Another reason may be the lack of use of validated tools such as the ones used in this study to ensure optimized quality during the development of websites.

    Identifying cultural considerations, interactivity, and features of web pages is a value added to this study. We found that very few websites addressed culture or had interactive features such as multilingual options, chatbots, or read out loud functions. With the rise in immigrants from culturally and linguistically diverse communities [87], more and more people are facing access barriers to health information and eHealth services due to the lack of language support and culturally appropriate health information through the internet [88-90]. This demonstrates the need to engage end users throughout the web development to ensure high-quality outcomes and meet consumers’ needs and expectations.

    Implications for Practice

    With the wide spread of internet use and wealth of information available on the web [91], it is imperative for parents and health professionals to be guided to the optimal and most accurate sources of information. Websites should be screened for authorship, ownership, information date, and HONcode certification before use. There is a need for web developers to bear in mind their end users. One way to overcome this challenge is by involving consumers in website development through co-design workshops [92]. This will ensure developers get a good understanding of end users’ access requirements, literacy demands, preference for alternative presentation formats of information, and cultural considerations. An establishment of a regulatory body is also recommended to ensure that newly developed websites are built on validated tools and that all websites comply with government guidelines standards.

    Strengths and Limitations

    To the best of our knowledge, this is the first study to evaluate website interactivity, features, and cultural considerations for web-based infant health information. This review provides a comprehensive global overview of the available web-based information about infant health behaviors and identifies ways for improvement.

    Although this study adhered to a rigorous systematic search process, there were several limitations. First, most websites included in this evaluation differed from that which was evaluated in the 2015 review. Hence, this demonstrates the dynamic nature and constant change of the internet. Thus, the website search during this review reflected a period that could potentially change. In addition, the website search was conducted in the English language using Google. Although Google is a highly used search engine, we acknowledge that some international users have access restraints [93]. Therefore, the results may not have identified websites present on other search engines or written in other languages. Furthermore, deleting cookies and search histories was intended to reduce unknown bias in the search strategy. However, it is acknowledged that the likelihood of most web users doing this is unlikely, and their searches might identify sites of which we were unaware.

    Another limitation is that the assessment criteria used Australian guidelines. Therefore, there was a potential for websites following non-Australian guidelines to obtain an incorrect score on a few subtopics. Moreover, the interactivity scale used was originally meant to capture consumers’ perceptual assessment of websites. However, due to the lack of published validated tools used to measure interactivity of websites, the interactivity scale was adopted for the purpose of this study. Thus, the interactivity scale used was a subjective measurement based on the researcher’s experience on the websites. Nevertheless, the tools used offer a standardized way to best capture the quality and interactivity of web-based information.

    Conclusions

    As more parents seek web-based guidance on infant health behaviors globally, there remains a significant concern on the quality, readability, interactivity, and accessibility of websites promoting health behaviors during infancy. This systematic assessment revealed that there is a need for researchers and health care providers to leverage innovative web-based platforms to provide culturally responsive evidence-based information accessible to those with limited English proficiency. Furthermore, a focus is needed on continuously updating existing health websites in addition to recommending an establishment of a regulatory body to ensure compliance with government standards. Moreover, the development of new eHealth technology should be based on validated tools to ensure the optimal quality of websites.

    Acknowledgments

    This work is a component of PhD degree of DJ at the University of Sydney.

    Authors' Contributions

    DJ and ST designed the study and the main conceptual ideas. DJ undertook the website search and evaluation. DJ and HC undertook the interrater reliability testing. DJ undertook statistical analysis. ST, LMW, LB, and CR cross-checked uncertain websites until consensus was reached. DJ wrote the first draft of the article; all authors critically revised the paper and provided comments.

    Conflicts of Interest

    None declared.

    ‎
    Multimedia Appendix 1

    Website evaluation tools.

    DOCX File , 31 KB
    ‎
    Multimedia Appendix 2

    Protocol.

    DOCX File , 38 KB
    ‎
    Multimedia Appendix 3

    Summary score for all websites.

    DOCX File , 24 KB
    ‎
    Multimedia Appendix 4

    Supplementary tables and graphs.

    DOCX File , 71 KB
    1. Poushter J. Internet access growing worldwide but remains higher in advanced economies. Pew Research Center. 2016 Feb 22.   URL: https:/​/www.​pewresearch.org/​global/​2016/​02/​22/​internet-access-growing-worldwide-but-remains-higher-in-advanced-economies/​ [accessed 2022-03-28]
    2. Roser M, Ritchie H, Ortiz-Ospina E. Internet. Our World in Data. 2015.   URL: https://ourworldindata.org/internet [accessed 2022-03-28]
    3. World Internet Users and 2022 Population Stats. Internet World Stats.   URL: https://www.internetworldstats.com/stats.htm [accessed 2022-03-28]
    4. Mobile Search Engine Host Market Share Worldwide. StatCounter Global Stats.   URL: https://gs.statcounter.com/platform-market-share/desktop-mobile-tablet [accessed 2022-03-28]
    5. Eysenbach G, Kohler C. What is the prevalence of health-related searches on the World Wide Web? Qualitative and quantitative analysis of search engine queries on the internet. AMIA Annu Symp Proc 2003;2003:225-229 [FREE Full text] [Medline]
    6. Schneider JA, Holland CP. eHealth search patterns: a comparison of private and public health care markets using online panel data. J Med Internet Res 2017 Apr 13;19(4):e117 [FREE Full text] [CrossRef] [Medline]
    7. eHealth information search behaviour: Is there a difference between private and public health care markets? Research Features. 2020 Aug 12.   URL: https:/​/researchfeatures.​com/​ehealth-information-search-behaviour-difference-between-private-and-public-health-care-markets/​ [accessed 2022-03-28]
    8. Choi H, Ahn S. Classifications, changes, and challenges of online health information seekers during COVID-19. Int J Environ Res Public Health 2021 Sep 09;18(18):9495 [FREE Full text] [CrossRef] [Medline]
    9. Neely S, Eldredge C, Sanders R. Health information seeking behaviors on social media during the COVID-19 pandemic among American social networking site users: survey study. J Med Internet Res 2021 Jun 11;23(6):e29802 [FREE Full text] [CrossRef] [Medline]
    10. Homeniuk R, Collins C. How COVID-19 has affected general practice consultations and income: general practitioner cross-sectional population survey evidence from Ireland. BMJ Open 2021 Apr 08;11(4):e044685. [CrossRef]
    11. Scott A, Bai T, Zhang Y. Association between telehealth use and general practitioner characteristics during COVID-19: findings from a nationally representative survey of Australian doctors. BMJ Open 2021 Mar 24;11(3):e046857 [FREE Full text] [CrossRef] [Medline]
    12. Kubb C, Foran HM. Online health information seeking by parents for their children: systematic review and agenda for further research. J Med Internet Res 2020 Aug 25;22(8):e19985 [FREE Full text] [CrossRef] [Medline]
    13. Bouche G, Migeot V. Parental use of the Internet to seek health information and primary care utilisation for their child: a cross-sectional study. BMC Public Health 2008 Aug 28;8:300 [FREE Full text] [CrossRef] [Medline]
    14. Lupton D, Maslen S. How women use digital technologies for health: qualitative interview and focus group study. J Med Internet Res 2019 Jan 25;21(1):e11481 [FREE Full text] [CrossRef] [Medline]
    15. Nievas-Soriano BJ, Castro-Luna GM, García-Duarte S, González-López MD, Parrón-Carreño T. Profile of the users and the most visited topics of a pediatric eHealth website. Int J Environ Res Public Health 2021 Oct 26;18(21):11248 [FREE Full text] [CrossRef] [Medline]
    16. Mahony C, Heavin C, Sammon D. Health-related information resource use: an exploratory study of expectant mothers. J Decis Syst 2022;31(3):234-257 [FREE Full text] [CrossRef]
    17. Bernhardt JM, Felter EM. Online pediatric information seeking among mothers of young children: results from a qualitative study using focus groups. J Med Internet Res 2004 Mar 01;6(1):e7 [FREE Full text] [CrossRef] [Medline]
    18. Dexter B, Frank S, Seguin L. How and where parents of infants and young children want to receive nutrition information. Can J Diet Pract Res 2016 Dec;77(4):203-205. [CrossRef] [Medline]
    19. Alianmoghaddam N, Phibbs S, Benn C. "I did a lot of Googling": a qualitative study of exclusive breastfeeding support through social media. Women Birth 2019 Apr;32(2):147-156. [CrossRef] [Medline]
    20. Slomian J, Bruyère O, Reginster JY, Emonts P. The internet as a source of information used by women after childbirth to meet their need for information: a Web-based survey. Midwifery 2017 May;48:46-52. [CrossRef] [Medline]
    21. Asiodu IV, Waters CM, Dailey DE, Lee KA, Lyndon A. Breastfeeding and use of social media among first-time African American mothers. J Obstet Gynecol Neonatal Nurs 2015;44(2):268-278 [FREE Full text] [CrossRef] [Medline]
    22. Wright AL, Jack SM, Ballantyne M, Gabel C, Bomberry R, Wahoush O. Indigenous mothers' experiences of using primary care in Hamilton, Ontario, for their infants. Int J Qual Stud Health Well-being 2019 Dec;14(1):1600940 [FREE Full text] [CrossRef] [Medline]
    23. Rhodes A. Sources of child health information: what parents use and trust. Royal Children's Hospital National Child Health Poll. Melbourne, Australia: The Royal Children's Hospital; 2016.   URL: https://www.rchpoll.org.au/polls/sources-of-child-health-information-what-parents-use-and-trust/ [accessed 2022-03-28]
    24. Jaks R, Baumann I, Juvalta S, Dratva J. Parental digital health information seeking behavior in Switzerland: a cross-sectional study. BMC Public Health 2019 Feb 21;19(1):225 [FREE Full text] [CrossRef] [Medline]
    25. Taki S, Campbell KJ, Russell CG, Elliott R, Laws R, Denney-Wilson E. Infant feeding websites and apps: a systematic assessment of quality and content. Interact J Med Res 2015 Sep 29;4(3):e18 [FREE Full text] [CrossRef] [Medline]
    26. Cheng H, Tutt A, Llewellyn C, Size D, Jones J, Taki S, et al. Content and quality of infant feeding smartphone apps: five-year update on a systematic search and evaluation. JMIR Mhealth Uhealth 2020 May 27;8(5):e17300 [FREE Full text] [CrossRef] [Medline]
    27. Son HM, Je M, Sohn YS. Quality evaluation of online health information related to young child. Child Health Nurs Res 2018 Jan 31;24(1):91-100 [FREE Full text] [CrossRef]
    28. Dol J, Richardson B, Boates T, Campbell-Yeo M. Learning to parent from Google? Evaluation of available online health evidence for parents of preterm infants requiring neonatal intensive care. Health Informatics J 2019 Dec;25(4):1265-1277 [FREE Full text] [CrossRef] [Medline]
    29. Król K. Evolution of online mapping: from Web 1.0 to Web 6.0. Geomat Landmanag Landsc 2020;1:33-51 [FREE Full text] [CrossRef]
    30. Web 1.0, web 2.0 and web 3.0 with their difference. GeeksforGeeks. 2022.   URL: https://www.geeksforgeeks.org/web-1-0-web-2-0-and-web-3-0-with-their-difference/ [accessed 2022-03-28]
    31. Cisco Annual Internet Report (2018–2023) White Paper. Cisco. 2022.   URL: https:/​/www.​cisco.com/​c/​en/​us/​solutions/​collateral/​executive-perspectives/​annual-internet-report/​white-paper-c11-741490.​html [accessed 2022-03-28]
    32. International bandwidth in the Asia-Pacific region in 2020, by country or territory. Statista. 2020.   URL: https://www.statista.com/statistics/1293199/apac-international-bandwidth-by-country/ [accessed 2022-03-28]
    33. Armstrong M, Richter F. Infographic: How many websites are there? Statista. 2021.   URL: https://www.statista.com/chart/19058/number-of-websites-online/ [accessed 2022-03-28]
    34. Harrison C, Britt H, Wong C, Henderson J. Patient use of the internet for health information. Aust Fam Physician 2014 Dec;43(12):875-877 [FREE Full text] [Medline]
    35. Digital Health. Australian Institute of Health and Welfare. 2022 Jul 7.   URL: https://www.aihw.gov.au/reports/australias-health/digital-health [accessed 2022-03-28]
    36. Sawyer R. The Impact of New Social Media on Intercultural Adaptation. Senior Honors Projects. 2011 May.   URL: https://digitalcommons.uri.edu/srhonorsprog/242/ [accessed 2022-03-29]
    37. Morrison C, Doherty G. Analyzing engagement in a Web-based intervention platform through visualizing log-data. J Med Internet Res 2014 Nov 13;16(11):e252 [FREE Full text] [CrossRef] [Medline]
    38. Perski O, Crane D, Beard E, Brown J. Does the addition of a supportive chatbot promote user engagement with a smoking cessation app? An experimental study. Digit Health 2019 Sep 30;5:2055207619880676 [FREE Full text] [CrossRef] [Medline]
    39. Adamopoulou E, Moussiades L. An overview of chatbot technology. In: Proceedings of the 16th International Conference on Artificial Intelligence Applications and Innovations. 2020 Presented at: AIAI '20; June 5–7, 2020; Neos Marmaras, Greece p. 373-383. [CrossRef]
    40. Wong J, Foussat AC, Ting S, Acerbi E, van Elburg RM, Mei Chien C. A chatbot to engage parents of preterm and term infants on parental stress, parental sleep, and infant feeding: usability and feasibility study. JMIR Pediatr Parent 2021 Oct 26;4(4):e30169 [FREE Full text] [CrossRef] [Medline]
    41. Jacobsen M. How Far Down the Search Engine Results Page Will Most People Go? Leverage Marketing. 2017.   URL: https://www.theleverageway.com/blog/how-far-down-the-search-engine-results-page-will-most-people-go/ [accessed 2022-04-03]
    42. Infant Feeding Guidelines: Information for Health Workers. National Health and Medical Research Council. Canberra, Australia: Australian Government; 2012.   URL: https:/​/www.​eatforhealth.gov.au/​sites/​default/​files/​files/​the_guidelines/​n56_infant_feeding_guidelines.​pdf [accessed 2022-03-29]
    43. Australian 24-Hour Movement Guidelines for the Early Years (Birth to 5 Years). Department of Health, Australian Government. Canberra, Australia: Australian Government; 2010.   URL: https:/​/www.​health.gov.au/​sites/​default/​files/​documents/​2021/​05/​24-hour-movement-guidelines-birth-to-5-years-brochure.​pdf [accessed 2022-09-20]
    44. For infants, toddlers and preschoolers (birth to 5 years). Department of Health and Aged Care, Australian Government. 2015.   URL: https:/​/www.​health.gov.au/​health-topics/​physical-activity-and-exercise/​physical-activity-and-exercise-guidelines-for-all-australians/​for-infants-toddlers-and-preschoolers-birth-to-5-years [accessed 2022-03-29]
    45. Pealer LN, Dorman SM. Evaluating health-related Web sites. J Sch Health 1997 Aug;67(6):232-235. [CrossRef] [Medline]
    46. Ghezzi P, Chumber S, Brabazon T. Educating medical students to evaluate the quality of health information on the web. In: Floridi L, Illari P, editors. The Philosophy of Information Quality. Cham, Switzerland: Springer; 2014:183-199.
    47. Martins EN, Morse LS. Evaluation of Internet websites about retinopathy of prematurity patient education. Br J Ophthalmol 2005 May;89(5):565-568 [FREE Full text] [CrossRef] [Medline]
    48. Methodology. HONcode.   URL: https://protect-au.mimecast.com/s/OFi7CROND2urv1q5xFOtFjF?domain=hon.ch [accessed 2022-04-03]
    49. Doak L, Doak C, Root J. Suitability Assessment of Materials For Evaluation of Health-Related Information for Adults. Aspirius Library. 2008.   URL: https://protect-au.mimecast.com/s/LM_uCVARKgClx6ogYHQ1YDU?domain=aspiruslibrary.org [accessed 2022-03-29]
    50. Kincaid JP, Fishburne Jr RP, Rogers RL, Chissom BS. Derivation Of New Readability Formulas (Automated Readability Index, Fog Count And Flesch Reading Ease Formula) For Navy Enlisted Personnel. Institute for Simulation and Training, University of Central Florida. 1975 Jan 1.   URL: https://stars.library.ucf.edu/istlibrary/56 [accessed 2022-03-29]
    51. McLaughlin HG. SMOG grading - a new readability formula. J Read 1969;12(8):639-646 [FREE Full text]
    52. Automatic Readability Checker - (Our Free Text Readability Consensus Calculator). Readability Formulas.   URL: https://readabilityformulas.com/free-readability-formula-tests.php [accessed 2022-04-03]
    53. Style manual. Australian Government Coat of Arms.   URL: https://www.stylemanual.gov.au/accessible-and-inclusive-content/literacy-and-access [accessed 2022-04-03]
    54. National Report on schooling in Australia 2009. Australian Curriculum, Assessment and Reporting Authority. 2011.   URL: https://docs.acara.edu.au/resources/National_Report_on_Schooling_in_Australia_2009_live.pdf [accessed 2022-04-03]
    55. Health literacy: taking action to improve safety and quality. Australian Commission on Safety and Quality in Health Care. Sydney, Australia: ACSQHC; 2014 Aug.   URL: https:/​/www.​safetyandquality.gov.au/​sites/​default/​files/​migrated/​Health-Literacy-Taking-action-to-improve-safety-and-quality.​pdf [accessed 2022-04-03]
    56. Grabeel KL, Russomanno J, Oelschlegel S, Tester E, Heidel RE. Computerized versus hand-scored health literacy tools: a comparison of Simple Measure of Gobbledygook (SMOG) and Flesch-Kincaid in printed patient education materials. J Med Libr Assoc 2018 Jan;106(1):38-45 [FREE Full text] [CrossRef] [Medline]
    57. Liu Y. Developing a scale to measure the interactivity of websites. J Advert Res 2003 Jun 01;43(2):207-216. [CrossRef]
    58. Proper storage and preparation of breast milk. Centers for Disease Control and Prevention. 2022.   URL: https://www.cdc.gov/breastfeeding/recommendations/handling_breastmilk.htm [accessed 2022-04-03]
    59. Food for infants and young children. Food Safety.   URL: https:/​/ec.​europa.eu/​food/​safety/​labelling-and-nutrition/​specific-groups/​food-infants-and-young-children_en [accessed 2022-04-03]
    60. Lobato MZ, Martins AC. Quality assessment of online information on Brazilian domain sites related to dental implants. Brazilian J Implant Health Sci 2020;2(7):21-22 [FREE Full text] [CrossRef]
    61. Nasser S, Mullan J, Bajorek B. Assessing the quality, suitability and readability of internet-based health information about warfarin for patients. Australas Med J 2012;5(3):194-203 [FREE Full text] [CrossRef] [Medline]
    62. Floridi L, Illari P. The Philosophy of Information Quality. Cham, Switzerland: Springer International Publishing; 2014.
    63. Chumber S, Huber J, Ghezzi P. A methodology to analyze the quality of health information on the Internet: the example of diabetic neuropathy. Diabetes Educ 2015 Feb;41(1):95-105. [CrossRef] [Medline]
    64. Zarnowiecki D, Mauch CE, Middleton G, Matwiejczyk L, Watson WL, Dibbs J, et al. A systematic evaluation of digital nutrition promotion websites and apps for supporting parents to influence children's nutrition. Int J Behav Nutr Phys Act 2020 Feb 10;17(1):17 [FREE Full text] [CrossRef] [Medline]
    65. Swire-Thompson B, Lazer D. Public health and online misinformation: challenges and recommendations. Annu Rev Public Health 2020 Apr 02;41:433-451. [CrossRef] [Medline]
    66. Kortum P, Edwards C, Richards-Kortum R. The impact of inaccurate Internet health information in a secondary school learning environment. J Med Internet Res 2008 Jun 30;10(2):e17 [FREE Full text] [CrossRef] [Medline]
    67. Kim R, Kim HJ, Jeon B. The good, the bad, and the ugly of medical information on the Internet. Mov Disord 2018 May;33(5):754-757. [CrossRef] [Medline]
    68. Martin CA, Khan S, Lee R, Do AT, Sridhar J, Crowell EL, et al. Readability and suitability of online patient education materials for glaucoma. Ophthalmol Glaucoma (forthcoming) 2022 Mar 14:S2589-4196(22)00051-5. [CrossRef] [Medline]
    69. Doubleday AR, Novin S, Long KL, Schneider DF, Sippel RS, Pitt SC. Online information for treatment for low-risk thyroid cancer: assessment of timeliness, content, quality, and readability. J Cancer Educ 2021 Aug;36(4):850-857. [CrossRef] [Medline]
    70. Guan Y, Maloney KA, Roter DL, Pollin TI. Evaluation of the informational content, readability and comprehensibility of online health information on monogenic diabetes. J Genet Couns 2018 Jun;27(3):608-615 [FREE Full text] [CrossRef] [Medline]
    71. Institute of Medicine (US) Committee on Health Literacy. In: Nielsen-Bohlman L, Panzer AM, Kindig DA, editors. Health Literacy: A Prescription to End Confusion. Washington, DC, USA: National Academies Press (US); 2004.
    72. Hughson JA, Marshall F, Daly JO, Woodward-Kron R, Hajek J, Story D. Health professionals' views on health literacy issues for culturally and linguistically diverse women in maternity care: barriers, enablers and the need for an integrated approach. Aust Health Rev 2018 Feb;42(1):10-20. [CrossRef] [Medline]
    73. Peterson EB, Chou WY, Kelley DE, Hesse B. Trust in national health information sources in the United States: comparing predictors and levels of trust across three health domains. Transl Behav Med 2020 Oct 08;10(4):978-988 [FREE Full text] [CrossRef] [Medline]
    74. Daraz L, Morrow AS, Ponce OJ, Farah W, Katabi A, Majzoub A, et al. Readability of online health information: a meta-narrative systematic review. Am J Med Qual 2018;33(5):487-492. [CrossRef] [Medline]
    75. Parr O, Dunmall K. An evaluation of online information available for women with breast implants aged 47-73 who have been invited to attend the NHS Breast Screening Programme. Radiography (Lond) 2018 Nov;24(4):315-327. [CrossRef] [Medline]
    76. Fefer M, Lamb CC, Shen AH, Clardy P, Muralidhar V, Devlin PM, et al. Multilingual analysis of the quality and readability of online health information on the adverse effects of breast cancer treatments. JAMA Surg 2020 Aug 01;155(8):781-784 [FREE Full text] [CrossRef] [Medline]
    77. Chesser A, Burke A, Reyes J, Rohrberg T. Navigating the digital divide: a systematic review of eHealth literacy in underserved populations in the United States. Inform Health Soc Care 2016;41(1):1-19. [CrossRef] [Medline]
    78. Manser G. Writing Health Information for Consumers. Canberra Health Literacy. 2022.   URL: https://cbrhl.org.au/health-services-providers/writing-patient-information/ [accessed 2022-04-04]
    79. Coulter A, Ellins J. Patient-Focused Interventions: A Review of the Evidence. Quest for Quality and Improved Performance. Oxford, UK: Picker Institute Health Foundation; 2006.   URL: https://www.health.org.uk/sites/default/files/PatientFocusedInterventions_ReviewOfTheEvidence.pdf [accessed 2022-04-04]
    80. Bandyopadhyay M, Stanzel K, Hammarberg K, Hickey M, Fisher J. Accessibility of Web-based health information for women in midlife from culturally and linguistically diverse backgrounds or with low health literacy. Aust N Z J Public Health 2022 Jun;46(3):269-274. [CrossRef] [Medline]
    81. Romantika IW, Lusmilasari L, Prabandari YS, Syahrul S. Application of video-based health education in improving mother's knowledge and attitudes about behavioral problems among preschool children. Enfermería Clínica 2020 Mar;30:172-176. [CrossRef]
    82. Shah N, Mathur VP, Kathuria V, Gupta T. Effectiveness of an educational video in improving oral health knowledge in a hospital setting. Indian J Dent 2016;7(2):70-75 [FREE Full text] [CrossRef] [Medline]
    83. Mihrshahi S, Jawad D, Richards L, Hunter KE, Ekambareshwar M, Seidler AL, et al. A review of registered randomized controlled trials for the prevention of obesity in infancy. Int J Environ Res Public Health 2021 Mar 02;18(5):2444 [FREE Full text] [CrossRef] [Medline]
    84. Pietrobelli A, Agosti M, MeNu Group. Nutrition in the first 1000 days: ten practices to minimize obesity emerging from published science. Int J Environ Res Public Health 2017 Dec 01;14(12):1491 [FREE Full text] [CrossRef] [Medline]
    85. Ross J, Stevenson F, Lau R, Murray E. Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update). Implement Sci 2016 Oct 26;11(1):146 [FREE Full text] [CrossRef] [Medline]
    86. Fischer B, Peine A, Östlund B. The importance of user involvement: a systematic review of involving older users in technology design. Gerontologist 2020 Sep 15;60(7):e513-e523 [FREE Full text] [CrossRef] [Medline]
    87. World Migration Report 2020. International Organization for Migration (IOM). New York, NY, USA: United Nations; 2019.   URL: https://publications.iom.int/system/files/pdf/wmr_2020.pdf [accessed 2022-03-26]
    88. Pandey M, Maina G, Amoyaw J, Li Y, Kamrul R, Michaels C, et al. Impacts of English language proficiency on healthcare access, use, and outcomes among immigrants: a qualitative study. BMC Health Serv Res 2021 Jul 26;21(1):741 [FREE Full text] [CrossRef] [Medline]
    89. Chae S, Lee YJ, Han HR. Sources of health information, technology access, and use among non-english-speaking immigrant women: descriptive correlational study. J Med Internet Res 2021 Oct 29;23(10):e29155 [FREE Full text] [CrossRef] [Medline]
    90. Baker AE, Procter NG, Ferguson MS. Engaging with culturally and linguistically diverse communities to reduce the impact of depression and anxiety: a narrative review. Health Soc Care Community 2016 Jul;24(4):386-398. [CrossRef] [Medline]
    91. Coughler C, Burke SM, Cardy JO. Analysis of the quality of online resources for parents of children who are late to talk. Autism Dev Lang Impairments 2020 Apr 06;5:239694152091794 [FREE Full text] [CrossRef]
    92. Co-Design Position Statement. Health Care Consumers’ Association Executive Committee. 2020.   URL: https://www.hcca.org.au/wp-content/uploads/2020/06/HCCA-Co-design-position-statement-Final.pdf [accessed 2022-04-04]
    93. Zhang C. Who bypasses the Great Firewall in China? First Monday 2020 Mar 22;25(4):10256 [FREE Full text] [CrossRef]

    ‎
    F-K: Flesch-Kincaid test
    HONcode: Health on the Net Foundation Code of Conduct
    HRWEF: Health-Related Website Evaluation Form
    ICC: intraclass correlation coefficient
    IRR: interrater reliability
    QCSS: Quality Component Scoring System
    REDCap: Research Electronic Data Capture
    SAM: Suitability Assessment of Materials
    SMOG: Simple Measure of Gobbledygook

    Edited by R Kukafka; submitted 11.04.22; peer-reviewed by Q Zhou, B Nievas Soriano; comments to author 29.05.22; revised version received 03.06.22; accepted 15.07.22; published 07.10.22

    Copyright

    ©Danielle Jawad, Heilok Cheng, Li Ming Wen, Chris Rissel, Louise Baur, Seema Mihrshahi, Sarah Taki. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 07.10.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.