This paper is in the following e-collection/theme issue:

Research Instruments, Questionnaires, and Tools (576) Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (832) Instruments and Questionnaires for Nutrition and Food Intake (166)

Published on in Vol 16, No 8 (2014): August

Online Dietary Intake Estimation: Reproducibility and Validity of the Food4Me Food Frequency Questionnaire Against a 4-Day Weighed Food Record

Online Dietary Intake Estimation: Reproducibility and Validity of the Food4Me Food Frequency Questionnaire Against a 4-Day Weighed Food Record

Online Dietary Intake Estimation: Reproducibility and Validity of the Food4Me Food Frequency Questionnaire Against a 4-Day Weighed Food Record

Authors of this article:

Rosalind Fallaize1 Author Orcid Image ;   Hannah Forster2 Author Orcid Image ;   Anna L Macready1 Author Orcid Image ;   Marianne C Walsh2 Author Orcid Image ;   John C Mathers3 Author Orcid Image ;   Lorraine Brennan2 Author Orcid Image ;   Eileen R Gibney2 Author Orcid Image ;   Michael J Gibney2 Author Orcid Image ;   Julie A Lovegrove1 Author Orcid Image
Article Authors Cited by (137) Tweetations (10) Metrics

    Original Paper

    1Hugh Sinclair Unit of Human Nutrition and Institute for Cardiovascular and Metabolic Research, Department of Food and Nutritional Sciences, University of Reading, Reading, United Kingdom

    2UCD Institute of Food and Health, UCD Centre for Molecular Innovation, University College Dublin, Dublin, Ireland

    3Human Nutrition Research Centre, Institute for Health and Ageing, Newcastle University, Newcastle, United Kingdom

    Corresponding Author:

    Julie A Lovegrove, BSc (Hons), PhD

    Hugh Sinclair Unit of Human Nutrition and Institute for Cardiovascular and Metabolic Research

    Department of Food and Nutritional Sciences

    University of Reading

    Whiteknights

    PO Box 226

    Reading, RG6 6AP

    United Kingdom

    Phone: 44 118 378 6418

    Fax:44 118 378 7708

    Email: j.a.lovegrove@reading.ac.uk


    Background: Advances in nutritional assessment are continuing to embrace developments in computer technology. The online Food4Me food frequency questionnaire (FFQ) was created as an electronic system for the collection of nutrient intake data. To ensure its accuracy in assessing both nutrient and food group intake, further validation against data obtained using a reliable, but independent, instrument and assessment of its reproducibility are required.

    Objective: The aim was to assess the reproducibility and validity of the Food4Me FFQ against a 4-day weighed food record (WFR).

    Methods: Reproducibility of the Food4Me FFQ was assessed using test-retest methodology by asking participants to complete the FFQ on 2 occasions 4 weeks apart. To assess the validity of the Food4Me FFQ against the 4-day WFR, half the participants were also asked to complete a 4-day WFR 1 week after the first administration of the Food4Me FFQ. Level of agreement between nutrient and food group intakes estimated by the repeated Food4Me FFQ and the Food4Me FFQ and 4-day WFR were evaluated using Bland-Altman methodology and classification into quartiles of daily intake. Crude unadjusted correlation coefficients were also calculated for nutrient and food group intakes.

    Results: In total, 100 people participated in the assessment of reproducibility (mean age 32, SD 12 years), and 49 of these (mean age 27, SD 8 years) also took part in the assessment of validity. Crude unadjusted correlations for repeated Food4Me FFQ ranged from .65 (vitamin D) to .90 (alcohol). The mean cross-classification into “exact agreement plus adjacent” was 92% for both nutrient and food group intakes, and Bland-Altman plots showed good agreement for energy-adjusted macronutrient intakes. Agreement between the Food4Me FFQ and 4-day WFR varied, with crude unadjusted correlations ranging from .23 (vitamin D) to .65 (protein, % total energy) for nutrient intakes and .11 (soups, sauces and miscellaneous foods) to .73 (yogurts) for food group intake. The mean cross-classification into “exact agreement plus adjacent” was 80% and 78% for nutrient and food group intake, respectively. There were no significant differences between energy intakes estimated using the Food4Me FFQ and 4-day WFR, and Bland-Altman plots showed good agreement for both energy and energy-controlled nutrient intakes.

    Conclusions: The results demonstrate that the online Food4Me FFQ is reproducible for assessing nutrient and food group intake and has moderate agreement with the 4-day WFR for assessing energy and energy-adjusted nutrient intakes. The Food4Me FFQ is a suitable online tool for assessing dietary intake in healthy adults.

    J Med Internet Res 2014;16(8):e190

    doi:10.2196/jmir.3355

    Keywords

    food frequency questionnaire; weighed food record; validity; reproducibility; dietary assessment; Food4Me; Web-based 


    Given the continuing rise in some noncommunicable diseases and the growing burden of diet-related ill health [1-3], researchers are seeking new and innovative ways of facilitating dietary change. These include the application of digital technologies, which are revolutionizing the delivery of health-related services because of their reduced costs and wide reach. Online interventions are particularly promising because they have the potential to increase exposure to health promotion material. Recent estimates show that Internet use has increased by >150% in North America and by nearly 400% in Europe since 2000, with a total of 78.6% and 63.2% of these populations, respectively, now classified as Internet users [4]. Given their lower costs, Internet-based services have the potential to enhance the cost-benefit ratio for interventions aimed at prevention of diet-related noncommunicable diseases [5-6]. Furthermore, interactive Web-based interventions have been shown to increase patient activation and self-management capabilities in chronically ill adults [7] and enhance weight loss in obese individuals (compared with non-Web-based interventions) [8].

    To quantify dietary change in response to an intervention, an accurate and validated means of assessing food intake is essential [9]. Population-level food intake is usually assessed in 1 of 3 ways: a food frequency questionnaire (FFQ), 24-hour recall, or estimated or weighed food record (WFR). The WFR, which involves weighing all foods and drinks consumed over a 3-7 day period, is often considered the most accurate measure of intake and has been referred to as the imperfect gold standard [10]. However, prospective recording of food consumption can alter the type and quantity of foods eaten and, therefore, introduce bias into the estimate of food intake [11-13]. The FFQ and 24-hour recall, which rely on retrospective recording of food consumption, are also prone to reporting bias, including overestimated consumption of “healthy” foods, such as fruit and vegetables, and underestimation of “unhealthy” food intake. WFR require participants to be highly motivated and are labor-intensive for both participants and researchers. Conversely, FFQ are inexpensive to process and can be self-administered electronically, making them suitable for online interventions. Other advantages include reducing paper use, postage costs, and the space; security; and organization required for paper file storage [14]. For this reason, FFQ are most commonly used in large-scale epidemiological and intervention studies to determine food and nutrient intake [15].

    The present research was conducted as part of the Food4Me study, which aims to test the utility of online personalized dietary advice using an online FFQ to assess dietary intake [16,17]. The Food4Me FFQ includes 157 food items and food portion photographs and has been described previously by Forster et al [18]. FFQ are generally validated against existing dietary assessment methods, such as WFR [19], and several FFQ have been validated for electronic and online use recently [14,20-22].

    The Food4Me FFQ has been shown to have good agreement with the European Prospective Investigation of Cancer (EPIC)-Norfolk FFQ for the estimation of energy-adjusted nutrient intakes [18]. The aim of this study is to further validate the Food4Me FFQ against a WFR and to assess its reproducibility using a test-retest methodology.


    Study Sample

    To accurately estimate the Bland-Altman limits of agreement between 2 methods, a sample size of 50-100 is required [23]. Allowing for 20% dropout, 121 participants aged ≥18 years were recruited from the University of Reading, UK, via email and poster advertising. Participants were provided with a study information sheet before participation and were asked to sign an informed consent form. A participant information form, which included self-reported weight and height measurements, was used to assess suitability for the study. Individuals reporting health issues or ill health, self-reported or diagnosed food intolerances, or special nutritional requirements (eg, pregnancy or lactation) were ineligible to participate. Ethical approval for the study was obtained from the School of Chemistry, Food and Pharmacy Research Ethics Committee, University of Reading, UK (01-12-Lovegrove).

    Study Design

    Reproducibility of the Food4Me FFQ was determined by asking participants to complete the questionnaire on 2 occasions 4 weeks apart, mimicking its application in the Food4Me study. To assess the validity of the FFQ against a 4-day WFR, half the sample (those recruited first) were asked to complete a 4-day WFR 1 week following the first administration of the Food4Me-FFQ. Participants who completed both the Food4Me FFQ and 4-day WFR were also asked to complete a dietary record usability-rating questionnaire on Survey Monkey (Survey Monkey Inc, Palo Alto, CA, USA) in the week following the completion of the second Food4Me FFQ. The usability-rating questionnaire included questions about ease of use and willingness to complete the records. Participants were asked not to change their diet during the study.

    Weighed Food Record

    Participants were asked to record all foods and beverages consumed over a nonconsecutive 4-day period that included 3 weekdays (Monday to Thursday) and 1 weekend day (Saturday to Sunday). Before completing the WFR, participants were coached on how to describe food products by a dietitian and provided with weighing scales (Salter Disc Electronic Kitchen Scales SKU# 1036 WHSSDR). When participants were unable to provide weighed portion size information, this was estimated retrospectively within 1 week using the Ministry of Agriculture, Fisheries and Food Portion Size Atlas [24].

    Food4Me FFQ

    The self-administered Food4Me FFQ is an online, semiquantitative food frequency questionnaire (developed by University College Dublin and Crème Software Ltd). To complete the questionnaire, participants were provided with a website address and unique log-in details. On logging into the server, participants were directed to a webpage containing detailed instructions on how to complete the FFQ. The questionnaire contained questions on the average consumption of 157 food items over the previous month. The food items were divided into the following 11 categories: (1) cereal, (2) bread and savory biscuits, (3) potatoes, rice and pasta, (4) meat and fish, (5) dairy products and fat, (6) fats and spreads, (7) sweets and snacks, (8) soups, sauces and spreads, (9) drinks, (10) fruit, and (11) vegetables. During completion of the Food4Me FFQ, participants were required to provide information on frequency of consumption and portion size. Frequency of consumption was measured by selecting one of the following options: never or less than once a month, 1-3 times a month, once a week, 2-4 times a week, 5-6 times per week, once a day, 2-3 times per day, 5-6 times per day, and >6 times per day. Food portion size was estimated using photographs. Each food item had 3 photographs representing small, medium, and large portions and these descriptors were provided below the appropriate image. Participants could select one of the following options: very small, small, small/medium, medium, medium/large, large, or very large which were linked electronically to portion sizes (in grams) (see Figure 1). Food intake (g/day) was calculated by multiplying frequency of consumption by the specified portion size (see Forster et al for detailed methods [18]). Further screenshots of the online Food4Me FFQ are shown in Multimedia Appendix 1.

    ‎
    Figure 1. Screenshot of the online Food4Me food frequency questionnaire.
    View this figure

    Dietary Record Usability-Rating Questionnaire

    The dietary record usability-rating questionnaire was comprised of 5 questions about the completion of the Food4Me FFQ and the 4-day WFR. Participants who completed both the Food4Me FFQ and the 4-day WFR (n=49) were asked to select one of the following responses to indicate their level of agreement: strongly agree, agree, neither agree nor disagree, disagree, or strongly disagree to the following questions:

    1. I found the Food4Me FFQ / 4-day WFR easy to complete
    2. I found the Food4Me FFQ / 4-day WFR too time consuming
    3. I found the Food4Me FFQ / 4-day WFR interesting to complete
    4. I found the Food4Me FFQ / 4-day WFR made me reflect on my intake
    5. In the future I would be willing to complete more Food4Me FFQ/ 4-day WFR

    Misreporting

    The Henry equation [25] was used to calculate basal metabolic rate (BMR), and BMR was multiplied by a physical activity level (PAL) of 1.1 to calculate the lowest possible estimated energy requirements (EER) for each participant [26]. Participants reporting energy intakes lower than their EER were classified as underreporters. Participants reporting a daily energy intake greater than 4500 kcal, which is considered implausibly high, were excluded from the analysis [27].

    Nutritional Intake Analysis

    Estimated nutritional intake data from the Food4Me FFQ were generated automatically by the online Food4Me programmed system, as described by Forster et al [18]. Composition of the food items listed in the FFQ were derived from WISP (Tinuviel Software, Anglesey, UK) [28] and modified to include recipes of composite dishes, generic commercial foods, new foods on the market, and current manufacturers information. The 4-day WFR intakes were analyzed using WISP (Tinuviel Software, Anglesey, UK) [28]. For the purpose of the current study, consumption of dietary supplements was not included in the analyses.

    Statistical Analysis

    Statistical analyses were performed using SPSS version 20.0 (IBM Corp, Armonk, NY, USA). Mean nutrient intakes and standard deviations were calculated for baseline characteristics, repeated Food4Me FFQ, and 4-day WFR. Differences in participant characteristics and energy intakes (kcal) were assessed using a paired 2-sample t test. Nutrient intakes were compared using general linear model (GLM) analysis controlling for energy and gender where there was significant interaction between gender and nutrient intake. Data were checked for normality using the Shapiro-Wilk test and, depending on the outcome, the association between dietary intake methods and repeated Food4Me FFQ were assessed using either Pearson product-moment correlation or Spearman correlation coefficient (SCC, rho). A P value of <.05 was considered statistically significant.

    The relative agreement between the dietary intake methods and repeated Food4Me FFQ was assessed using cross-classification of nutrient intakes to estimate the percentage of participants classified into quartiles as follows: exact agreement (percentage of cases classified into the same quartile), exact agreement plus adjacent (percentage of cases cross-classified into the same or adjacent quartile), disagreement (percentage of cases cross-classified 2 quartiles apart), and extreme disagreement (percentage of cases cross-classified into extreme quartiles). For intakes of energy and macronutrients, the Bland-Altman method [29] was used to further assess the limits of agreement between the 2 methods (Food4Me FFQ and WFR) and between the repeated Food4Me FFQ. As per the Bland-Altman methodology, dietary records were considered comparable/ repeatable if greater than 95% of data plots lay within 2 standard deviations of the mean. GraphPad PRISM version 6 was used to produce the Bland-Altman plots (GraphPad Software, Inc, La Jolla, CA, USA).

    Differences in food group intakes between the FFQ and WFR and repeated Food4Me FFQ were also examined. For this purpose, food items in the Food4Me FFQ and 4-day WFR were arranged into 35 food groups as per previous validation by Forster et al [18]. SCC were calculated to assess the strength of association between methods for estimated intakes of the 35 food groups. To assess the relative agreement between the dietary methods and repeated FFQ for daily food group intake, food groups were also cross-classified to estimate the percentage of participants classified by the 2 methods into quartiles of exact agreement, exact agreement plus adjacent, disagreement, and extreme disagreement.


    Summary

    A total of 121 participants were screened for inclusion in the study, of which 113 were deemed eligible. Reasons for exclusion included self-diagnosed food intolerance (n=7) and medication use (n=1). Before completion, 10 participants dropped out of the study and a further 3 were excluded from analysis due to a reported energy intake >4500 kcal [26]. The final dataset for analysis included 100 participants, of whom 49 had also completed the 4-day WFR, as illustrated in Figure 2. A total of 48 participants completed the Diet Record Usability-Rating Questionnaire.

    ‎
    Figure 2. Flow of participants through the study.
    View this figure

    Overview of the Study Population

    Self-reported demographic characteristics of the participants in the reproducibility and validation study are shown in Table 1. No significant differences were observed between age and body mass index (BMI) for males and females. Participants who completed the WFR (validation study) were, on average, 4.6 years younger than the participant group as a whole.

    Table 1. Demographic characteristics of the participants who completed the validation and reproducibility studies according to gender.
    StudyNDemographic characteristics, mean (SD)


    		Age (years)BMI (kg/m2)a
    Reproducibility



    		Males3130.1 (12.7)24.3 (3.1)

    		Females6932.1 (11.6)23.3 (3.3)

    		All10031.5 (11.9)23.6 (3.3)
    Validation



    		Males1524.2 (7.6)23.1 (3.2)

    		Females3427.9 (8.6)22.2 (2.6)

    		All4926.9 (8.4)22.5 (2.8)

    aBMI based on self-reported weight and height.

    Reproducibility of the Food4Me FFQ

    Comparison of Nutrient Intakes Between Repeated Food4Me FFQ

    Mean energy and nutrient intakes estimated by repeated measures of the Food4Me FFQ (FFQ1 and FFQ2) are presented in Table 2. Estimated energy intakes were significantly higher in the first administration of the FFQ compared with the second administration (difference=135 kcal/day, equivalent to 6.5% higher, P<.05). With the exception of carbohydrate, no significant differences were observed between macronutrient and micronutrient intakes estimated by FFQ1 and FFQ2. Overall, the Food4Me FFQ showed good reproducibility for energy-adjusted nutrient intakes. A total of 16 participants were found to underreport in both FFQ with a further 3 underreporting in FFQ1 and 5 in FFQ2. The removal of underreporters from both FFQ (n=24) did not impact on the reproducibility of the questionnaire (data not shown).

    Bland-Altman plots for estimates of energy (kcal), total fat (% total energy, TE), protein (%TE) and carbohydrate (%TE) intakes are shown in Figure 3. The Food4Me FFQ showed good reproducibility for the estimation of daily protein intake, with less than 5% of cases falling outside of the limits of agreement. For energy and total fat intake, 6% of cases fell outside of the limits of agreement and for carbohydrate 7%, indicating similar reproducibility. The mean difference (bias) between energy intakes was relatively small (135 kcal/day) with greater values being estimated at FFQ1. Conversely, estimates of energy-adjusted protein and total fat intake were higher at FFQ2 with biases of –0.22 %TE and –1.23 %TE, respectively. In contrast with the energy-adjusted macronutrient intakes, variation between estimates of energy increased with higher mean energy intakes (Figure 3), suggesting poorer reproducibility for those participants reporting higher energy intakes.

    Correlation coefficients for estimates of energy and nutrient intakes between repeated administrations of the Food4Me FFQ are shown in Table 3. Correlation coefficients ranged from .65 (vitamin D) to .90 (alcohol) with a mean value of .75. Correlations were significant for all nutrients (P<.01). The cross-classification of quartiles of mean estimated daily energy and nutrient intakes between repeated administrations of the Food4Me FFQ is also shown in Table 3. The percentage of participants classified into quartiles of exact agreement ranged from 45% (polyunsaturated fatty acids %TE) to 74% (vitamin A retinol equivalents, RE). For classifications of exact agreement plus adjacent, values were consistently high, ranging from 87% (vitamin D) to 98% (vitamin A RE). The mean percentage of participants classified into quartiles of disagreement was 7% with less than 1% of participants on average classified into extreme disagreement.

    Table 2. Mean daily energy and nutrient intakes estimated by repeated measures of the online Food4Me FFQ and general linear model (GLM) results (N=100).
    NutrientaQuestionnaire, mean (SD)GLM analysis, P

    		Food4Me FFQ1Food4Me FFQ2Controlled for energyControlled for energy and genderb
    Energy (kcal)2223.8 (766.2)2088.8 (705.4).008c—
    Total fat (g)85.3 (33.8)82.0 (32.5).24.24
    Total fat (%TE)34.1 (4.9)35.1 (6.1).21.21
    SFA (g)33.2 (14.2)32.2 (14.5).13.13
    SFA (%TE)13.2 (2.4)13.6 (3.0).23.23
    MUFA (g)32.0 (13.4)30.6 (12.7).38.38
    MUFA (%TE)12.8 (2.5)13.1 (2.8).36.36
    PUFA (g)14.4 (5.6)13.7 (5.3).91.98
    PUFA (%TE)5.8 (1.27)6.0 (1.5).48.62
    Omega 3 (g)1.70 (0.71)1.61 (0.66).92.92
    Protein (g)90.6 (35.2)84.4 (30.8).70.70
    Protein (%TE)16.3 (2.8)16.4 (4.5).84.84
    Carbohydrate (g)263.9 (87.6)238.6 (89.6).06.03
    Carbohydrate (%TE)45.1 (6.5)43.1 (7.9).05.03
    Total sugars (g)125.6 (49.3)115.9 (56.1).65.79
    Total sugars (%TE)21.4 (5.8)20.6 (5.8).31.32
    Alcohol (g)12.9 (16.2)12.8 (15.7).61.60
    Calcium (mg)1085.0 (378.1)1008.0 (416.5).59.64
    Total folate (µg)361.0 (120.0)335.9 (416.5).43.43
    Iron (mg)14.9 (5.3)13.4 (4.3).05.05
    Total carotene (µg)6209.6 (4590.8)5482.1 (3645.2).43.60
    Riboflavin (mg)2.24 (0.80)2.11 (0.89).87.98
    Thiamin (mg)2.87 (2.80)2.91 (3.33).76.76
    Vitamin B6 (mg)2.54 (0.86)2.35 (0.78).31.31
    Vitamin B12 (µg)7.32 (3.57)6.72 (3.60).64.64
    Vitamin C (mg)167.8 (82.5)153.8 (74.7).42.51
    Vitamin A RE (µg)1160.9 (1015.8)1057.8 (907.9).79.79
    Retinol (µg)502.9 (408.0)470.1 (400.5).99.99
    Vitamin D (µg)3.89 (2.39)3.51 (1.90).48.38
    Vitamin E (mg)10.61 (4.05)9.77 (3.86).39.29
    Salt (g)6.30 (2.70)5.92 (2.24).97.97

    aMUFA: monounsaturated fatty acids; PUFA: polyunsaturated fatty acids; RE: retinol equivalents; SFA: saturated fatty acids; TE: total energy.

    bControlled for gender where appropriate. No significant interactions were observed between method and gender.

    cP value derived using 2-samples paired t test.

    Table 3. Unadjusted correlation coefficients and cross-classification of quartiles of mean energy and nutrient intakes derived from repeat measures of the online Food4Me FFQ (N=100).
    NutrientaCorrelationbQuartiles, %


    		Exact agreementcExact agreement plus adjacentdDisagreementeExtreme disagreementf
    Energy (kcal).77g579091
    Total fat (g).81649280
    Total fat (%TE).72569172
    SFA (g).81609190
    SFA (%TE).704688120
    MUFA (g).80519640
    MUFA (%TE).705389101
    PUFA (g).78519532
    PUFA (%TE).68459262
    Omega 3 (g).78589190
    Protein (g).805688120
    Protein (%TE).73599370
    Carbohydrate (g).74539640
    Carbohydrate (%TE).73628992
    Total sugars (g).77669442
    Total sugars (%TE).696188111
    Alcohol (g).90709640
    Calcium (mg).73559271
    Total folate (µg).74539361
    Iron (mg).75539541
    Total carotene (µg).766090100
    Riboflavin (mg).73569082
    Thiamin (mg).71519163
    Vitamin B6 (mg).725689101
    Vitamin B12 (µg).73649550
    Vitamin C (mg).72609550
    Vitamin A (RE) (µg).90749820
    Retinol (µg).67509073
    Vitamin D (µg).655287121
    Vitamin E (mg).75569172
    Salt (g).78579082

    aMUFA: monounsaturated fatty acids; PUFA: polyunsaturated fatty acids; RE: retinol equivalents; SFA: saturated fatty acids; TE: total energy.

    bCorrelation is significant at the .01 level (2-tailed) for all nutrients analyzed.

    cExact agreement, % of cases cross-classified into the same quartile.

    dExact agreement plus adjacent, % of cases cross-classified into the same or adjacent quartile.

    eDisagreement, % of cases cross-classified 2 quartiles apart.

    fExtreme disagreement, % of cases cross-classified into extreme quartiles.

    gPearson correlation.

    ‎
    Figure 3. Reproducibility study Bland-Altman plots for (a) energy, (b) total fat, (c) protein, and (d) carbohydrate with the mean difference and limits of agreement. The solid line represents the mean difference and the dotted lines represent the limits of agreement.
    View this figure
    Comparison of Food Group Intakes Between Repeated Food4Me FFQ

    To assess differences in food group intake between repeated administrations of the online Food4Me FFQ, food items were categorized into 35 food groups. Correlation coefficients and cross-classification of mean food group intakes are presented in Table 4. SCC ranged from .55 (tinned fruit or vegetables) to .92 (alcoholic beverages) with a mean value of .75. Correlations were significant for all food groups (P<.01). The percentage of participants classified into quartiles of exact agreement ranged from 46% (fats and oils) to 86% (tinned fruit or vegetables). For classifications of exact agreement plus adjacent values were consistently high, ranging from 81% (eggs and egg dishes) to 99% (alcoholic beverages). The mean percentage of participants classified into quartiles of disagreement was 7% and for extreme disagreement was 1%.

    Table 4. Spearman correlation coefficients (SCC) and cross-classification of quartiles of food group intake derived from repeat measures of the online Food4Me FFQ (n=100).
    Food groupSCCaQuartile, %


    		Exact agreementbExact agreement plus adjacentcDisagreementdExtreme disagreemente
    Rice, pasta, grains and starches.78569280
    Savories (lasagne, pizza).70528992
    White bread (rolls, tortillas, crackers).83629460
    Wholemeal, brown breads, and rolls.77609181
    Breakfast cereals and porridge.90679640
    Biscuits.564886113
    Cakes, pastries and buns.645287103
    Milk.74499172
    Cheeses.665287112
    Yogurts.79589721
    Ice cream, creams and desserts.77559442
    Eggs and egg dishes.695681163
    Fats and oils (eg, butter, low-fat spreads, hard cooking fats).64468983
    Potatoes and potato dishes.61518695
    Chipped, fried & roasted potatoes.615787121
    Peas, beans and lentils and vegetable and pulse dishes.75629271
    Green vegetables.76549361
    Carrots.68549082
    Salad vegetables (eg, lettuce).77589262
    Other vegetables (eg, onions).85569721
    Tinned fruit or vegetables.558686140
    Bananas.81609550
    Other fruits (eg, apples, pears, oranges).86619730
    Nuts and seeds, herbs and spices.776884133
    Fish and fish products/dishes.84579550
    Bacon and ham.88739730
    Red meat (eg, beef, veal, lamb, pork).74679091
    Poultry (chicken and turkey).75549361
    Meat products (eg, burgers, sausages, pies, processed meats).85629370
    Alcoholic beverages.92719910
    Sugars, syrups, preserves, and sweeteners.78809460
    Confectionary and savory snacks.73529280
    Soups, sauces, and miscellaneous foods.69599082
    Teas and coffees.85699631
    Other beverages (eg, fruit juices, carbonated beverages, squash).75549541

    aCorrelation is significant at the .01 level (2-tailed) for all nutrients analyzed.

    bExact agreement, % of case cross-classified into the same quartile.

    cExact agreement plus adjacent, % of cases cross-classified into the same or adjacent quartile.

    dDisagreement, % of cases cross-classified 2 quartiles apart.

    eExtreme disagreement, % of cases cross-classified into extreme quartiles.

    Validation of the Food4Me FFQ

    Comparison of Nutrient Intakes Between the Food4Me FFQ and 4-Day WFR

    Mean energy and nutrient intakes estimated by the 4-day WFR and Food4Me FFQ (FFQ1) are presented in Table 5. There were no significant differences between estimates of energy intake. However, when underreporters had been removed (n=19), a significant difference in intake was observed (P<.05) (Table 5). In total, 12 participants underreported in the 4-day WFR and 15 underreported in the Food4Me FFQ1, with 8 of these underreporting in both methods.

    Table 5. Mean daily energy and nutrient intakes estimated by online Food4Me FFQ and 4-day WFR and general linear model (GLM) results (n=49).
    NutrientaQuestionnaire, mean (SD)GLM analysis, P

    		Food4Me FFQ14-day WFRControlled for energyControlled for energy and genderb
    Energy (kcal)2115.2 (809.1)1936.9 (505.8).11c—
    Total fat (g)79.6 (36.2)68.6 (22.2).10.10
    Total fat (%TE)33.1 (4.5)31.6 (5.1).13.13
    SFA (g)45.6 (15.6)24.3 (10.4)<.001<.001
    SFA (%TE)13.1 (2.3)11.0 (2.9)<.001<.001
    MUFA (g)29.8 (4.5)21.4 (7.3)<.001<.001
    MUFA (%TE)12.4 (2.6)9.8 (2.0)<.001<.001
    PUFA (g)12.7 (4.9)10.7 (4.6).12.12
    PUFA (%TE)5.44 (0.9)4.97 (1.6).10.10
    Protein (g)87.2 (36.0)77.2 (21.4).31.31
    Protein (%TE)16.5 (2.9)16.1 (2.6).40.40
    Carbohydrate (g)253.4 (94.1)248.3 (54.9).20.20
    Carbohydrate (%TE)45.6 (6.6)48.9 (6.5).01.01
    Total sugars (g)119.1 (46.7)102.8 (37.8).18.18
    Total sugars (%TE)21.5 (5.5)20.1 (6.2).25.25
    Alcohol (g)13.0 (14.5)11.6 (22.2).50.50
    Calcium (mg)1043.8 (386.8)865.8 (285.5).003.001
    Total folate (µg)337.6 (124.6)273.8 (139.5).05.11
    Iron (mg)14.1 (5.4)13.0 (5.6).98.98
    Total carotene (µg)5011.4 (3321.2)2725.3 (2995.3).001.001
    Riboflavin (mg)2.27 (0.83)1.85 (0.82).04.04
    Thiamin (mg)2.22 (1.56)2.19 (3.26).98.98
    Vitamin B6 (mg)2.44 (0.83)2.09 (0.70).06.06
    Vitamin B12 (µg)6.85 (3.31)4.63 (2.16)<.001<.001
    Vitamin C (mg)148.2 (77.0)106.6 (73.1).02.05
    Retinol (µg)426.1 (330.3)236.2 (137.7).001.001
    Vitamin D (µg)3.47 (2.15)2.55 (1.61).049.049
    Vitamin E (mg)9.11 (3.36)7.84 (2.77).13.13
    Salt (g)5.91 (2.7)6.48 (2.1)<.001<.001

    aMUFA: monounsaturated fatty acids; PUFA: polyunsaturated fatty acids; RE: retinol equivalents; SFA: saturated fatty acids; TE: total energy.

    bControlled for gender where appropriate. No significant interactions were observed between method and gender.

    cP value derived using 2-samples paired t test.

    After controlling for energy, estimated intakes of macronutrients were similar for both the WFR and the Food4Me FFQ with no significant differences between intakes of total fat (g, TE), polyunsaturated fatty acids (PUFA g, %TE), protein (g, %TE), carbohydrate (g), and total sugars (g, %TE) (Table 5). However, estimated intakes of saturated fatty acids (SFA) (g, %TE) and monounsaturated fatty acids (MUFA) (g, %TE) were significantly higher (P<.001), and estimated intake of carbohydrate (%TE) was significantly lower (P=.01), for the FFQ than for the WFR. For micronutrients, no significant differences were observed between energy-controlled estimates of folate, iron, thiamin, vitamin B6, and vitamin E. Estimated intakes of calcium, total carotene, riboflavin, vitamin B12, vitamin C, retinol, and vitamin D intakes were significantly different between 4-day WFR and FFQ1 (all were higher for the FFQ). After controlling for energy and, where appropriate, gender, vitamin C intakes were no longer significantly different. Removing underreporters from the dataset reduced the agreement between the 2 methods for folate, vitamin B6, and vitamin E, but improved agreement for energy-controlled carbohydrate and vitamin D, with no significant differences observed between estimates of these nutrients (data not shown).

    Bland-Altman plots for mean energy (kcal), total fat (%TE), protein (%TE) and carbohydrate (%TE) for the 4-day WFR and FFQ1 are shown in Figure 4. Overall, less than 5% of cases fell outside of the limits of agreement for all plots indicating good agreement between the methods. The mean difference (bias) between energy intakes was relatively small (178 kcal/day) with greater values being estimated in the Food4Me FFQ, as was the case for energy derived from total fat and protein.

    Correlation coefficients for estimates of energy and nutrient intakes and cross-classification of quartiles of mean daily intakes between 4-day WFR and FFQ1 are presented in Table 6. Correlation coefficients ranged from .23 (vitamin D) to .65 (protein, %TE) with a mean value of .47. Correlation was significant for the majority of nutrients at the P<.01 level, with the exception of total fat (%TE), PUFA (%TE), and vitamin D. Retinol and vitamin E showed significant correlation at the P<.05 level. The percentage of participants classified into quartiles of exact agreement ranged from 22% (total fat, %TE) to 53% (MUFA, g). For classifications of exact agreement plus adjacent, values were consistently high, ranging from 65% (sodium) to 88% (total fat, g, and total sugars, g, %TE). The mean percentage of participants classified into quartiles of disagreement was 16% with less than 4% of participants classified into extreme disagreement.

    Table 6. Unadjusted correlation coefficients and cross-classification of quartiles of mean energy and nutrient intakes derived from the online Food4Me FFQ and 4-day WFR (n=49).
    NutrientaCorrelationQuartiles, %


    		Exact agreementbExact agreement plus adjacentcDisagreementdExtreme disagreemente
    Energy (kcal).53f,h4184122
    Total fat (g) .56h3788102
    Total fat (%TE).272276168
    SFA (g).48h3782144
    SFA (%TE).38f,h2478148
    MUFA (g).56h538668
    MUFA (%TE).45h478686
    PUFA (g).45h4976222
    PUFA (%TE).242771244
    Protein (g).59h4584144
    Protein (%TE).65f,h4586140
    Carbohydrate (g).43f,h3782108
    Carbohydrate (%TE).59f,h4982180
    Total sugars (g).60f,h4188102
    Total sugars (%TE).61f,h4588102
    Alcohol (g).61h4580164
    Calcium (mg).47f,h4173170
    Total folate (µg).58h4586104
    Iron (mg).50h4182144
    Total carotene (µg).42h3378184
    Riboflavin (mg).50h4584142
    Thiamin (mg).60h4382162
    Vitamin B6 (mg).44h3778202
    Vitamin B12 (µg).46h3978202
    Vitamin C (mg).54h3784142
    Retinol (µg).31g3776186
    Vitamin D (µg).232767276
    Vitamin E (mg).30g3378148
    Sodium (mg).37h4965314
    Salt (g).37h4965314

    aMUFA: monounsaturated fatty acids; PUFA: polyunsaturated fatty acids; RE: retinol equivalents; SFA: saturated fatty acids; TE: total energy.

    bExact agreement, % of cases cross-classified into the same quartile.

    cExact agreement plus adjacent, % of cases cross-classified into the same or adjacent quartile.

    dDisagreement, % of cases cross-classified 2 quartiles apart.

    eExtreme disagreement, % of cases cross-classified into extreme quartiles.

    fPearson correlation.

    gP<.05.

    hP<.01.

    ‎
    Figure 4. Validation study Bland-Altman plots for (a) energy, (b) total fat, (c) protein, and (d) carbohydrate with the mean difference and limits of agreement. The solid line represents the mean difference and the dotted lines represent the limits of agreement.
    View this figure
    Comparison of Food Group Intakes Between the Food4Me FFQ and 4-Day WFR

    To assess differences in food group intakes between 4-day WFR and FFQ1, food items were categorized into 35 food groups. Correlation coefficients and cross-classification of mean food group intakes are presented in Table 7. SCC ranged widely from .11 (soups, sauces and miscellaneous foods) to .73 (yogurts) with a mean value of .2. Correlations were significant for the 63% of food groups (22 of 35).

    The percentage of participants classified into quartiles of exact agreement ranged from 18% (nuts and seeds, herbs and spices) to 55% (teas and coffees). For classifications of exact agreement plus adjacent, values were high ranging from 55% (soups, sauces and miscellaneous foods) to 90% (milk, chipped, fried and roast potatoes, teas and coffees, and other beverages) with a mean of 78%. The mean percentage of participants classified into quartiles of disagreement was 17% and for extreme disagreement was 5%.

    Table 7. Spearman correlation coefficients (SCC) and cross-classification of quartiles of food group intake derived from the online Food4Me FFQ and 4-day WFR (n=49).
    Food groupSCCQuartiles, %


    		Exact agreementaExact agreement plus adjacentbDisagreementcExtreme disagreementd
    Rice, pasta, grains and starches.34e4573224
    Savories (lasagne, pizza).162276222
    White bread (rolls, tortillas, crackers).46f4273224
    Wholemeal and brown breads and rolls.33e3165296
    Breakfast cereals and porridge.273178148
    Biscuits.47f4178166
    Cakes, pastries and buns.31e3173188
    Milk.68f519082
    Cheeses.46f4182144
    Yogurts.73f439640
    Ice cream, creams and desserts.213578184
    Eggs and egg dishes.55f4376204
    Fats and oils (eg, butter, low-fat spreads, hard cooking fats) .35e3580128
    Potatoes and potato dishes.38f3171208
    Chipped, fried and roasted potatoes.52f399082
    Peas, beans and lentils and vegetable and pulse dishes.232476168
    Green vegetables.44f3786104
    Carrots.142082162
    Salad vegetables (eg, lettuce).2339691812
    Other vegetables (eg, onions).1541731016
    Tinned fruit or vegetables.162971226
    Bananas.45f2980200
    Other fruits (eg, apples, pears, oranges).47f4976222
    Nuts and seeds, herbs and spices.231878148
    Fish and fish products/dishes.60f4982162
    Bacon and ham.53f3984160
    Red meat (eg, beef, veal, lamb, pork).264380164
    Poultry (chicken and turkey).58f4586104
    Meat products (eg, burgers, sausages, pies, processed meats).202778220
    Alcoholic beverages.59f4373224
    Sugars, syrups, preserves and sweeteners.36f3776168
    Confectionary and savory snacks.252971226
    Soups, sauces and miscellaneous foods.1124553114
    Teas and coffees.62f559064
    Other beverages (eg, fruit juices, carbonated beverages, squash).66f439082

    aExact agreement, % of case cross-classified into the same quartile.

    bExact agreement plus adjacent, % of cases cross-classified into the same or adjacent quartile.

    cDisagreement, % of cases cross-classified 2 quartiles apart.

    dExtreme disagreement, % of cases cross-classified into extreme quartiles.

    eP<.05.

    fP<.01.

    Usability Rating

    Mean values and standard deviations for responses to the dietary record usability-rating questionnaire are shown in Table 8.

    The Food4Me FFQ was considered significantly easier and less time consuming to complete than the 4-day WFR. However, the 4-day WFR was rated as significantly more interesting than the Food4Me FFQ, making participants reflect more on their food intake. Participants were more willing to complete further Food4Me FFQ than 4-day WFR.

    Table 8. Responses to Dietary Record Usability-Rating Questionnaire (n=48).
    QuestionQuestionnairea, mean (SD)P valueb

    		Food4Me FFQ4-day WFR
    1. Easy to complete1.89 (0.71)2.13 (0.88)<.001
    2. Too time consuming3.43 (1.09)3.00 (0.94)<.001
    3. Interesting to complete2.20 (0.69)2.07 (0.72).006
    4. Made me reflect on my intake2.13 (0.58)1.89 (0.77).002
    5. I would be willing to complete more1.78 (0.70)2.07 (0.77)<.001

    a1=strongly agree, 2=agree, 3=neither agree nor disagree, 4=disagree, 5=strongly disagree.

    bP value derived using 2-samples paired t test.


    Main Findings and Comparisons With Other Work

    Previous validation of the Food4Me FFQ has demonstrated good agreement with the printed EPIC-Norfolk FFQ for the estimation of food and nutrient intake. In this study, participants were asked to complete the Food4Me and EPIC-Norfolk FFQ in a random order, 4 weeks apart. Good agreement between cross-classifications of daily energy and nutrient intakes, estimated using the 2 FFQ, demonstrated the utility of the Food4Me FFQ for ranking individuals based on their nutrient intake. However, it was noted that further testing of the Food4Me FFQ was required to establish its wider utility [18]. The present study thus aimed to demonstrate the reproducibility of the Food4Me FFQ and its validity against a 4-day WFR.

    Overall, the Food4Me FFQ demonstrated good reproducibility for the estimation of intakes of nutrients and food groups. Reported energy intakes were significantly lower with the second administration of the Food4Me FFQ, but correlations between energy intakes were high (r=.77). Correlation coefficients for nutrient intakes ranged from .65-.90, showing above-average performance compared with the range of .50-.80 proposed by Willet [15]. The mean unadjusted correlation coefficient (r=.75) for energy and nutrient intake compared well with previous studies on both computerized [21,30-31] and non-Web-based FFQ [23,32-35]. Associations between food group intakes were similarly strong with an average unadjusted SCC of .75; previous studies have reported correlations of .66 and .72 [36-37]. Cross-classification analysis of repeated measures of intakes of energy, nutrients, and food groups indicated a high level of reproducibility with classification into quartiles of exact agreement plus adjacent averaging 92% for energy and nutrient intake and for food group intakes. Cross-classifications were within the range reported by previous studies [21,32,38]. Bland-Altman plots demonstrated a good level of reproducibility for energy-controlled total fat, protein, and carbohydrate intake, which reinforces evidence for the reliability of the Food4Me FFQ.

    Estimated energy and nutrient intakes were higher on the first administration of the Food4Me FFQ than on the second administration. This pattern has been observed in numerous other reproducibility studies [21,31-32], and is proposed to result from learning effects and questionnaire fatigue [39]. The above-average reproducibility of the Food4Me FFQ could be attributed to the addition of food photographs to the FFQ for the estimation of portion size intake. Use of tools that allow participants to report their own portion sizes tend to report higher correlation coefficients between repeat administrations [19]. The relatively short interval between repeat administrations of the FFQ is another factor likely to have contributed to the questionnaire’s good performance. Tsubono et al [40] found that correlation coefficients tended to be lower when FFQ were repeated after a long time interval (6 months to 1 year) compared with a shorter time interval (1 to 6 months), and proposed that the temporal difference may be due to changes in dietary habits which are more likely to occur with longer time intervals. In addition, it has also been suggested that for very short time intervals between administrations, respondents may remember and replicate their entries rather than reporting their diet intake accurately [23]. However, with a large FFQ containing 157 food items, as used in the present study, it is unlikely that many participants would be able to remember their earlier responses.

    The degree of underreporting between the Food4Me FFQ and 4-day WFR varied, with 12 (24%) and 15 (15%) participants deemed to be underreporting in the 4-day WFR and Food4Me FFQ, respectively. Given that the WFR is described as the gold standard for assessing intake, our observation that estimates of energy intake were similar between the Food4Me FFQ and the 4-day WFR and that a smaller proportion of participants appeared to underreport with the Food4Me FFQ suggests that the Food4Me FFQ is a promising tool for estimating habitual food intake.

    Overall, the results of the validation study showed moderate agreement between the Food4Me FFQ and 4-day WFR for the estimation of energy and nutrient intake. Ranks of energy and nutrient intake estimated using the Food4Me FFQ were highly comparable to the 4-day WFR with the percentage of individuals classified into quartiles of exact agreement and exact agreement plus adjacent averaging 40% and 80%, respectively. Previous studies comparing FFQ with food records have reported average exact agreement classifications between 34% and 49% [32,38,41] and exact agreement plus adjacent quartile classifications of 77% [21]. Estimates of intake showing disagreement between measurement tools in the present study were small and were comparable with the aforementioned studies. Cross-classifications of estimates of food group intake were similar to that of the nutrients, with classification into quartiles of exact agreement plus adjacent averaging 78% and Bland-Altman plots demonstrated good agreement between the 2 methods for estimates of energy and energy-adjusted macronutrient intakes.

    In the present study, 22 of 30 nutrients assessed had a correlation coefficient greater than the .4 threshold that was proposed by Cade at al. [23], and 13 of 30 achieved a correlation greater than or equal to the “desirable” .5 proposed by Masson et al [42]. The average unadjusted correlation coefficient of .47 compared favorably with the range reported by similar validation studies comparing FFQ with food records: .34-.46 [31-32,34,41]. SCC for food group intakes were highly variable, ranging from .11 (soups, sauces and miscellaneous foods) to .73 (yogurts), with a mean value of .2. Similar studies have reported correlations ranging from .09 to .83 [37], .17 to .95 [36], and .09 to .58 [41] with mean values of .38, .63, and .58, respectively. However, it is difficult to compare results from these studies because the type of food records and time intervals between dietary assessments differed substantially and there may be substantial differences in the food items included in particular food groups in each of the studies. Variation between Food4Me FFQ and 4-day WFR estimates were greatest for soups, sauces and spreads, carrots, other fruit and vegetables, and tinned fruit and vegetables. It is possible that intakes of these foods might have been overestimated in the Food4Me FFQ, as has been observed previously when several food items within a food group are listed separately in a questionnaire (eg, carrots could be counted under both fresh and tinned carrots and under tinned vegetables) [43]. In addition, foods perceived as healthy, such as fruit and vegetables, are prone to overestimation in FFQ. Furthermore, because they refer to just 4 days’ intake, WFR provide a limited snapshot of an individual’s diet only and are less able to assess patterns of dietary intake than the Food4Me FFQ, which attempts to capture intakes over the previous month. It is thought that individuals may be able to more accurately estimate the consumption of some foods (eg, alcoholic beverages) than others, as was the case in the present study [21,44]. Alcohol is often considered a confounder in nutrition research given that it constitutes the difference between food and total dietary energy intake; therefore, it is important that it is estimated reliably using the Food4Me FFQ. It is also encouraging that estimates of fish products were well correlated, as these foods are eaten less frequently and may be prone to underrepresentation in 4-day WFR. However, it is surprising that some more commonly consumed foods such as breakfast cereals and porridge show much weaker correlation (r=.27).

    Our observation that participants in the present study reported that the Food4Me FFQ was easier to use and less time consuming compared with the 4-day WFR, is promising given the movement of health service delivery toward Web-based interventions. Moreover, completion of the Food4Me FFQ was associated with less reflection by participants on their dietary intake, which is known to influence eating behavior. Minimizing the impact of a questionnaire on dietary behavior is beneficial in nutrition intervention studies to ensure that study outcomes are not biased by the methods used for dietary assessment.

    Strengths and Limitations

    Strengths of the present study include the comparison of the Food4Me FFQ with the gold standard, a WFR, and the use of multiple methods to assess the validation and reproducibility of the Food4Me FFQ. In addition, this validation study had an adequate sample size [15] similar to those used in previous studies [30,45-46]. It should be noted that the validation of the Food4Me FFQ was assessed in a convenient rather than a nationally representative sample of the population, although the inclusion/exclusion criteria used were the same as those intended for the Food4Me study [16]. The use of a convenient university population, with a potentially higher education level, may have implications on the ability of the wider population to complete the online Food4Me FFQ.

    Limitations of the study include the use of those recruited first to complete the 4-day WFR because these individuals may have been more motivated to comply with the guidelines. A further limitation is the use of nonconsecutive days in the 4-day WFR, which may have resulted in participants making up food intake on the days they do not record (eg, eating healthy for the record days and overconsuming between record days). However, nonconsecutive recording does have the advantage of capturing a greater diversity of food intake over a week’s period.

    A potential criticism of the assessment of reproducibility is the short duration between repeated measures. It has been suggested that for very short time intervals, respondents may remember and replicate their entries rather than accurately reporting their dietary intake [23], but this is unlikely to be a significant problem in the present study where the Food4Me FFQ contained 157 food items. Remembering their responses to such a long list of questions after a period of 4 weeks is a memory challenge beyond most people’s abilities. The average reported correlation coefficient for crude total fat intake using FFQs repeated after 1 month or less was .68 [19], which compares very favorably with the correlation coefficient of .81 in the present study, showing above-average performance for the Food4Me FFQ. Cade et al [23] suggested that the time interval between repeated measures using a dietary instrument should be chosen to minimize changes in dietary intake and our use of 4 weeks fits that criterion.

    Conclusions

    In conclusion, the self-administered online Food4Me FFQ demonstrates good reproducibility for the estimation of energy, nutrient, and food group intakes and moderate agreement for the assessment of energy and nutrient intakes when compared with a 4-day WFR in an adult population. Consequently, the online Food4Me FFQ was considered suitable for the assessment of dietary intake in healthy adults.

    Acknowledgments

    This work was funded by Food4Me (KBBE.2010.2.3-02, Project no 265494). The authors would like to thank Kate Currie and Marzena Kucharska-Kelly for their assistance with the running of the study, Crème Global (Dublin, Ireland) for their support with the online Food4Me FFQ, and Breige McNulty for her assistance with analysis of the WFR.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Food4Me Food Frequency Questionnaire screenshots.

    PDF File (Adobe PDF File), 242KB
    1. Rayner M, Scarborough P. The burden of food related ill health in the UK. J Epidemiol Community Health 2005 Dec;59(12):1054-1057 [FREE Full text] [CrossRef] [Medline]
    2. Scarborough P, Bhatnagar P, Wickramasinghe KK, Allender S, Foster C, Rayner M. The economic burden of ill health due to diet, physical inactivity, smoking, alcohol and obesity in the UK: an update to 2006-07 NHS costs. J Public Health (Oxf) 2011 Dec;33(4):527-535 [FREE Full text] [CrossRef] [Medline]
    3. Bloom DE, Cafiero E, Jané-Llopis E, Abrahams-Gessel S, Bloom LR, Fathima S, et al. Program on the Global Demography of Aging. Geneva: World Economic Forum; 2011. The global economic burden of noncommunicable diseases   URL: http:/​/www.​hsph.harvard.edu/​program-on-the-global-demography-of-aging/​WorkingPapers/​2012/​PGDA_WP_87.​pdf [accessed 2014-02-25] [WebCite Cache]
    4. Miniwatts Marketing Group. World Internet Usage and Population Statistics. Colombia: Miniwatts Marketing Group 2012   URL: http://www.internetworldstats.com/stats.htm [accessed 2014-08-06] [WebCite Cache]
    5. Kolasa KM, Guy Miller M. New developments in nutrition education using computer technology. Journal of Nutrition Education 1996 Jan;28(1):7-14. [CrossRef]
    6. Sacks N, Cabral H, Kazis LE, Jarrett KM, Vetter D, Richmond R, et al. A web-based nutrition program reduces health care costs in employees with cardiac risk factors: before and after cost analysis. J Med Internet Res 2009;11(4):e43 [FREE Full text] [CrossRef] [Medline]
    7. Solomon M, Wagner SL, Goes J. Effects of a Web-based intervention for adults with chronic conditions on patient activation: online randomized controlled trial. J Med Internet Res 2012;14(1):e32 [FREE Full text] [CrossRef] [Medline]
    8. Kodama S, Saito K, Tanaka S, Horikawa C, Fujiwara K, Hirasawa R, et al. Effect of Web-based lifestyle modification on weight control: a meta-analysis. Int J Obes (Lond) 2012 May;36(5):675-685. [CrossRef] [Medline]
    9. Adamson AJ, Mathers JC. Effecting dietary change. Proc Nutr Soc 2004 Nov;63(4):537-547. [Medline]
    10. Coulston AM, Boushey CJ. Nutrition in the Prevention and Treatment of Disease. Amsterdam: Academic Press; 2008.
    11. Rebro SM, Patterson RE, Kristal AR, Cheney CL. The effect of keeping food records on eating patterns. J Am Diet Assoc 1998 Oct;98(10):1163-1165. [CrossRef] [Medline]
    12. Block G. A review of validations of dietary assessment methods. Am J Epidemiol 1982 Apr;115(4):492-505. [Medline]
    13. Livingstone MB, Prentice AM, Strain JJ, Coward WA, Black AE, Barker ME, et al. Accuracy of weighed dietary records in studies of diet and health. BMJ 1990 Mar 17;300(6726):708-712 [FREE Full text] [Medline]
    14. Matthys C, Pynaert I, De Keyzer W, De Henauw S. Validity and reproducibility of an adolescent web-based food frequency questionnaire. J Am Diet Assoc 2007 Apr;107(4):605-610. [CrossRef] [Medline]
    15. Willett W. Nutritional Epidemiology. New York: Oxford University Press; 1998.
    16. Food4Me. 2011.   URL: http://www.food4me.org [WebCite Cache]
    17. Walsh M, Kuhn S. Developments in personalised nutrition. Nutrition Bulletin 2012 Nov 08;37(4):380-383 [FREE Full text] [CrossRef]
    18. Forster H, Fallaize R, Gallagher C, O'Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: the Food4Me food frequency questionnaire. J Med Internet Res 2014;16(6):e150 [FREE Full text] [CrossRef] [Medline]
    19. Cade JE, Burley VJ, Warm DL, Thompson RL, Margetts BM. Food-frequency questionnaires: a review of their design, validation and utilisation. Nutr Res Rev 2004 Jun;17(1):5-22. [CrossRef] [Medline]
    20. Swierk M, Williams PG, Wilcox J, Russell KG, Meyer BJ. Validation of an Australian electronic food frequency questionnaire to measure polyunsaturated fatty acid intake. Nutrition 2011 Jun;27(6):641-646. [CrossRef] [Medline]
    21. Labonté MÈ, Cyr A, Baril-Gravel L, Royer MM, Lamarche B. Validity and reproducibility of a web-based, self-administered food frequency questionnaire. Eur J Clin Nutr 2012 Feb;66(2):166-173. [CrossRef] [Medline]
    22. Christensen SE, Möller E, Bonn SE, Ploner A, Wright A, Sjölander A, et al. Two new meal- and web-based interactive food frequency questionnaires: validation of energy and macronutrient intake. J Med Internet Res 2013;15(6):e109 [FREE Full text] [CrossRef] [Medline]
    23. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr 2002 Aug;5(4):567-587. [CrossRef] [Medline]
    24. Nelson M, Atkinson M, Meyer J. Food Portion Sizes: A Photographic Atlas. London: MAFF Publications; 1997.
    25. Henry CJ. Basal metabolic rate studies in humans: measurement and development of new equations. Public Health Nutr 2005 Oct;8(7A):1133-1152. [Medline]
    26. Goldberg GR, Black AE, Jebb SA, Cole TJ, Murgatroyd PR, Coward WA, et al. Critical evaluation of energy intake data using fundamental principles of energy physiology: 1. Derivation of cut-off limits to identify under-recording. Eur J Clin Nutr 1991 Dec;45(12):569-581. [Medline]
    27. Hébert JR, Peterson KE, Hurley TG, Stoddard AM, Cohen N, Field AE, et al. The effect of social desirability trait on self-reported dietary measures among multi-ethnic female health center employees. Ann Epidemiol 2001 Aug;11(6):417-427. [Medline]
    28. Tinuviel Software. 2003. Food composition databank   URL: http://www.tinuvielsoftware.com/wisp41.htm [accessed 2013-08-23] [WebCite Cache]
    29. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986 Feb 8;1(8476):307-310. [Medline]
    30. Engle A, Lynn LL, Koury K, Boyar AP. Reproducibility and comparability of a computerized, self-administered food frequency questionnaire. Nutr Cancer 1990;13(4):281-292. [CrossRef] [Medline]
    31. Beasley JM, Davis A, Riley WT. Evaluation of a web-based, pictorial diet history questionnaire. Public Health Nutr 2009 May;12(5):651-659 [FREE Full text] [CrossRef] [Medline]
    32. Goulet J, Nadeau G, Lapointe A, Lamarche B, Lemieux S. Validity and reproducibility of an interviewer-administered food frequency questionnaire for healthy French-Canadian men and women. Nutr J 2004 Sep 13;3:13 [FREE Full text] [CrossRef] [Medline]
    33. Boucher B, Cotterchio M, Kreiger N, Nadalin V, Block T, Block G. Validity and reliability of the Block98 food-frequency questionnaire in a sample of Canadian women. Public Health Nutr 2006 Feb;9(1):84-93. [Medline]
    34. Collins CE, Boggess MM, Watson JF, Guest M, Duncanson K, Pezdirc K, et al. Reproducibility and comparative validity of a food frequency questionnaire for Australian adults. Clin Nutr 2013 Oct 9. [CrossRef] [Medline]
    35. Rimm EB, Giovannucci EL, Stampfer MJ, Colditz GA, Litin LB, Willett WC. Reproducibility and validity of an expanded self-administered semiquantitative food frequency questionnaire among male health professionals. Am J Epidemiol 1992 May 15;135(10):1114-1126. [Medline]
    36. Feskanich D, Rimm EB, Giovannucci EL, Colditz GA, Stampfer MJ, Litin LB, et al. Reproducibility and validity of food intake measurements from a semiquantitative food frequency questionnaire. J Am Diet Assoc 1993 Jul;93(7):790-796. [Medline]
    37. Hu FB, Rimm E, Smith-Warner SA, Feskanich D, Stampfer MJ, Ascherio A, et al. Reproducibility and validity of dietary patterns assessed with a food-frequency questionnaire. Am J Clin Nutr 1999 Feb;69(2):243-249 [FREE Full text] [Medline]
    38. Chiu TH, Huang HY, Chen KJ, Wu YR, Chiu JP, Li YH, et al. Relative validity and reproducibility of a quantitative FFQ for assessing nutrient intakes of vegetarians in Taiwan. Public Health Nutr 2014 Jul;17(7):1459-1466. [CrossRef] [Medline]
    39. Wong SS, Boushey CJ, Novotny R, Gustafson DR. Evaluation of a computerized food frequency questionnaire to estimate calcium intake of Asian, Hispanic, and non-Hispanic white youth. J Am Diet Assoc 2008 Mar;108(3):539-543. [CrossRef] [Medline]
    40. Tsubono Y, Nishino Y, Fukao A, Hisamichi S, Tsugane S. Temporal change in the reproducibility of a self-administered food frequency questionnaire. Am J Epidemiol 1995 Dec 1;142(11):1231-1235. [Medline]
    41. Torheim LE, Ouattara F, Diarra MM, Thiam FD, Barikmo I, Hatløy A, et al. Nutrient adequacy and dietary diversity in rural Mali: association and determinants. Eur J Clin Nutr 2004 Apr;58(4):594-604. [CrossRef] [Medline]
    42. Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L, et al. Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic. Public Health Nutr 2003 May;6(3):313-321. [CrossRef] [Medline]
    43. Bohlscheid-Thomas S, Hoting I, Boeing H, Wahrendorf J. Reproducibility and relative validity of food group intake in a food frequency questionnaire developed for the German part of the EPIC project. European Prospective Investigation into Cancer and Nutrition. Int J Epidemiol 1997;26 Suppl 1:S59-S70 [FREE Full text] [Medline]
    44. Fidanza F, Gentile MG, Porrini M. A self-administered semiquantitative food-frequency questionnaire with optical reading and its concurrent validation. Eur J Epidemiol 1995 Apr;11(2):163-170. [Medline]
    45. Munger RG, Folsom AR, Kushi LH, Kaye SA, Sellers TA. Dietary assessment of older Iowa women with a food frequency questionnaire: nutrient intake, reproducibility, and comparison with 24-hour dietary recall interviews. Am J Epidemiol 1992 Jul 15;136(2):192-200. [Medline]
    46. Riley MD, Blizzard L. Comparative validity of a food frequency questionnaire for adults with IDDM. Diabetes Care 1995 Sep;18(9):1249-1254. [Medline]

    ‎
    BMI: body mass index
    EPIC: European Prospective Investigation of Cancer
    FFQ: food frequency questionnaire
    MUFA: monounsaturated fatty acids
    PUFA: polyunsaturated fatty acids
    RE: retinol equivalents
    SFA: saturated fatty acids
    TE: total energy
    WFR: weighed food record

    Edited by G Eysenbach; submitted 27.02.14; peer-reviewed by A Lennox, S Fulton; comments to author 07.07.14; accepted 22.07.14; published 11.08.14

    Copyright

    ©Rosalind Fallaize, Hannah Forster, Anna L Macready, Marianne C Walsh, John C Mathers, Lorraine Brennan, Eileen R Gibney, Michael J Gibney, Julie A Lovegrove. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 11.08.2014.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.