This paper is in the following e-collection/theme issue:

Research Instruments, Questionnaires, and Tools (541) Electronic/Mobile Data Capture, Internet-based Survey & Research Methodology (788) Instruments and Questionnaires for Nutrition and Food Intake (159)

Published on in Vol 20, No 7 (2018): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/9113, first published .
Relative Validity and Reproducibility of a New 44-Item Diet and Food Frequency Questionnaire Among Adults: Online Assessment

Relative Validity and Reproducibility of a New 44-Item Diet and Food Frequency Questionnaire Among Adults: Online Assessment

Relative Validity and Reproducibility of a New 44-Item Diet and Food Frequency Questionnaire Among Adults: Online Assessment

Authors of this article:

Aurélie Affret1, 2, 3 Author Orcid Image ;   Douae El Fatouhi1, 2, 3 Author Orcid Image ;   Courtney Dow1, 2, 3 Author Orcid Image ;   Emmanuelle Correia1, 2, 3 Author Orcid Image ;   Marie-Christine Boutron-Ruault1, 2, 3 Author Orcid Image ;   Guy Fagherazzi1, 2, 3 Author Orcid Image
Article Authors Cited by (16) Tweetations (15) Metrics

    Original Paper

    1Inserm U1018, Center for Research in Epidemiology and Population Health, Villejuif, France

    2Université Paris-Sud, Orsay, France

    3Université de Versailles-Saint-Quentin-en-Yvelines, Versailles, France

    Corresponding Author:

    Marie-Christine Boutron-Ruault, PhD

    Inserm U1018

    Center for Research in Epidemiology and Population Health

    114 Rue Edouard Vaillant

    Villejuif, 94805

    France

    Phone: 33 33 142116466

    Fax:33 33 142114000

    Email: marie-christine.boutron@gustaveroussy.fr


    Background: Dietary questionnaires currently available which can assess the habitual diet are timely, costly, or not adapted well to the modern diet; thus, there is a need for a shorter food frequency e-Questionnaire (FFeQ) adapted to Western diets, in order to properly estimate energy and macronutrient intakes or rank individuals according to food and nutrient intakes.

    Objective: The aim of this study was to evaluate the relative validity and reproducibility of a 30-minute and 44-item FFeQ in a sample of adults obtained from the general population.

    Methods: A sample of French adults was recruited through social media and an advertising campaign. A total of 223 volunteers completed the FFeQ twice at one-year intervals and were included in the reproducibility study. During that interval, 92 participants completed three-to-six 24-hour recalls and were included in the validity study. Nutrient and dietary intakes were computed for all validity and reproducibility participants. The level of agreement between the two methods was evaluated for nutrient and food group intakes using classification into quintiles of daily intake, correlation coefficients and Bland-Altman plots.

    Results: For relative validity, correlation coefficients ranged from 0.09 to 0.88 (unadjusted correlation coefficients, median: 0.48) and 0.02 to 0.68 (deattenuated and energy adjusted correlation coefficients, median: 0.50) for food group and nutrient intakes, respectively. The median proportion of subjects classified into the same or adjacent quintile was 73% and 66% for food and nutrient intakes, respectively. Bland-Altman plots showed good agreement across the range of intakes. Regarding reproducibility, intraclass correlation coefficients ranged from 0.33 to 0.72 (median: 0.60) and 0.55 to 0.73 (median: 0.64), for food and nutrient intakes, respectively.

    Conclusions: The FFeQ showed acceptable validity and reproducibility in a sample of adults based on their food and nutrient intakes. The FFeQ is a promising and low-cost tool that can be used in large-scale online epidemiological studies or clinical routines and could be integrated into evidence-based smartphone apps for assessing diet components.

    J Med Internet Res 2018;20(7):e227

    doi:10.2196/jmir.9113

    Keywords

    Short Food Frequency e-Questionnaire; Web-based; validity; reproducibility; online dietary assessment tool 


    A healthy lifestyle, characterized by an adequate, balanced diet combined with regular physical activity, is one of the determinants for good health [1]. Moderate to strong associations between healthy dietary patterns and decreased risk of obesity and chronic diseases like cardiovascular disease, hypertension, type 2 diabetes, and some cancers are strongly highlighted in the literature [2,3].

    Among the many available tools that evaluate individual dietary intakes, the food frequency questionnaire (FFQ) has been frequently used in nutritional epidemiology studies since the 1980s [4-6]. Despite limitations (difficulties to estimate habitual intakes, memory bias, errors in perception of portion sizes, and use of restricted food lists), FFQs collect valuable information allowing researchers to assess the typical diet at a low cost and logistic burden, and it can be self-administered [5-7]. FFQs also allow researchers to accurately rank subjects according to their dietary and nutritional intake, which is important when comparing risks in various subgroups [6].

    As previously described [8], a consortium of six cohort studies (E3N [9], E4N [10], CKD-REIN [11], i-Share [12], Elfe [13] and Psy-COH) was established to create a unique Food Frequency Questionnaire (FFQ) that could quickly assess the habitual diet in several populations: adults, the elderly, adolescents, students, patients with mental disorders, and patients with chronic kidney disease. It was decided that the questionnaire would be limited to 50 items to quickly assess the diet.

    Existing FFQs of 50 items or less have not been proven suitable to quickly and accurately assess the diet in several French population subgroups: i) the FFQ by Vercambre et al [14] was developed for senior women and focused on specific nutrients for that population, ii) one third of the items of the FFQ developed by Giovannelli et al [15] were not valid and a food composition table was not available to study nutritional intakes, iii) the FFQ developed by Barrat et al [16] referred to intakes during the preceding week, and did not consider seasonal variability, and iv) two FFQs were developed solely in the context of cardiovascular prevention [17,18]. Therefore, the consortium decided to develop a new and unique FFQ (40 items) adapted to several population subgroups of interest. They agreed that some additional, specific questions (10 items maximum, as performed in the present study) could be added to the questionnaire to help estimate specific nutrients of interest or to obtain qualitative information about the dietary context, consistent with the targeted populations. In a pilot study, a paper version of the consortium FFQ was validated in a sample of patients with chronic kidney disease [8] and showed acceptable validity and reproducibility. Then, a web version of the questionnaire was developed by the consortium to meet the need of a food frequency e-Questionnaire (FFeQ) adapted to the diet of those in Western countries, able to accurately estimate energy and macronutrient intakes, and to rank individuals according to food and nutrient intakes.

    Web-based dietary assessments provide a lot of advantages [19,20]: they have the potential to save time and financial resources, may be preferred by participants, and response quality can be improved directly by including cutoff values and alert messages in case of inconsistencies, abnormal or missing data. Several examples of 24-hour recall and Web-based FFQs already exist in the literature [21-26]. Because 24-hour recalls need to be repeated to assess the overall diet, Web-based FFQs might be more feasible in large-scale studies. However, most of the existing Web-based FFQs are long questionnaires and therefore time-consuming [21,24,25].

    Before using a newly developed or modified FFQ, it must first be validated to be considered an acceptable method of dietary assessment [27]. The aim of the present study is to study the reproducibility of the online version of the newly developed FFQ (FFeQ) and evaluate its relative validity against 24-hour recalls in a sample of French adults.


    Study Population and Design

    According to Willett [6], the number of subjects necessary to conduct reproducibility and validity studies is approximately 110. Between January and February 2016, a national invitation to participate in the present reproducibility and validity study was advertised in the Inserm (French National Institute of Health and Medical Research) network and through the Inserm’s Twitter and Facebook accounts. In total, 441 adults volunteered (from which 214 participants agreed to participate in the validity study), provided their informed consents, and were invited to complete the FFeQ. They were asked to complete the FFeQ twice, at a one-year interval, in February-April 2016 and again in February-April 2017. In total, 319 participants completed the FFeQ once (the 122 participants who failed to complete the first questionnaire were more likely to be women and live in the South and East of France and overseas). Two hundred twenty-nine participants completed it twice (participants who completed both questionnaires were more likely to be women and live in the South of France). Participants who under- or over- reported energy intake in one of the FFeQs, ie, were in the top and bottom 1% of the energy intake-to-energy requirement ratio distribution, were excluded as previously described [28]. Energy requirement was calculated as follows: Basal Metabolic Rate (BMR)* Physical Activity Level (the cutoff value of 1.55 for a minimal activity level was chosen [29]). BMR was computed based on sex, age, height, and weight, using the Schofield formula [30]. After exclusion, a total of 223 participants were included in the reproducibility study. Among them, 92 patients answered at least three (out of six) 24-hour recalls and were therefore included in the relative validity study (a flow diagram is presented in Multimedia Appendix 1). We decided to include participants with at least three recalls in the validity study to ensure i) reasonable intra-individual variations, ii) seasonal representation and iii) sufficient statistical power. Participants who completed at least three 24-hour recalls were more likely to be older than participants who completed the first questionnaire but did not complete three 24-hour recalls. They had higher energy intake and healthier dietary habits than their counterparts (data not tabulated).

    From the 214 participants who volunteered for the validity study, only 130 were interviewed for the 24-hour recalls. The main reasons were contact difficulties as they were mostly active people, and that this study was conducted by only one dietitian which prevented us from interviewing several participants simultaneously.

    Food Frequency Questionnaire

    The food list for the FFeQ was developed by the investigators based on existing national food questionnaires [14-16,31-34] and data from the second national study of individual food intakes of French adults [35] to ensure that all food groups contributing to at least 5% of the mean energy, macronutrient, vitamin, or mineral intake of the French population were represented in the FFeQ, [16]. The FFeQ was self-administered online. The questionnaire asked participants to report their usual dietary intake over the past year. In epidemiological studies, one-year memory FFQ are mainly used because they assess long-term diets (diets tend to remain stable year on year) and season variability of intakes is considered [6].

    The questionnaire was divided into two parts: The first part comprised 40 food groups. It quantified consumption by frequency (never or less than once a month, x times a day, x times a week or x times a month) and portion sizes per food group item. Photos previously validated [36] were directly integrated into the questionnaire to help participants estimate the consumed quantities of 21 food items (see Multimedia Appendices 2 and 3). Most of the time, there were three photos showing increasing portion sizes with five possible answers (less than the lowest portion, the lowest portion, an intermediate portion, the biggest portion, more than the biggest portion). For items not having a photo, participants were asked to quantify their consumption based on a standard portion size (typical household measurements like measuring spoons or standard units such as individual containers of yogurt).

    The second part was specific to the study population. It was composed of 10 questions. Six were qualitative questions about eating habits (eg, meal frequency, socialization during meals, source of food supplies), and four questions were used to obtain nutritional data, of which two provided more detailed information about some food groups from the first part of the questionnaire (ie, fish and soft drinks).

    In total, 44 items were used to obtain the nutritional data (see Multimedia Appendix 4). Daily intakes for each food group item were computed: frequencies were converted into numbers of servings per day and multiplied by the portion size. An ad hoc composition table was developed using data from the INCA2 French representative population survey (35) to estimate the percentage of contribution of each food included in a food group item. Nutritional data were then obtained using the French food composition database established by the French Data Centre on Food Quality (Ciqual, last updated in 2013) [37]. Besides nutritional and diet context information, information on sex, birth date, and anthropometric data was elicited. It also questioned participants about potential changes in their food habits during the past year due to specific situations (diet, pregnancy, a move, surgery or depression).

    The FFeQ was adapted for laptops, tablets and smartphones. Compared to the paper version of the questionnaire [8], the online version presented here had a higher data quality thanks to alerts, restricted answers and automated checks. The design of the FFeQ consisted of one web page per food item and a pilot study previously demonstrated an average time of 30 minutes to complete the questionnaire.

    24-Hour Recalls

    The reference method used to compare results from the FFeQ consisted of six 24-hour recalls carried out every two months during the year between the first and the second FFeQ. Study participants were asked to recall all foods and beverages consumed on the previous day (due to logistics, data for Saturdays were collected on Mondays). Participants were not informed in advance of the day of the recall. To account for intra-individual variation (because dietary habits may differ according to weekdays or weekends and seasons), all days and all seasons were covered by the recalls as recommended [6] ie, days for recalls were randomly selected every two months per participant. Phone interviews were carried out by a trained dietitian who entered the data into the Nutrilog Software (v3.10b). These data were instantly converted into nutrient intakes by the software using the Ciqual food composition database [37]. A validated photo album of 42 foods [34] was previously e-mailed to the participants to help them quantify the amount of food consumed during the phone interview.

    Statistical Analysis

    We computed descriptive statistics (median and interquartile range) for nutrients and foods for both FFeQs and the average of the 24-hour recalls. Wilcoxon signed rank tests were performed to assess whether the mean ranks differed between groups.

    Relative Validity

    To study relative validity, data evaluated by the second FFeQ (FFeQ2) were compared with the mean of the 24-hour recalls, since both methods covered the same period. A list of concordance was established between food group items from the FFeQ and food items provided by 24-hour recalls. Few rarely consumed foods declared during the 24-hour recalls were not covered by the FFeQ items and were not taken into consideration.

    Unadjusted Spearman correlation coefficients were calculated for food groups. Unadjusted and energy-adjusted Pearson correlation coefficients were calculated for nutrient intakes. Energy-adjusted coefficients, corrected for attenuation for within-person variation in the reference method (deattenuated coefficients) [6,38], were produced. Energy adjustment was performed using the residual method [6]. To improve the normal distribution, nutrient intakes were logarithmically transformed before analysis.

    In terms of food group and nutrient intakes, we examined the level of agreement in ranking subjects between the two methods through cross-classification into quintiles. The percentage of participants classified in the lowest quintile in the FFeQ and the highest quintile in the 24-hour recalls (and vice versa) was studied. Because several food groups had a proportion of non-consumers >20%, we established three categories as follows: class=1 for null consumption; class=2 for consumption below or equal to the median value in consumers; class=3 for consumption over the median value in consumers. For food groups having a proportion of non-consumers <20%, subjects were classified into tertiles of consumption.

    We evaluated agreement between the FFeQ and the 24-hour recalls performing Bland-Altman plots on energy-adjusted values [39-41]. Mean differences between the two assessment methods were plotted against the average estimation of the two methods. The 95% limit of agreement was calculated as the mean difference (SD 1.96).

    Reproducibility

    To evaluate reproducibility, data obtained from the first and second FFeQs (FFeQ1 and FFeQ2) were compared. For food groups, unadjusted Spearman correlation coefficients and intraclass correlation coefficients (ICC) were estimated. Unadjusted and energy-adjusted Pearson correlation coefficients as well as ICC were calculated for nutrient intakes. Nutrient intakes were logarithmically transformed before analysis, to improve the normal distribution. The level of agreement in ranking subjects between the two FFeQs (in terms of food group and nutrient intakes) was examined through cross-classification into quintiles. All statistical analyses were performed on SAS 9.4 (SAS Institute Inc, Cary, NC, USA). A P value <.05 was considered statistically significant.


    Baseline characteristics of the participants included in the relative validity and reproducibility studies are presented in Table 1. Participants included in both studies were mostly women (63.0%, 58 out of 92 participants and 74.9%, 67 out of 223 participants in the validity and reproducibility studies, respectively). Relative validity study participants were older than reproducibility study participants (47.7 years old, SD 14.9 vs 40.5 years old, SD 14.9) but the mean BMI was similar in both studies (23.5 kg/m2, SD 4.2 in the reproducibility study). Participants lived in all regions of France, with higher proportions living in Paris and suburbs, and in the South. Most of the participants included in the relative validity study had complete data for six 24-hour recalls (73.9%, 68 out of 92 participants).

    Relative Validity

    Dietary intakes estimated by the FFeQ2 and the mean of the 24-hour recalls are presented in Table 2. Some food items such as “whole-grain pasta, rice, and wheat,” “legumes,” “milk” or “fruit” tended to be overestimated with the FFeQ whereas other food groups such as “raw vegetables,” “pizza, lasagna, and quiche,” “sausages and processed meat,” “cheese,” “sweet snacks, chocolate, and Danish pastries” or “alcoholic beverages excluding wine” were underestimated with the FFeQ2.

    Unadjusted Spearman coefficients ranged from 0.09 (variety meats) to 0.88 (tea and herb teas), the median value being 0.48. Eight food groups had correlation coefficients below 0.3.

    Table 1. Descriptive characteristics of the subjects included in the relative validity and reproducibility study.
     CharacteristicValidity (n=92)Reproducibility (n=223)
    Sex (women), n (%)58 (63.0)167 (74.9)
    Age (years), mean (SD)47.7 (14.9)40.5 (14.9)
    Body mass index (kg/m2), mean (SD)23.6 (3.3)23.5 (4.2)
    Area of residence, n (%)


    		South15 (16.3)60 (27.0)

    		West10 (10.9)19 (8.6)

    		North1 (1.1)8 (3.6)

    		East7 (7.6)13 (5.9)

    		Center6 (6.5)13 (5.9)

    		Overseas departments0 (0)3 (1.4)

    		Paris and suburbs53 (57.6)106 (47.7)
    Number of 24-hour recall days, n (%)


    		35 (5.4)N/Aa

    		44 (4.3)N/A

    		515 (16.3)N/A

    		668 (73.9)N/A
    Distribution of 24-hour recall days (average %)


    		Weekday67 (73.3)N/A

    		Weekend25 (26.7)N/A

    		Autumn and winter49 (53.0)N/A

    		Spring and summer43 (47.0)N/A

    aN/A: not applicable.

    Table 2. Relative validity of the Food Frequency Questionnaire (FFeQ) for food groups (n=92). For food groups with a proportion of non-consumers >20%, tertiles and quintiles classifications were not performed. Instead, participants were classified as follows: class=1 for null consumption; class=2 for consumption below or equal to the median value in consumers, class=3 for consumption above the median value in consumers.
    Food groupsDaily intakesaFFeQ2 vs mean three to six 24-hour recalls

    		24-hour recalls, median (IQR)FFeQ2, median (IQR)FFeQ2-24-hour recalls, mean difference (SD)Unadjusted Spearman correlation coefficientsCross-classification of food group distribution, %

    		Subjects classified in same tertileSubjects classified in same or adjacent quintileSubjects classified in opposite quintiles
    Whole-grain bread and substitutesb18.7 (44.8)10.7 (40.0)-1.2 (54.1)0.3553—c—
    White bread and substitutes50.9 (57.7)32.0 (69.3)-2.2 (47.3)0.6354781
    Breakfast cerealsb0.0 (0.0)0.0 (0.5)1.9 (9.2)0.6879——
    Whole-grain pasta, rice and wheatb,d0.0 (0.0)2.3 (32.0)24.2d (46.4)0.14e49——
    White pasta, rice and wheat68.3 (82.5)45.3 (90.7)-11.3 (68.2)0.5249711
    Legumesb0.0 (16.7)16.7 (20.0)8.8d (30.8)0.2627——
    French fries and other fried tubersb0.0 (15.0)8.7 (15.3)2.3 (23.8)0.3540——
    Potatoes and other tubers (not fried)b50.0 (64.6)40.0 (53.3)-4.8 (66.9)0.2645——
    Cooked vegetables179.6 (163.0)176.7 (166.7)-8.8 (114.7)0.4950681
    Raw vegetables63.1 (66.7)34.4 (73.7)-22.1d (51.7)0.5358741
    Pizza, lasagna and quicheb27.9 (58.6)14.2 (15.0)-23.2d (40.3)0.4448——
    Sandwich, burgers and kebabb0.0 (23.3)0.0 (12.0)-2.6 (18.2)0.5261——
    Fish fingers/breaded meatb0.0 (0.0)0.0 (3.3)-4.1d (14.2)0.2764——
    Sausages and other processed meatb30.0 (30.3)11.3 (28.0)-14.9d (29.5)0.3854——
    Poultry/rabbitb24.2 (47.5)20.0 (30.0)4.1 (52.8)0.4541——
    Meat43.3 (37.8)26.3 (34.7)-2.3 (44.4)0.3938611
    Variety meatsb0.0 (0.0)0.0 (1.3)1.7d (10.0)0.09e74——
    Eggsb8.3 (20.0)14.1 (21.2)6.5d (26.8)0.3535——
    Fishb15.8 (38.8)13.3 (20.0)-3.0 (26.2)0.3441——
    Seafood (excluding fish)b0.0 (2.5)0.0 (6.7)-3.2 (19.1)0.3359——
    Milkb0.0 (63.3)0.0 (180.0)70.6d (175.2)0.7368——
    Yogurt, white cheese, cottage cheese96.3 (109.8)125.0 (123.3)28.0d (70.6)0.7560880
    Cream dessertb0.0 (20.8)2.0 (16.7)5.1 (36.3)0.4053——
    Cheese31.0 (30.6)28.0 (18.0)-7.1d (25.2)0.5152681
    Butter, fresh cream4.2 (6.7)10.0 (20.0)10.1d (20.2)0.6651790
    Margarine, mayonnaiseb0.0 (0.4)0.0 (0.8)1.0d (4.4)0.6076——
    Olive oil3.8 (6.3)10.0 (14.7)6.9d (10.0)0.2849622
    Rapeseed oil, walnut oil, mixed oilb0.3 (2.0)0.7 (2.7)0.9d (4.5)0.3547——
    Sunflower oil, groundnut oilb0.0 (0.0)0.0 (0.7)0.5 (2.8)0.2165——
    Salty snacksb0.0 (8.3)4.0 (8.0)-0.1 (7.5)0.4143——
    Sweet snacks, chocolate, and Danish pastriesb46.7 (50.4)13.3 (40.0)-26.6d (61.0)0.2639——
    Fruit202.8 (220.7)351.0 (260.0)122.3d (274.7)0.6761771
    Water992.8 (746.7)1550.0 (1200.0)1134.5d (2410.6)0.5252731
    Coffeeb214.2 (303.3)200.0 (380.0)74.7 (303.4)0.8172——
    Tea and herb teasb135.0 (557.1)90.0 (400.0)-3.1 (310.4)0.8871——
    Fruit juiceb50.0 (143.3)50.0 (206.7)29.8d (132.6)0.6158——
    Sweet beveragesb0.0 (2.1)0.0 (0.0)-11.5d (50.1)0.7379——
    Artificially-sweetened beveragesb0.0 (0.0)0.0 (0.0)10.8 (66.7)0.5588——
    Wineb25.0 (90.0)11.0 (40.5)-16.4d (68.7)0.8167——
    Alcoholic beverages excluding wineb13.3 (52.5)2.5 (20.0)-24.3d (65.5)0.4854——

    aMeasured in grams (food) or milliliters (beverages).

    bThese food groups have a large proportion of non-consumers (>20%).

    cDashes indicate food groups that have a large proportion of nonconsumers (>20%). Classification into quintiles of consumption was not performed.

    dThe mean rank of the values of the three to six 24-hour recalls was significantly different to the mean rank of the values of the SFFeQ, according to Wilcoxon signed rank tests.

    eUnadjusted Spearman correlation coefficients for which the statistical tests did not provide P values <.05.

    The median proportion of participants classified in the same or adjacent quintiles of food group consumption by the FFeQ2, as well as by the mean of the 24-hour recalls, was 73%. The median proportion of participants classified in opposite quintiles was 1%. The median proportion of participants classified in the same tertile was 54%.

    Mean macronutrient intakes estimated using the FFeQ2 did not differ in our study from those estimated in the 24-hour recalls (Table 3). Calcium and retinol intakes tended to be overestimated by the FFeQ (median 1113.7 mg/d, IQR 625.1 vs median 853.4 mg/d, IQR 322.5 and median 383.3 mg/d, IQR 393.4 vs median 0.0 μg/d, IQR 0.0, respectively) whereas alcohol and sodium intakes were underestimated by the FFeQ (median 2.0 g/d, IQR 6.9 vs median 4.7 mg/d, IQR 14.0 and median 2376.7 mg/d, IQR 945.7 vs median 2463.5 μg/d, IQR 1072.0, respectively).

    Unadjusted correlation coefficients ranged from 0.08 (manganese and copper) to 0.77 (alcohol), with a 0.41 median value. Deattenuation mainly improved energy-adjusted correlation coefficients. Deattenuated energy-adjusted CC ranged from 0.05 (manganese) to 0.68 (potassium, carotene and vitamin C), with a 0.50 median value. A total of eight nutrients (sodium, magnesium, manganese, iron, copper, zinc, iodine and vitamin B12) had correlation coefficients lower than 0.3.

    The median of percentages of participants classified in the same or adjacent quintiles of nutrient intakes by FFeQ2, and by the mean of the 24-hour recalls, was 66%. The median proportion of participants classified in opposite quintiles was 3%.

    The Bland-Altman plot analysis graphs displayed good agreement between the two methods of estimation across the range of intake for energy (Figure 1), protein (Figure 2), carbohydrates (Figure 3), lipids (Figure 4), alcohol (Figure 5), cholesterol (Figure 6), sodium (Figure 7), and calcium intakes (Figure 8). For all 33 studied nutrients, the mean difference between methods (FFeQ2 vs means of 24-hour recalls) was close to zero for all levels of intake, except for calcium (data not shown). Across the range of intakes, calcium was systematically overestimated by the FFeQ2 which was consistent with the results displayed in Table 3. The percentage of points that were outside the limits of agreement ranged from 1.1% (zinc and iodine) to 7.6% (sugars, and vitamins D, B1, and B6), with a median value of 4.3%, which, theoretically is the percentage of values outside the mean (SD 1.96). Finally, the agreement did not differ between subjects with high and low intakes.

    Table 3. Relative validity of the FFeQ for nutrients (n=92). Means and cross-classification were computed on crudes variables. All variables were log transformed before computing Pearson correlation coefficients to improve normality.
    NutrientsDaily intakesFFeQ2 vs mean of three to six 24-hour recalls

    		24-hour recalls, median (IQR)FFeQ2, median (IQR)FFeQ2-24-hour recalls, mean (SD)Pearson correlation coefficientsaCross-classification of nutrient distribution, %

    		UnadjustedEnergy-adjustedbDeattenuatedcSubjects classified in same or adjacent quintileSubjects classified in opposite quintiles
    Energy (kcal)1882.2 (659.6)1859.5 (819.8)-77.4 (565.3)0.47N/Ad0.50e663
    Protein (g)76.2 (22.6)72.2 (34.1)-3.5 (23.6)0.570.470.52722
    Carbohydrates (g)203.7 (74.6)202.3 (98.0)-1.3 (72.6)0.440.490.54713
    Fat (g)74.2 (28.4)67.7 (34.8)-3.6 (27.2)0.470.550.61593
    SFAf (g)26.9 (13.9)25.8 (14.2)-2.4g (10.5)0.610.540.61671
    MUFAh (g)24.2 (10.4)26.3 (12.0)3.1g (12.2)0.350.460.53587
    PUFAi (g)8.6 (3.7)7.8 (3.6)-0.1 (4.5)0.320.480.55584
    Cholesterol (mg)244.2 (133.6)243.3 (130.2)-11.4 (149.9)0.460.310.39655
    Sugars (g)79.8 (34.3)82.5 (46.2)3.6 (42.9)0.280.390.45718
    Fiber (g)21.6 (9.0)20.9 (9.3)-0.7 (7.9)0.400.520.60643
    Alcohol (g)4.7 (14.0)2.0 (6.9)-3.5g (8.5)0.77j—k—k881
    Water (g)2736.5 (868.5)3182.4 (1744.0)1310.8g (2654.0)0.470.470.51681
    Sodium (mg)2463.5 (1072.0)2376.7 (945.7)-402.5g (1483.9)0.370.050.07641
    Magnesium (mg)319.1 (104.6)344.0 (194.9)52.0g (142.4)0.260.250.29628
    Phosphorus (mg)1139.6 (334.3)1114.7 (491.7)1.6g (352.8)0.500.400.45672
    Potassium (mg)3099.1 (1044.0)3091.6 (1555.0)133.1 (955.3)0.490.590.68712
    Calcium (mg)853.4 (322.5)1113.7 (625.1)442.2g (553.1)0.420.380.44705
    Manganese (mg)2.8 (1.5)11.5 (6.5)9.3g (4.9)0.080.050.05598
    Iron (mg)9.4 (4.3)10.6 (5.0)1.1g (4.5)0.280.230.27623
    Copper (mg)1.4 (0.6)2.2 (1.5)0.8g (1.6)0.080.060.07577
    Zinc (mg)8.6 (2.9)9.1 (4.7)0.4 (5.0)0.400.220.26611
    Iodine (μg)120.2 (68.5)122.1 (57.2)-3.6 (69.1)0.430.240.29682
    Retinol (μg)0.0 (0.0)383.3 (393.4)506.6g (479.8)0.11j—k—k4118
    Carotene (μg)3298.9 (2559.0)3165.6 (2944.0)-306.4 (2833.2)0.520.540.68773
    Vitamin D (μg)2.2 (1.5)2.1 (1.2)-0.4 (1.9)0.320.230.31613
    Vitamin E (mg)9.0 (4.3)10.2 (5.1)2.1g (5.4)0.250.430.53584
    Vitamin C (mg)118.6 (69.7)124.7 (90.3)13.3 (68.5)0.540.560.68782
    Vitamin B1 (mg)1.1 (0.4)1.0 (0.4)-0.1g (0.4)0.310.330.41664
    Vitamin B2 (mg)1.6 (0.6)1.5 (0.8)0.0 (0.6)0.450.480.53681
    Vitamin B3 (mg)16.5 (6.8)15.6 (8.9)0.0 (6.7)0.490.490.58734
    Vitamin B5 (mg)4.8 (2.0)4.8 (2.4)0.2 (1.8)0.470.570.64701
    Vitamin B6 (mg)1.7 (0.7)1.6 (0.6)-0.1 (0.7)0.280.380.46582
    Vitamin B9 (μg)305.1 (115.1)321.0 (141.0)26.9 (126.0)0.360.490.58641
    Vitamin B12 (μg)3.3 (2.3)5.6 (3.8)2.1g (6.0)0.310.210.26657

    aThe statistical tests provided P values <.05 for each Pearson correlation coefficient, except for those in italics.

    bEnergy adjustment according to the residual method.

    cEnergy-adjusted and deattenuated Pearson correlation coefficient (corrected for within-person variation in the three to six 24-hour recalls).

    dN/A: not applicable

    eUnadjusted and de-attenuated Pearson correlation coefficient.

    fSFA: saturated fatty acids.

    gThe mean rank of the values of the three to six 24-hr recalls was significantly different to the mean rank of the values of the SFFeQ, according to Wilcoxon signed rank tests. The statistical tests provided P values <.05 for each Pearson correlation coefficient, except for those in italics.

    hMUFA: monounsaturated fatty acids.

    iPUFA: polyunsaturated fatty acids.

    jSpearman correlation coefficients were performed because normality was not respected.

    kNormality was not respected. Energy-adjusted and deattenuated coefficients were not performed.

    ‎
    Figure 1. Bland-Altman plots related to energy. Difference in the daily intake of energy (crude variable) derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean, SD 1.96; n=92).
    View this figure
    ‎
    Figure 2. Bland-Altman plots related to protein. Difference in the daily intake of energy-adjusted protein derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean energy-adjusted daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean ± 1.96 SD) (n=92).
    View this figure
    ‎
    Figure 3. Bland-Altman plots related to carbohydrate. Difference in the daily intake of energy-adjusted carbohydrate derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean energy-adjusted daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean, SD 1.96; n=92).
    View this figure
    ‎
    Figure 4. Bland-Altman plots related to lipid. Difference in the daily intake of energy-adjusted lipid derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean energy-adjusted daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean, SD 1.96; n=92).
    View this figure
    ‎
    Figure 5. Bland-Altman plots related to alcohol. Difference in the daily intake of alcohol (crude variable) derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean, SD 1.96; n=92).
    View this figure
    ‎
    Figure 6. Bland-Altman plots related to cholesterol. Difference in the daily intake of energy-adjusted cholesterol derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean energy-adjusted daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean, SD 1.96; n=92).
    View this figure
    ‎
    Figure 7. Bland-Altman plots related to sodium. Difference in the daily intake of energy-adjusted sodium derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean energy-adjusted daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean, SD 1.96; n=92).
    View this figure
    ‎
    Figure 8. Bland-Altman plots related to calcium. Difference in the daily intake of energy-adjusted calcium derived from the three to six 24-hour recalls and the short food frequency e-questionnaire (SFFeQ2) plotted against the corresponding mean energy-adjusted daily intakes derived from the two methods. Solid lines represent mean difference, and dashed lines show lower and upper 95% limits of agreement (mean, SD 1.96; n=92).
    View this figure

    Reproducibility

    Absolute daily intakes of food groups were mostly comparable between the two FFeQs (see Multimedia Appendix 5). A rather large statistically significant decrease was observed for the “sweet beverages” and “artificially-sweetened beverages” between FFeQ1 and FFeQ2.

    Unadjusted Spearman correlation coefficients ranged from 0.34 (sunflower and groundnut oils) to 0.90 (wine), with the median value being 0.65. Intraclass correlation coefficients ranged from 0.33 (sweet snacks, chocolate, and Danish pastries) to 0.72 (poultry or rabbit, fish, and fruit), with the median value being 0.60.

    The median of percentages of subjects classified in the same or adjacent quintiles of food group consumption by both FFeQs was 80%. The median proportion of participants classified in opposite quintiles was 1%. The median percentage of subjects classified in the same tertile was 64%.

    Absolute daily intake of energy and nutrients were comparable between the two FFeQs, although all nutrient intakes (excluding alcohol) showed a slight but statistically significant decrease between FFeQ1 and FFeQ2 (see Multimedia Appendix 6).

    Crude correlation coefficients ranged from 0.58 (iron) to 0.89 (alcohol), with a 0.65 median value. Energy-adjusted Pearson correlation coefficients ranged from 0.54 (vitamin B1) to 0.77 (vitamin E), with a 0.65 median value. Intraclass correlation coefficients ranged from 0.55 (carbohydrates) to 0.73 (magnesium and manganese), the median value being 0.65.

    The median proportion of subjects classified in the same or adjacent quintiles of nutrient intakes by both FFeQs was 79%. The median proportion of participants classified in opposite quintiles was 1%.


    Principal Findings

    The present study investigated the relative validity and reproducibility of a new FFeQ, in a sample of French adults. The FFeQ was designed to accurately estimate energy intake and rank participants according to their dietary and nutrient intakes. The overall results indicate acceptable relative validity (for nutrient intakes, median correlation coefficient=0.50 and median proportion of subjects classified in the same or adjacent quintiles by the FFeQ2 and the 24-hour recalls=66%), and good reproducibility (for nutrient intakes, median correlation coefficient=0.65, and median proportion of subjects classified in the same or adjacent quintiles=79%). Our tool demonstrated an acceptable ability to rank participants for most nutrients and food groups, making it sufficiently informative when studying associations with health outcomes and when adjusting for nutritional intake in epidemiological and clinical studies [6,42]. It can also be used to derive dietary patterns using collected food data.

    Because combinations of different assessment methods are becoming increasingly popular and can address several methodological limitations [43], it would be interesting, in further analyses to study under which circumstances a combination of the FFeQ with dietary recalls would be more efficient than either the FFeQ or the dietary recalls alone to address precision, power, and sample size, as it has been previously done [44].

    Our FFeQ was composed of 50 items, of which 44 were used to obtain nutritional data. In their review, Cade et al reported that the number of items in FFQs published between 1980 and 1999 ranged from three to 350 items [45], with the median being 79 items. However, according to Willett [6], there is a rapidly decreasing marginal gain in information obtained with increasingly detailed questionnaires. Considering the relative validity and reproducibility of our FFeQ, it appears that the chosen 44 items were sufficient to assess the overall diet and to describe major food and nutrient intakes. The remaining six items of the online FFeQ will help provide meaningful insights as they could be used for qualitative studies or to stratify statistical analyses according to eating habits like meal frequency, socialization during meals or source of food supplies.

    Relative Validity

    Recovery biomarkers are considered gold-standard measures to validate self-reported intakes. However, because of their costliness and because only a few recovery biomarkers are currently known, they are rarely used in validity studies [43]. When selecting a reference method to validate a tool, errors of both methods must be as independent as possible [46]. Even if correlated errors related to memory, perception of serving sizes and social desirability exist between FFQs and 24-hour recalls, multiple 24-hour recalls have often been considered as the best feasible reference method [47]. Here, to study the FFeQ relative validity, three to six 24-hour recalls were used as the reference method.

    Our study showed acceptable relative validity for food (correlation coefficients median and range: 0.51 [0.09-0.88]) and nutrients (correlation coefficients median and range: 0.40 [0.05-0.68]), and our results were comparable to those from other studies [14,21,32,33,46,48-52]. In the NIH–AARP study, validity coefficients for energy-adjusted nutrients ranged from 0.36 to 0.76 [46].

    For validation studies, it has been suggested that correlation coefficients should be ≥ 0.3, preferably over 0.4, and optimally in the range of 0.5-0.7 [6,45,53]. In our study, only eight food groups out of 40 and eight nutrients out of 34 had correlation coefficients lower than 0.3.

    For food groups, the lowest correlation coefficients were found for foods that are not consumed regularly such as “variety meats,” “whole-grain pasta, rice and wheat,” “legumes,” and “sunflower oil and groundnut oil”. Such findings on rarely consumed foods have been previously reported [32-34]. Low correlation coefficients were also observed for mixed items such as the “sweet snacks, chocolate, and Danish pastries” item. Due to the number of foods included in these groups, consumption may be difficult to estimate. Finally, a low correlation coefficient was observed for the “potatoes and other tubers” item. The estimation of its consumption may have been difficult due to the large consumption of mixed dishes including potatoes in France. Underestimation of water during the 24-hour recall has previously been reported [34] arguing that even if its consumption is systematically asked, it is easily forgotten.

    Because of social desirability [42,54], food groups such as “alcoholic beverages” and “fruit” may have been under- and overestimated respectively. Over-reporting of fruit and vegetable intake by subjects seeking social approval is a common bias [55]. As previously reported [56,57], correlation coefficients were lower for vegetables than for fruit. According to Wakai [57], it may be partly because the frequency of fruit consumption is easier to report than vegetables because fruit is more often consumed raw whereas vegetables are more frequently part of cooked dishes and therefore not integrally recalled. Furthermore, fruit is frequently consumed as a single food item and comes in natural or typical units, whereas vegetables are often sliced or cut which makes them more difficult to quantify [56].

    In our study, unadjusted correlation coefficients for macronutrients ranged from 0.47 to 0.57. Similar results were previously obtained for online FFQs (range: 0.06-0.68 [21,48,50], FFQs of 50 items max; range: 0.22-0.53 [14,51,52] and FFQs developed for adults; range: 0.29-0.61 [32,33,49]).

    All macronutrients had correlation coefficients in the range of 0.5-0.7. Regarding the nutrients, the highest correlation coefficient was observed for alcohol (unadjusted correlation coefficient of 0.77). One of the lowest coefficients was observed for sodium (deattenuated energy adjusted correlation coefficient of 0.07). As reported in the pilot study (validation study of the paper version of the FFQ among patients with chronic kidney disease [8]), even though a question about salt added after food preparation was asked in the specific part of the questionnaire, it was still difficult to estimate its intake. However, when looking at individuals’ rankings, 64% of participants were classified in the same or adjacent quintile in terms of sodium intake when comparing the FFeQ2 and the 24-hour recalls.

    After adjustment for energy some correlation coefficients were increased, and others were decreased. According to Willett et al [58], energy adjustment can increase the correlation coefficients when the variability of the nutrient intake is related to energy intake, or it can decrease when the variability of the nutrient is subject to systematic errors of under or overestimation of reported food consumption.

    Despite some differences in estimations in both foods and nutrients by the questionnaire, agreement in classification was comparable to what other studies have shown [59] or slightly lower than shown in other studies [14,32]. Our results were close to the recommended 70% [60]. The highest level of participants classified in opposite quintiles was observed for retinol (18%). One of the main sources of retinol are variety meats, which were rarely consumed and for which consumption was probably difficult to evaluate with only three to six 24-hour recalls.

    Reproducibility

    Our study showed acceptable reproducibility for most foods (ICC range: 0.33-0.72, median 0.60) and nutrients (ICC range: 0.55-0.73, median 0.65). Our findings were comparable to prior reported correlation coefficients for reproducibility [25,49,61]. An important factor influencing reproducibility is the period between the two questionnaires. We adopted a one-year time interval which is long but frequently used and reported as acceptable [45,62,63]. However, we cannot exclude that some dietary changes may have occurred during the period. The reproducibility observed here may therefore be lower than the true value.

    As previously reported in the literature, a slight decrease in food and nutrient intakes was observed between FFeQ1 and FFeQ2 [16,25,32,64,65]. Due to the completion of the 24-hour recalls and the FFeQ1, a learning effect may explain this trend [66]. In favor of this hypothesis, several authors found that the second FFQ, which indicated reduced nutrient intake, was more valid than the first one when compared to 24-hour recalls [32,65,67].

    According to a review, correlation coefficients of 0.5 to 0.7 between two administrations are commonly reported [45]. In our study, 75% and 100% of the studied food groups and nutrients had correlation coefficients ≥0.5. Not one food group had a correlation coefficient lower than 0.3.

    Agreement in classification was very good (median of 80% and 79% for food and nutrient intakes respectively). For all nutrient intakes 74% to 94% of participants were classified in the same or adjacent quintile.

    Strengths and Limitations

    The current work has some limitations. People involved in the current study were volunteers. Volunteers may be more health-conscious, pay more attention to their diets than the average population, and therefore provide more accurate responses to questionnaires. However, subjects participating in observational epidemiological or clinical studies that are likely to use this tool in the future are also volunteers.

    Our work has several strengths. The tool we developed was easy to complete and not time consuming. The implementation of photographs helped the participants estimate the amounts of food consumed and it has previously been shown that the use of photographs improves the ability to report the true quantity of dietary intakes [68].

    In addition, a total of 92 participants were included in our study. It is higher than the reported number in recent studies [21,69,70].

    Here, we present relative validity results for the online FFQ in a sample of French adults. A paper version of the FFQ has previously been validated in a sample of patients with chronic kidney disease [8] and further validity studies will now be conducted in specific population subgroups (for example, adolescents or cancer survivors). One of the main strengths of the consortium is that we will have a unique tool (due to the shared 40 items in the first part of the FFQ), that is useful for the comparison between several populations. The questionnaire is now available for other epidemiological and clinical studies interested in assessing the habitual diet quickly. We validated a Web-based version of the FFQ which provides valuable insights: it enables an interactive interface for participants and improves the quality of answers by directly including cutoff values and messages of alert in case of inconsistent, abnormal, or missing data.

    Conclusions

    For most food groups and nutrients, the FFeQ showed acceptable relative validity and reproducibility in a sample of French adults. It appears to be valid to rank individuals based on their food and nutrient intakes and can now be used in large-scale epidemiological studies as well as in clinical routine to easily and quickly assess the habitual diet. Developing an evidence-based smartphone application from the FFeQ is the next step. This type of tool may further be used to monitor patients’ nutrient intakes and provide them with instantaneous feedback and nutritional recommendations about their diets.

    Acknowledgments

    This work was supported by a grant from the French Research Institute in Public Health (IReSP) and the French multi-organism thematic Institutes in Public Health (ITMO Santé Publique). AA is supported by a doctoral grant from the French National Cancer Institute. GF is supported by the French Research Agency (ANR, Agence Nationale de la Recherche) via an “Investissement d’Avenir” grant (investment for the future grant, ANR-10-COHO-0006) that supports the E4N study. The IReSP, ITMO Santé Publique, and ANR had no role in the design, analysis or writing of this article. The authors are grateful to all participants for providing the data used in this relative validity and reproducibility study. We thank Maryvonne Niravong and Françoise Clavel-Chapelon for their advice and contributions.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Flow diagram.

    JPG File, 110KB
    ‎

    Multimedia Appendix 2

    Extract from the SFFeQ, French and original version.

    JPG File, 76KB
    ‎

    Multimedia Appendix 3

    Extract from the SFFeQ, version translated in English.

    JPG File, 86KB
    ‎

    Multimedia Appendix 4

    SFFQ items used to obtain nutritional data (n=44).

    JPG File, 140KB
    ‎

    Multimedia Appendix 5

    Reproducibility of food group consumptions of the SFFeQ (n=223).

    JPG File, 239KB
    ‎

    Multimedia Appendix 6

    Reproducibility of nutrient intakes of the SFFeQ (n=223).

    JPG File, 167KB
    1. WHO | Nutrition Internet. WHO   URL: http://www.who.int/topics/nutrition/en/ [accessed 2018-05-30] [WebCite Cache]
    2. WCRF / AICR. Food, Nutrition, Physical Activity, and the Prevention of Cancer: a Global Perspective 2009   URL: http://www.aicr.org/assets/docs/pdf/reports/Second_Expert_Report.pdf [accessed 2018-05-30] [WebCite Cache]
    3. Dietary GAC. Scientific Report of the 2015 Dietary Guidelines Advisory Committee. Scientific Advisory Report of the 2015 Dietary Guidelines Advisory Committee to the Secretary of HealthHuman Servicesthe Secretary of Agriculture: U.S. Department of Agriculture, Agricultural Research Service, Washington D. C; 2015.   URL: https:/​/health.​gov/​dietaryguidelines/​2015-scientific-report/​pdfs/​scientific-report-of-the-2015-dietary-guidelines-advisory-committee.​pdf [accessed 2018-05-30] [WebCite Cache]
    4. Satija A, Yu E, Willett WC, Hu FB. Understanding nutritional epidemiology and its role in policy. Adv Nutr 2015 Jan;6(1):5-18 [FREE Full text] [CrossRef] [Medline]
    5. Subar AF, Kushi LH, Lerman JL, Freedman LS. Invited Commentary: The Contribution to the Field of Nutritional Epidemiology of the Landmark 1985 Publication by Willett et al. Am J Epidemiol 2017 Jun 01;185(11):1124-1129. [CrossRef] [Medline]
    6. Willett W. Nutritional Epidemiology. Third Edition. Oxford, New York: Oxford University Press; 2012:978.
    7. Shim J, Oh K, Kim HC. Dietary assessment methods in epidemiologic studies. Epidemiol Health 2014;36:e2014009 [FREE Full text] [CrossRef] [Medline]
    8. Affret A, Wagner S, El FD, Dow C, Correia E, Niravong M, CKD-REIN study investigators, et al. Validity and reproducibility of a short food frequency questionnaire among patients with chronic kidney disease. BMC Nephrol 2017 Sep 15;18(1):297 [FREE Full text] [CrossRef] [Medline]
    9. Clavel-Chapelon F, van LMJ, Giubout C, Niravong MY, Goulard H, Le CC, et al. E3N, a French cohort study on cancer risk factors. E3N Group. Etude Epidémiologique auprès de femmes de l'Education Nationale. Eur J Cancer Prev 1997 Oct;6(5):473-478. [Medline]
    10. Clavel-Chapelon F, E3N Study Group. Cohort Profile: The French E3N Cohort Study. Int J Epidemiol 2015 Jun;44(3):801-809. [CrossRef] [Medline]
    11. Stengel B, Combe C, Jacquelinet C, Briançon S, Fouque D, Laville M, et al. [The French Chronic Kidney Disease-Renal Epidemiology and Information Network (CKD-REIN) cohort study: To better understand chronic kidney disease]. Nephrol Ther 2016 Apr;12 Suppl 1:S49-S56. [CrossRef] [Medline]
    12. Montagni I, Guichard E, Carpenet C, Tzourio C, Kurth T. Screen time exposure and reporting of headaches in young adults: A cross-sectional study. Cephalalgia 2016 Oct;36(11):1020-1027. [CrossRef] [Medline]
    13. Vandentorren S, Bois C, Pirus C, Sarter H, Salines G, Leridon H, Elfe team. Rationales, design and recruitment for the Elfe longitudinal study. BMC Pediatr 2009 Sep 21;9:58 [FREE Full text] [CrossRef] [Medline]
    14. Vercambre M, Boutron-Ruault M, Niravong M, Berr C, Clavel-Chapelon F, Ragusa S. Performance of a short dietary questionnaire to assess nutrient intake using regression-based weights. Public Health Nutr 2009 Apr;12(4):547-552. [CrossRef] [Medline]
    15. Giovannelli J, Dallongeville J, Wagner A, Bongard V, Laillet B, Marecaux N, et al. Validation of a short, qualitative food frequency questionnaire in French adults participating in the MONA LISA-NUT study 2005-2007. J Acad Nutr Diet 2014 Apr;114(4):552-561. [CrossRef] [Medline]
    16. Barrat E, Aubineau N, Maillot M, Derbord E, Barthes P, Lescuyer J, et al. Repeatability and relative validity of a quantitative food-frequency questionnaire among French adults. Food Nutr Res Internet 2012 Oct 30;56:18472.
    17. Béliard S, Coudert M, Valéro R, Charbonnier L, Duchêne E, Allaert FA, et al. Validation of a short food frequency questionnaire to evaluate nutritional lifestyles in hypercholesterolemic patients. Ann Endocrinol (Paris) 2012 Dec;73(6):523-529. [CrossRef] [Medline]
    18. Laviolle B, Froger-Bompas C, Guillo P, Sevestre A, Letellier C, Pouchard M, et al. Relative validity and reproducibility of a 14-item semi-quantitative food frequency questionnaire for cardiovascular prevention. Eur J Cardiovasc Prev Rehabil 2005 Dec;12(6):587-595. [Medline]
    19. Benedik E, Koroušić SB, Simčič M, Rogelj I, Bratanič B, Ding EL, et al. Comparison of paper- and web-based dietary records: a pilot study. Ann Nutr Metab 2014;64(2):156-166. [CrossRef] [Medline]
    20. Illner A, Freisling H, Boeing H, Huybrechts I, Crispim SP, Slimani N. Review and evaluation of innovative technologies for measuring diet in nutritional epidemiology. Int J Epidemiol 2012 Aug;41(4):1187-1203 [FREE Full text] [CrossRef] [Medline]
    21. Bentzen SMR, Knudsen VK, Christiensen T, Ewers B. Relative validity of a web-based food frequency questionnaire for patients with type 1 and type 2 diabetes in Denmark. Nutr Diabetes 2016 Sep 26;6(9):e232 [FREE Full text] [CrossRef] [Medline]
    22. Arab L, Tseng C, Ang A, Jardack P. Validity of a multipass, web-based, 24-hour self-administered recall for assessment of total energy intake in blacks and whites. Am J Epidemiol 2011 Dec 01;174(11):1256-1265 [FREE Full text] [CrossRef] [Medline]
    23. Carter MC, Albar SA, Morris MA, Mulla UZ, Hancock N, Evans CE, et al. Development of a UK Online 24-h Dietary Assessment Tool: myfood24. Nutrients 2015 Jun;7(6):4016-4032 [FREE Full text] [CrossRef] [Medline]
    24. Forster H, Fallaize R, Gallagher C, O'Donovan CB, Woolhead C, Walsh MC, et al. Online dietary intake estimation: the Food4Me food frequency questionnaire. J Med Internet Res 2014;16(6):e150 [FREE Full text] [CrossRef] [Medline]
    25. Labonté M, Cyr A, Baril-Gravel L, Royer M, Lamarche B. Validity and reproducibility of a web-based, self-administered food frequency questionnaire. Eur J Clin Nutr 2012 Feb;66(2):166-173. [CrossRef] [Medline]
    26. Subar AF, Kirkpatrick SI, Mittl B, Zimmerman TP, Thompson FE, Bingley C, et al. The Automated Self-Administered 24-hour dietary recall (ASA24): a resource for researchers, clinicians, and educators from the National Cancer Institute. J Acad Nutr Diet 2012 Aug;112(8):1134-1137 [FREE Full text] [CrossRef] [Medline]
    27. Kipnis V, Midthune D, Freedman L, Bingham S, Day NE, Riboli E, et al. Bias in dietary-report instruments and its implications for nutritional epidemiology. Public Health Nutr 2002 Dec;5(6A):915-923. [CrossRef] [Medline]
    28. Cottet V, Touvier M, Fournier A, Touillaud MS, Lafay L, Clavel-Chapelon F, et al. Postmenopausal breast cancer risk and dietary patterns in the E3N-EPIC prospective cohort study. Am J Epidemiol 2009 Nov 15;170(10):1257-1267. [CrossRef] [Medline]
    29. FAO/WHO/ONU. Energy and Protein Requirements. Report of a Join Expert Consultation   URL: http://apps.who.int/iris/bitstream/10665/39527/1/WHO_TRS_724_%28chp1-chp6%29.pdf [accessed 2018-05-30] [WebCite Cache]
    30. Schofield WN. Predicting basal metabolic rate, new standards and review of previous work. Hum Nutr Clin Nutr 1985;39 Suppl 1:5-41. [Medline]
    31. Bonifacj C, Gerber M, Scali J, Daures JP. Comparison of dietary assessment methods in a southern French population: use of weighed records, estimated-diet records and a food-frequency questionnaire. Eur J Clin Nutr 1997 Apr;51(4):217-231. [Medline]
    32. Deschamps V, de LB, Lafay L, Borys J, Charles MA, Romon M. Reproducibility and relative validity of a food-frequency questionnaire among French adults and adolescents. Eur J Clin Nutr 2009 Feb;63(2):282-291 [FREE Full text] [CrossRef] [Medline]
    33. Kesse-Guyot E, Castetbon K, Touvier M, Hercberg S, Galan P. Relative validity and reproducibility of a food frequency questionnaire designed for French adults. Ann Nutr Metab 2010;57(3-4):153-162. [CrossRef] [Medline]
    34. van LMJ, Lucas F, Clavel F, Slimani N, Villeminot S. Relative validity and reproducibility of a French dietary history questionnaire. Int J Epidemiol 1997;26 Suppl 1:S128-S136 [FREE Full text] [Medline]
    35. AFSSA (Agence Française de Sécurité Sanitaire des Aliments). 2009. INCA2 - Etude Individuelle Nationale des Consommations Alimentaires 2   URL: https://www.anses.fr/fr/system/files/PASER-Ra-INCA2.pdf [accessed 2018-05-30] [WebCite Cache]
    36. Lucas F, Niravong M, Villeminot S, Kaaks R, Clavel-Chapelon F. Estimation of food portion size using photographs: validity, strengths, weaknesses and recommendations. J Hum Nutr Diet 1995 Feb;8(1):65-74. [CrossRef]
    37. Ciqual.   URL: https://ciqual.anses.fr/ [accessed 2018-05-30] [WebCite Cache]
    38. Rosner B, Willett WC. Interval estimates for correlation coefficients corrected for within-person variation: implications for study design and hypothesis testing. Am J Epidemiol 1988 Feb;127(2):377-386. [Medline]
    39. Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986 Feb 8;1(8476):307-310. [Medline]
    40. Bland JM, Altman DG. Agreed statistics: measurement method comparison. Anesthesiology 2012 Jan;116(1):182-185. [CrossRef] [Medline]
    41. Altman D. Practical Statistics for Medical Research. London: CRC Press; 1990:978.
    42. Hébert JR, Hurley TG, Steck SE, Miller DR, Tabung FK, Peterson KE, et al. Considering the value of dietary assessment data in informing nutrition-related health policy. Adv Nutr 2014 Jul;5(4):447-455 [FREE Full text] [CrossRef] [Medline]
    43. Naska A, Lagiou A, Lagiou P. Dietary assessment methods in epidemiological research: current state of the art and future prospects. F1000Res 2017;6:926 [FREE Full text] [CrossRef] [Medline]
    44. Carroll RJ, Midthune D, Subar AF, Shumakovich M, Freedman LS, Thompson FE, et al. Taking advantage of the strengths of 2 different dietary assessment instruments to improve intake estimates for nutritional epidemiology. Am J Epidemiol 2012 Feb 15;175(4):340-347 [FREE Full text] [CrossRef] [Medline]
    45. Cade J, Thompson R, Burley V, Warm D. Development, validation and utilisation of food-frequency questionnaires - a review. Public Health Nutr 2002 Aug;5(4):567-587. [CrossRef] [Medline]
    46. Willett WC, Sampson L, Stampfer MJ, Rosner B, Bain C, Witschi J, et al. Reproducibility and validity of a semiquantitative food frequency questionnaire. Am J Epidemiol 1985 Jul;122(1):51-65. [Medline]
    47. Freedman LS, Schatzkin A, Midthune D, Kipnis V. Dealing with dietary measurement error in nutritional cohort studies. J Natl Cancer Inst 2011 Jul 20;103(14):1086-1092 [FREE Full text] [CrossRef] [Medline]
    48. Apovian CM, Murphy MC, Cullum-Dugan D, Lin P, Gilbert KM, Coffman G, et al. Validation of a web-based dietary questionnaire designed for the DASH (dietary approaches to stop hypertension) diet: the DASH online questionnaire. Public Health Nutr 2010 May;13(5):615-622 [FREE Full text] [CrossRef] [Medline]
    49. Bassett JK, English DR, Fahey MT, Forbes AB, Gurrin LC, Simpson JA, et al. Validity and calibration of the FFQ used in the Melbourne Collaborative Cohort Study. Public Health Nutr 2016 Sep;19(13):2357-2368. [CrossRef] [Medline]
    50. Christensen SE, Möller E, Bonn SE, Ploner A, Wright A, Sjölander A, et al. Two New Meal- and Web-Based Interactive Food Frequency Questionnaires: Validation of Energy and Macronutrient Intake. J Med Internet Res Internet 2013 Jun 2013 Jun 05;15:e109-e124.
    51. Mannato LW, Pereira TSS, Velasquez-Melendez G, Cardoso LDO, Benseñor IM, Molina MDCB. Comparison of a short version of the Food Frequency Questionnaire with its long version--a cross-sectional analysis in the Brazilian Longitudinal Study of Adult Health (ELSA-Brasil). Sao Paulo Med J 2015;133(5):414-420 [FREE Full text] [CrossRef] [Medline]
    52. Verger EO, Armstrong P, Nielsen T, Chakaroun R, Aron-Wisnewsky J, Gøbel RJ, MetaCardis Consortium. Dietary Assessment in the MetaCardis Study: Development and Relative Validity of an Online Food Frequency Questionnaire. J Acad Nutr Diet 2017 Jun;117(6):878-888. [CrossRef] [Medline]
    53. Masson LF, McNeill G, Tomany JO, Simpson JA, Peace HS, Wei L, et al. Statistical approaches for assessing the relative validity of a food-frequency questionnaire: use of correlation coefficients and the kappa statistic. Public Health Nutr 2003 May;6(3):313-321. [CrossRef] [Medline]
    54. Hebert JR, Ma Y, Clemow L, Ockene IS, Saperia G, Stanek EJ, et al. Gender differences in social desirability and social approval bias in dietary self-report. Am J Epidemiol 1997 Dec 15;146(12):1046-1055. [Medline]
    55. Hebert JR, Clemow L, Pbert L, Ockene IS, Ockene JK. Social desirability bias in dietary self-report may compromise the validity of dietary intake measures. Int J Epidemiol 1995 Apr;24(2):389-398. [Medline]
    56. De KW, Dekkers A, Van VV, Ottevaere C, Van OH, De HS, et al. Relative validity of a short qualitative food frequency questionnaire for use in food consumption surveys. Eur J Public Health 2013 Oct;23(5):737-742. [CrossRef] [Medline]
    57. Wakai K. A review of food frequency questionnaires developed and validated in Japan. J Epidemiol 2009;19(1):1-11 [FREE Full text] [Medline]
    58. Willett WC, Howe GR, Kushi LH. Adjustment for total energy intake in epidemiologic studies. Am J Clin Nutr 1997 Apr;65(4 Suppl):1220S-1228S; discussion 1229S. [Medline]
    59. Sluik D, Bezemer R, Sierksma A, Feskens E. Alcoholic Beverage Preference and Dietary Habits: A Systematic Literature Review. Crit Rev Food Sci Nutr 2016 Oct 25;56(14):2370-2382. [CrossRef] [Medline]
    60. Pietinen P, Hartman AM, Haapa E, Räsänen L, Haapakoski J, Palmgren J, et al. Reproducibility and validity of dietary assessment instruments. II. A qualitative food frequency questionnaire. Am J Epidemiol 1988 Sep;128(3):667-676. [Medline]
    61. Cantin J, Latour E, Ferland-Verry R, Morales SS, Lambert J, Faraj M, et al. Validity and reproducibility of a food frequency questionnaire focused on the Mediterranean diet for the Quebec population. Nutr Metab Cardiovasc Dis 2016 Feb;26(2):154-161. [CrossRef] [Medline]
    62. Kaaks R, Slimani N, Riboli E. Pilot phase studies on the accuracy of dietary intake measurements in the EPIC project: overall evaluation of results. European Prospective Investigation into Cancer and Nutrition. Int J Epidemiol 1997;26 Suppl 1:S26-S36. [Medline]
    63. Molag ML, de VJHM, Ocké MC, Dagnelie PC, van DBPA, Jansen MCJF, et al. Design characteristics of food frequency questionnaires in relation to their validity. Am J Epidemiol 2007 Dec 15;166(12):1468-1478. [CrossRef] [Medline]
    64. Fernández-Ballart JD, Piñol JL, Zazpe I, Corella D, Carrasco P, Toledo E, et al. Relative validity of a semi-quantitative food-frequency questionnaire in an elderly Mediterranean population of Spain. Br J Nutr 2010 Jun;103(12):1808-1816. [CrossRef] [Medline]
    65. Marques-Vidal P, Ross A, Wynn E, Rezzi S, Paccaud F, Decarli B. Reproducibility and relative validity of a food-frequency questionnaire for French-speaking Swiss adults. Food Nutr Res Internet 2011 May 2011 May 06;55:5905.
    66. Wong SS, Boushey CJ, Novotny R, Gustafson DR. Evaluation of a computerized food frequency questionnaire to estimate calcium intake of Asian, Hispanic, and non-Hispanic white youth. J Am Diet Assoc 2008 Mar;108(3):539-543. [CrossRef] [Medline]
    67. Malekshah AF, Kimiagar M, Saadatian-Elahi M, Pourshams A, Nouraie M, Goglani G, et al. Validity and reliability of a new food frequency questionnaire compared to 24 h recalls and biochemical measurements: pilot phase of Golestan cohort study of esophageal cancer. Eur J Clin Nutr 2006 Aug;60(8):971-977. [CrossRef] [Medline]
    68. Liu X, Wang X, Lin S, Song Q, Lao X, Yu IT. Reproducibility and Validity of a Food Frequency Questionnaire for Assessing Dietary Consumption via the Dietary Pattern Method in a Chinese Rural Population. PLoS One 2015;10(7):e0134627 [FREE Full text] [CrossRef] [Medline]
    69. Buscemi S, Rosafio G, Vasto S, Massenti FM, Grosso G, Galvano F, et al. Validation of a food frequency questionnaire for use in Italian adults living in Sicily. Int J Food Sci Nutr 2015;66(4):426-438. [CrossRef] [Medline]
    70. Fallaize R, Forster H, Macready AL, Walsh MC, Mathers JC, Brennan L, et al. Online dietary intake estimation: reproducibility and validity of the Food4Me food frequency questionnaire against a 4-day weighed food record. J Med Internet Res 2014;16(8):e190 [FREE Full text] [CrossRef] [Medline]

    ‎
    BMI: body mass index
    BMR: basal metabolic rate
    FFQ: Food Frequency Questionnaire
    FFeQ: Food Frequency e-Questionnaire

    Edited by G Eysenbach; submitted 05.10.17; peer-reviewed by M Boaz, P Wark, N Bragazzi, H Miller; comments to author 10.12.17; revised version received 01.02.18; accepted 18.02.18; published 05.07.18

    Copyright

    ©Aurélie Affret, Douae El Fatouhi, Courtney Dow, Emmanuelle Correia, Marie-Christine Boutron-Ruault, Guy Fagherazzi. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 05.07.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.