Evaluating User Preferences, Comprehension, and Trust in Apps for Environmental Health Hazards: Qualitative Case Study

Background Climate change is projected to increase environmental health hazard risks through fire-related air pollution and increased airborne pollen levels. To protect vulnerable populations, it is imperative that evidence-based and accessible interventions are available. The environmental health app, AirRater, was developed in 2015 in Australia to provide information on multiple atmospheric health hazards in near real time. The app allows users to view local environmental conditions, and input and track their personal symptoms to enable behaviors that protect health in response to environmental hazards. Objective This study aimed to develop insights into users’ perceptions of engagement, comprehension, and trust in AirRater to inform the future development of environmental health apps. Specifically, this study explored which AirRater features users engaged with, what additional features or functionality needs users felt they required, users’ self-perception of understanding app information, and their level of trust in the information provided. Methods A total of 42 adult AirRater users were recruited from 3 locations in Australia to participate in semistructured interviews to capture location- or context-specific experiences. Participants were notified of the recruitment opportunity through multiple avenues including newsletter articles and social media. Informed consent was obtained before participation, and the participants were remunerated for their time and perspectives. A preinterview questionnaire collected data including age range, any preexisting conditions, and location (postcode). All participant data were deidentified. Interviews were recorded, transcribed, and analyzed using thematic analysis in NVivo 12 (QSR International). Results Participants discussed app features and functionality, as well as their understanding of, and trust in, the information provided by the app. Most (26/42, 62%) participants used and valued visual environmental hazard features, especially maps, location settings, and hazard alerts. Most (33/42, 78%) found information in the app easy to understand and support their needs, irrespective of their self-reported literacy levels. Many (21/42, 50%) users reported that they did not question the accuracy of the data presented in the app. Suggested enhancements include the provision of meteorological information (eg, wind speed or direction, air pressure, UV rating, and humidity), functionality enhancements (eg, forecasting, additional alerts, and the inclusion of health advice), and clarification of existing information (eg, symptom triggers), including the capacity to download personal summary data for a specified period. Conclusions Participants’ perspectives can inform the future development of environmental health apps. Specifically, participants’ insights support the identification of key elements for the optimal development of environmental health app design, including streamlining, capacity for users to customize, use of real time data, visual cues, credibility, and accuracy of data. The results also suggest that, in the future, iterative collaboration between developers, environmental agencies, and users will likely promote better functional design, user trust in the data, and ultimately better population health outcomes.


Introduction
Background Climate change is predicted to increase health risks from environmental hazards worldwide. For example, climate projections suggest that many regions will experience an increased risk of air pollution from landscape fires [1], whereas exposure to heat-related health risks is projected to increase [2]. Adverse health outcomes associated with exposure to air pollution include lung and heart diseases, lung cancer, diabetes, neurological conditions, poor pregnancy outcomes, and premature death [3]. A changing climate is also likely to increase the health risks from airborne pollen owing to changes in pollen loads, allergenicity, and pollen season length [4]. The human and economic costs of such increases are likely to be substantial [5], and individuals living with pre-existing chronic conditions or from lower socioeconomic circumstances have been identified as the most vulnerable [6].
In this context, it is imperative to provide evidence-based, cost-effective interventions to reduce the impacts of environmental hazards on human health. Smartphone apps may offer part of the solution, for example, by providing individuals, including vulnerable populations, with easy and timely access to environmental hazard information or by providing tools to support the diagnosis and management of health outcomes triggered by environmental conditions [7][8][9]. Smartphone apps that collect symptom data can also provide a mechanism to track population-level health impacts of environmental hazards in real time [10]. For example, access to aggregated data on respiratory symptoms such as shortness of breath or heat-related symptoms such as light-headedness, can help public health departments monitor population health outcomes associated with environmental hazards and respond quickly to spikes in outcomes by releasing health alerts and targeted health campaigns [11].
Apps related to environmental health, such as air quality and pollen count apps, are rapidly proliferating and targeting different user groups, including children with asthma [12,13]. Although user perspectives and preferences regarding apps designed for other health conditions have been studied [14][15][16], there remains a paucity of evidence regarding how and why such apps do (or do not) support users in understanding and responding appropriately to atmospheric health hazard information. Some studies have investigated the effectiveness of environmental health apps with respect to messaging strategies [17] and behavior change support [18][19][20]; however, very few have tested user preferences related to app design or the extent to which users comprehend and trust the information presented. Given the plethora of possibilities available with respect to functionality, risk communication, and the design of both the overall app and individual features, it is imperative that we address this research gap to underpin effective and evidence-based environmental health app design in the future.

Protecting Health From Environmental Hazard Risks
The AirRater app offers an ideal opportunity to explore these critical questions. AirRater was developed in Australia in 2015 with the aim of protecting individuals from 3 key atmospheric health hazards: pollen, particulate pollution, and extreme heat [8]. The app was co-designed by a consortium of multidisciplinary researchers and environmental and health government agency representatives and is now freely available across Australia. The functionality of the app has been described in detail elsewhere [8,19]; however, its core features include (1) the provision of near real time information on air quality, temperature, and pollen counts; (2) notifications when atmospheric conditions are poor; and (3) the capacity for individuals to log their symptoms and learn about their personal sensitivities ( Figure 1). Users can input and track symptoms related to the nose (eg, itchy), eyes (eg, watery), lungs (eg, wheezy), throat, and heat (eg, light-headed). Users can also input custom symptoms related to other health outcomes and self-report the severity of the symptoms experienced (mild, moderate, and severe). The symptom tracking functionality also enables the app to support public health surveillance and ongoing epidemiological research [21,22]. For a full outline of the app's functionality and features, please see Multimedia Appendix 1. The app is supported by a website that provides users with information about data streams and sources.

Evaluating User Preferences, Comprehension, and Trust
Several factors make AirRater an ideal case study to explore how users perceive and react to different features of environmental health apps and the extent to which users understand and trust the information presented. First, AirRater represents an unusually holistic environmental health app because it provides information on multiple environmental health hazards (pollen, air pollution, and temperature) in multiple ways (via maps and by saved location) and couples environmental hazard information provision with features designed to help individuals understand and manage their health conditions (eg, symptom tracking, symptom modeling, and personalized alerts). In addition, AirRater has been well-evaluated, and detailed information on who uses the app and their motivations for doing so has been previously published [19,23]. Importantly, these prior evaluations have also demonstrated that the app is successful in supporting a diverse range of users to make decisions and implement behaviors to protect their health, both in the context of severe air pollution episodes and more routine conditions [8,19,23].
This study leverages these characteristics and understandings of AirRater to explore how users perceive and respond to various features of environmental health app design and how such design features do or do not support user comprehension and trust in the information presented. Using qualitative methods, we explore user experiences with AirRater from this perspective, aiming to inform the development of effective environmental health apps. We specifically address 4 research questions.
1. Which features of the AirRater app do users engage with? 2. What are users' additional information and functionality needs? 3. How do users perceive their comprehension of the information provided in the app? 4. What level of trust do users have in the information the app provides?

Overview
Numerous tools and frameworks exist to assess app integrity and quality, some of which have been validated [24][25][26][27][28][29][30][31][32][33][34]. Such tools are often prescriptive. Qualitative research methods provide the capacity to explore participant experiences in greater detail by enabling researchers to seek elaboration or clarification of initial participant responses. Accordingly, we developed a customized mixed methods evaluation framework based on the guide by the World Health Organization (WHO) to monitor and evaluate a digital health intervention [35] and the mobile app rating scale [34]. The WHO guide provides a practical framework and flexibility to explore specific questions of interest. The mobile app rating scale framework informed the thematic structure of the qualitative evaluation. On this basis, we developed a semistructured interview schedule that provided the capacity to prompt users for further details on their experiences based on their initial responses [36,37].
This study was conducted as part of a larger mixed methods study of AirRater users that covered multiple themes. The results with respect to usability and behavior change have been published elsewhere [19], and this paper specifically reports the results with respect to user perceptions of app design, comprehension, and trust.
In the interest of an independent evaluation, a qualitative researcher (AW) with no affiliation to the stakeholders involved in the app design and development was used to conduct the evaluation, undertake all interviews, and analyze all data. A qualitative researcher at the University of Tasmania, who was not affiliated with the app, reviewed the framework and protocol as an independent reviewer before its submission to ethics.

Recruitment
As reported elsewhere [19], adults (18 years and older) were selected for recruitment if they resided in one of the following three locations in Australia: (1) Tasmania, (2) the Australian Capital Territory (ACT), or (3) Port Macquarie, New South Wales. Users were recruited from multiple locations to capture any location-or context-specific experiences, which were strategically chosen based on known episodes of high pollen days (Tasmania) and prolonged poor air quality because of the 2019-2020 wildfire events (ACT and Port Macquarie). Users were recruited through calls for participation via multiple media platforms, including a targeted email to registered AirRater users, an article in the AirRater newsletter, social media posts (Facebook and Twitter), and requests for participation during local radio interviews with AirRater team members. In total, 42 users were recruited to participate in the qualitative evaluation, with numbers almost evenly divided between Tasmania (20/42, 48%) and ACT (21/42, 50%). Despite multiple attempts to secure more participants, only 2% (1/42) of participants were recruited from Port Macquarie given that both locations experienced substantial wildfire smoke from 2019-20 wildfire events, ACT and Port Macquarie data were aggregated to protect the anonymity of the Port Macquarie participants.

Ethical Considerations
An ethics application for the evaluation was submitted to and approved by the University of Tasmania Health and Human Research Ethics Committee (ID: H0015006). As per the ethics approval, participants who contacted the team expressing an interest in participating were sent an information sheet and consent form before confirming their eligibility. The information sheet included information on the purpose of the study, the study's funding arrangements, details about what participants would be asked to do during the study, and the benefits and risks of participation. The participants were also informed that they were free to withdraw from the evaluation at any time. The evaluation and data collection methods were discussed with potential participants, who then returned their consent form if they were still happy to proceed with participation. Participants were notified that deidentified data would be stored at the University of Tasmania for a minimum of 5 years in accordance with the Australian National Health and Medical Research Council guidelines and would only be accessible to a subset of investigators involved in the evaluation (AW, FHJ, SLC, NC, and PJJ). Consent forms, as well as deidentified electronic transcripts and audio recordings, were stored in separate folders on a secure server at the Menzies Institute for Medical Research, Tasmania. All data were password-protected and accessible only to a subset of investigators involved in the evaluation (AW, FHJ, SLC, NC, and PJJ). Informed consent was obtained from each participant before their involvement in the study. Participants were remunerated for their time and contributions with an AUD $20 (US $14.50) gift card following their participation in the interview.

Data Collection
Data were collected using 2 methods. First, a preinterview questionnaire delivered through SurveyMonkey collected key demographic details from the participants, including age range, any pre-existing conditions, and location (postcode, Multimedia Appendix 1). The questionnaire also collected data from participants on their personal descriptions of the app's purpose, period of use, primary motivation for downloading the app, and perceptions of use over time. Second, semistructured interviews were then conducted. The full interview schedule was published in the study by Workman et al [19]; the interview schedule components relevant to the results presented in this paper are provided in Multimedia Appendix 1. All interviews were conducted via telephone or a web-based conferencing platform because of COVID-19 restrictions. With participants' permission, all interviews were recorded to verify the accuracy of the transcripts.

Data Analysis
Demographic data collected by the preinterview questionnaire were aggregated for descriptive analysis. The qualitative analysis software NVivo 12 (QSR International) was used to support thematic coding and analysis of the interview data [38]. All 42 interviews were transcribed by 2 research team members and verified by 1 before being uploaded to NVivo. Starting with the themes underpinning the interview schedule as an initial coding framework [39], all interview transcripts were thematically analyzed by the qualitative researcher leading the evaluation (AW). The overarching themes supported the identification of subthemes that emerged from the data. The results of the analysis were initially discussed with 2 researchers affiliated with the design and development of the app (PJJ and SLC) and who were involved in previous AirRater evaluations of user surveys.

Questionnaire Data
The majority of the 42 participants (38/42, 90%) completed the preinterview questionnaire. Detailed results have been reported elsewhere [19]; however, the key results for contextualizing this study are presented in Table 1. Notably, most users had one or more respiratory conditions (asthma, allergic rhinitis, other lung conditions, or a combination), and most primarily used the app during seasons in which particular environmental triggers were present. Open-text responses from users indicated that they were most likely to use the app during the wildfire season (summer) or the pollen season (spring and summer), although some participants indicated that use was sporadic outside of these times.   [19].

Overview
The interview results are reported below and stratified into four sections aligned with our research questions: (1) engagement with different features of AirRater, (2) additional information and functionality needs, (3) comprehension of AirRater, and (4) trust in AirRater. Additional interview data covering different themes have been published elsewhere [19]. This paper focuses only on the data relevant to the themes outlined above. The key findings relevant to these 4 themes are summarized in Table 2, and the remainder of this section presents detailed results from each theme. Table 2. Summary of key findings from interview data, stratified by research question.

Summary of key findings Research question
Which features of the app do users engage with?

•
The visual map and location features are used most frequently.

•
Receipt of general alerts for elevated environmental hazard levels prompted app use.

•
Participants used the symptom reporting feature to varying extents. • A few participants indicated that the provision of real time information made AirRater their preferred information source.
What are users' additional information or functionality needs?
• Participants made a diverse range of suggestions and requests for additional data, features and functionality to enhance the app (see Multimedia Appendix 1 for a detailed summary).

•
Common requests included a more detailed breakdown of pollen data, automatic notification of nearest monitoring station, the inclusion of wind speed or direction, and the capacity to download personal summary data for a specified period.
How do users perceive their comprehension of the information in the app?
• Participants indicated they brought varying levels of scientific or air quality literacy to their interactions with the app.

•
The interface design, such as the color code, 1-word rating system, helped participants and their family members understand environmental hazard data. • Some participants indicated they were confused by the use of different metrics across different information sources.
Do users trust the information the app provides?
• Many participants did not question the accuracy of environmental hazard data presented in the app. • Some participants felt the app's affiliation with a university provided credibility. • Some participants expressed mistrust in AirRater data accuracy given their distance from a monitoring station.

•
The extent to which app ratings aligned with personal symptoms and visual cues (such as visible smoke) influenced trust in AirRater data. • Some participants questioned the conclusions of their personal health profile based on inputs to the symptom reporting feature.

Which Features of the App Do Users Engage With?
The participants were asked which features they used. Many participants (26/42, 62%) indicated using location ( Figure 1 and Multimedia Appendix 1) and maps ( Figure 1 and Multimedia Appendix 1) features most frequently. Some participants (4/42, 9%) specifically noted the value of the location function in supporting their decisions and monitoring the locations where they had a family:

What Are Users' Additional Information and Functionality Needs?
Participants were asked whether there were any additional features or information they wanted AirRater to provide. Most (38/42, 90%) participants recommended at least one enhancement to the app to supplement existing features or requested the provision of additional information. Given the diversity of participants, their specific health conditions, and their unique information needs, the responses were wide-ranging. However, common requests included a more detailed breakdown of the pollen count data, automatic notification of the nearest monitoring station, inclusion of wind speed or direction, and the capacity to download personal summary data for a specified period. A full summary is provided in Multimedia Appendix 1

How Do Users Perceive Their Comprehension of the Information in the App?
Participants were asked to describe their understanding of the information the app provided, how the interface does or does not support their comprehension, and whether additional information is needed to assist them in understanding the data. Participants indicated varying levels of perceived general air quality literacy; however, irrespective of this, most (33/42, 78%) participants indicated that they were able to sufficiently understand the information in the app to support their needs:

Do Users Trust the Information the App Provides?
Participants were asked whether they had ever questioned the data provided by AirRater. Participants indicated varying levels of trust in the information provided by AirRater. Many (21/42, 50%) participants did not question the accuracy of the data, citing reasons such as their inclusion on credible government agency websites and the comprehensiveness of the supplementary information. In contrast, several (10/42, 24%) participants indicated that they had questioned the information presented, in many cases attributing this lack of trust to a lack of data sensitivity or accuracy away from the monitoring stations:

Overview
Through semistructured interviews with 42 users, this study gained valuable insights into what features are useful, what facilitates user comprehension, and what influences users' levels of trust in the information provided by AirRater. Specifically, we found that location and map functions are the most useful features of AirRater for users. Furthermore, most participants were able to suggest app enhancements based on personal needs or preferences (Multimedia Appendix 1). We also found that, irrespective of self-reported literacy levels, most participants reported that the information provided in the app was easy to understand and supported their needs. Finally, we found that many participants did not question the accuracy of the data presented in the app. As discussed below, this new knowledge extends our understanding of effective design principles in environmental health apps and highlights key considerations for future app design (Multimedia Appendix 1). Furthermore, our results confirm that AirRater is used by a diverse group of individuals with unique health conditions, personal preferences, and information needs, highlighting the need for environmental health apps to manage the core design tension between meeting diverse user needs and streamlining for ease of use and comprehension. In the 4 sections below, we discuss our key findings in further detail and compare them with those of previous research. We also discuss implications for the development of future environmental health apps.

Engagement With Features
The map and location features of AirRater were highly valued because they supported users in monitoring the movement of hazards or environmental conditions in other locations where family members were based. This finding is pivotal for informing the future design of environmental health apps, as the ability to access hazard information quickly and easily at multiple locations supports app users not only to assess, plan, and mitigate their own risk but also that of family members located elsewhere. Furthermore, numerous participants reported that the provision of near real time data via the map and location functions was highly valued, particularly in the context of rapid changes in air quality during the Australian 2019-2020 wildfire seasons. Other studies have determined the importance of accurate and timely information. For example, Gooze et al [40] found that consumer satisfaction and use increased following the introduction of real time information for a transportation information tool. For environmental hazard information in particular, the presentation of near real time information is critical for users to adequately support decision-making.
We also found that alerts for hazard levels and symptom reporting were effective in prompting users to look at and engage with AirRater. This finding is congruent with earlier research findings of increased engagement and higher-frequency app visits from push notifications [18,41]. The role of alerts in hazard communication is particularly important for "invisible" threats, such as pollen and, at times, air quality, as it can raise awareness and literacy of such hazards. Finally, AirRater participant perspectives on the symptom reporting feature varied; a few participants indicated that it was useful, while other participants indicated that they had experienced both technical and practical difficulties with the feature. Difficulties reported included the inability to modify symptom reports and the repetitiveness of symptom reporting. Previous research findings have confirmed that repetitive and multistep processes can discourage users from engaging with a particular app or feature. In their analysis, Cho et al [42] examined cognitive factors, including eHealth literacy and health app use efficacy, in app use among a sample of 765 app users from South Korea. They found that health app use efficacy is correlated with continued app use and hypothesized that app features that require multiple steps to access information require users to invest time and energy, which may impact their perceived health app use efficacy [42]. Our findings underline the importance of simplified and streamlined symptom reporting processes to maximize the engagement and utility of such features in environmental health apps.

Additional Information and Functionality Needs
As detailed in the Multimedia Appendix 1, participants suggested numerous enhancements to the app, including the incorporation of additional meteorological information (eg, wind speed or direction, air pressure, UV rating, humidity), functionality enhancements (eg, forecasting functionality, additional alerts, the inclusion of health advice), and clarification of existing information (eg, symptom triggers). Four common requests were presented by the participants: (1) a more detailed breakdown of pollen data, (2) automatic notification of the nearest monitoring station, (3) the inclusion of local wind speed or direction, and (4) the capacity to download personal summary data for a specified period. Our results on additional information and functionality highlight that AirRater users are diverse in their personal information and functionality needs. Although this diversity demonstrates the capacity of an environmental health app to effectively engage various populations, it also implies the fundamental importance of app personalization and customization to meet individual user needs [43,44], particularly when balancing the simultaneous need for simplicity and streamlining to enhance usability and comprehension.

Comprehension
First, despite acknowledging that they brought varying levels of air quality literacy to their engagement with AirRater, participants indicated that the current AirRater design, particularly the use of colors, facilitated their understanding of the data. The relationship between interface design and information comprehension has been reported by other researchers. For example, Caburnay et al [45] assessed a random sample of 110 diabetes-related apps for health literate design. They found that most apps studied used enabling elements, such as everyday language (88/110, 80%), bold and contrasting colors (89/110, 80.9%), and included visual, customizable content (77/110, 70%). Similarly, in their study exploring stakeholder perspectives for improving storm surge risk communication, Morrow et al [46] found that stakeholders preferred maps that used multiple colors to convey different levels of storm surge risk. Our results reaffirm the importance of color as a part of a strategic design approach for environmental health app development. Another key result was that some participants indicated that the use of different indices and metrics across information sources was confusing during the wildfire season (eg, 24-hour average vs hourly or real time levels). In the absence of standardized information, our findings highlight the importance of providing clear and accessible information on how to interpret the specific metrics presented for a given setting and the value of supplementary information via multiple media to address confusion when it arises. Finally, Stonbraker et al [47] found a group of end users with low health literacy preferred simple bar graphs with emojis, again emphasizing the need for simple visualizations that convey the health implications of data. Stonbraker, Porras, and Schnall [47] also emphasized the importance of testing visualizations with target groups, a targeted approach that could be used in future iterations of AirRater to assess preferences and verify comprehension.

Trust
Many participants reported general trust in the information provided by the app, without questioning the accuracy of the information provided, with some reporting AirRater's affiliation with a university as a pivotal factor. Our findings are congruent with results reported elsewhere [48]. For example, in their scoping review on trust in digital health interventions, Adjekum et al [49] identified factors such as ease of use, self-efficacy, customizability, credibility, and stakeholder engagement as enabling trust in digital health interventions. Participants who expressed mistrust in the information provided by AirRater reported distance from an air quality monitor or the app's conclusions about their personal triggers as their reasons to question the data, although this did not seem to impact their overall willingness to use the app. This finding underscores the importance of data integrity in environmental health apps for protecting individuals from unnecessary or inadvertent exposure to environmental hazards. The need for a comprehensive and accurate air quality and pollen monitoring network is pivotal to ensuring that environmental health apps can effectively support populations. It also suggests opportunities exist for indoor air quality monitoring and the potential for environmental health apps to connect with devices, such as indoor air quality monitors or air purifiers, to provide supplementary data on potential triggers present in an indoor environment. Figure 2 synthesizes the core implications of our study with respect to the environmental health app design. First, our study clearly showed that users value a level of customization with the capacity to access and select information based on their individual needs. However, streamlined design is important: users prefer features that are visual and involve minimal steps. In addition, users value real time information with transparency around data sources. Our results also highlight the importance of using visual cues to support comprehension; for example, the use of a one-word color rating system in AirRater supports users in understanding scientific information irrespective of their perceived level of health or air quality literacy and to support their family members, including children. The credibility of the app developer also proved to be important for influencing user trust as well as the perceived accuracy of data, in this case, provided by established air quality and pollen monitoring networks. Accordingly, environmental health apps must be supported by sufficient data inputs from adequate and robust monitoring networks. To ensure that key elements are integrated into app design, iterative collaboration among developers, environmental agencies, and users is likely to support better app functionality, enhance user trust in the data presented, and support the ultimate goal of improving population health outcomes. With respect to future research, this evaluation of AirRater as an example of an environmental health app suggests several future research needs. First, as environmental health issues can be highly localized, there is merit in pursuing similar evaluations of app utility and efficacy in other jurisdictions and countries. Although AirRater is currently only available in Australia, the implementation and evaluation of environmental health apps, such as AirRater, in other settings would extend the conclusions outlined here. Second, there is a strong need to investigate usability and comprehension among a more diverse range of user groups, including caregivers, children, and the elderly, across environmental health apps, to establish specific preferences and needs and to optimize health outcomes. At last, with respect to trust, it is important to investigate whether levels of trust in app data change because of likely changes to the air quality and pollen monitoring network in the coming years, either through the expansion of the existing network or the introduction of low-cost air quality sensors for individual residential use.

Limitations
Several limitations impact the strength of the conclusions presented in this study. First, in sampling a cohort of current AirRater users, results presented here are most useful for the purposes, as intended, of an in-depth evaluation of AirRater. However, the sample is not likely to be representative of all AirRater users or other populations who may choose to use environmental health apps. In this context, it is not possible to determine the extent to which the results presented here are generalizable to other groups. None of the participants in this study were culturally and linguistically diverse. Our choice of a "within person" study design has been vindicated by recent digital health reviews however greater diversity of participants would have been useful [50]. Another recognized limitation is the lack of representation of older children and young adults. Similarly, while a small number of respondents indicated having children aged under 15 years, conversations focused on the personal use of and experience with the app. An opportunity exists to evaluate AirRater further by targeting older children (ie, 10-17-year-olds) and young adults (ie, 18-24-year-olds) to determine whether any age-specific customization is required to support further uptake of the app in these age groups. This is particularly important, given the app's educational potential, as indicated by some participants. Finally, this evaluation only captures insights from AirRater app users, who are both engaged with the app and agree to participate. The opinions and responses of historical and current AirRater app users disengaged from the app were not accounted for in this study. Actively seeking the perspectives of users who no longer engage with the app would prove valuable for gaining unique insights into potential barriers to app use, as well as additional suggested app enhancements.

Conclusions
This qualitative evaluation of the free smartphone health app AirRater explored user preferences, comprehension, and trust. The participants' perspectives can inform the future development of environmental health apps. Accordingly, the perspectives presented in this paper contribute to identifying key considerations for successful environmental health app design, including customization, streamlining, use of real time data, visual cues, credibility, and accuracy of the data. These considerations are more likely to be prioritized when apps are designed in collaboration with developers, environmental agencies, and users. In the future, environmental health apps are likely to play a pivotal role in supporting populations facing increasingly severe and frequent global environmental changes and extreme events. Ensuring that environmental health apps are fit for purpose, effective, accurate, comprehensible, and trustworthy will assist in maximizing the health outcomes of the population.