Barriers to Dissemination of Local Health Data Faced by US State Agencies: Survey Study of Behavioral Risk Factor Surveillance System Coordinators

Introduction

It is widely acknowledged in the public health field that progress in information technology has paved the way for exciting opportunities to disseminate local level health data more efficiently [1,2]. The growth of the internet, mobile technologies, artificial intelligence, and other technological advances have enabled health information to become more easily accessible and widely available to a broader population [2-4]. There has been growing enthusiasm for the application of big data and its utility in public health, particularly for population-level health data [5]. Access to high quality population-level health data is essential for public health, as it informs us of disease tracking, health problems, and health surveillance at the subpopulation level [6,7]. Health informatics has enabled public health practitioners to assess public and population health information by accurately combining data from a wide range of disparate sources [7]. Despite the vast advances in technology, there are many challenges associated with availability of high-quality population-level health data [7,8]. These problems have been even further exacerbated during the COVID-19 pandemic, as there is a lack of consistency in the data and their reporting [9,10].

As each US state is responsible for its own health surveillance, decisions regarding technology implementation have varied from state to state [11]. One popular option is for state agencies to design and develop dynamic web-based data query systems (WDQSs), which allow users to customize data queries by choosing data sets, variables, measures, and the format for presenting query results [12]. There are many benefits to WDQSs, as state agencies are able to respond to data requests in a timely manner, provide data to a broader population, and contribute to the development of community health assessments and policy decisions [13]. Another popular option is for states to make static reports available to the public on the World Wide Web. Static reports are generally manually compiled by staff and are available in formats such as PDF, Microsoft Excel, and HTML. Static reports limit user choice to precalculated statistics and do not allow users to choose parameters for a query [14]. Given the advances in information technology, deployment of these outdated technologies, which can be difficult to use and plagued by missing or incomplete data, by US states is unfortunate [15]. In the private sector, industries such as finance have been successful in maximizing the potential of the internet, as changes in stock prices are made available within seconds to end users.

WDQSs were first implemented in the late 1990s, as states developed systems in which queries could be specified and results returned on the World Wide Web without requiring any additional software [12]. At the time, strong efforts were being made to achieve data liberation and use of open-source information technology solutions and collaboration to promote public health [15]. Collaboration has contributed to states sharing developments, ideas, and knowledge to meet a variety of public health assessment needs [16-18]. To reduce the cost burden, there has been a push toward open-source software, which costs little or no money to procure [19]. Open-source code can be easily shared, and its key benefits include free redistribution, inclusion of source code, easy modification of the code, and lack of need for an additional license [20]. For example, in 2000, Utah’s legacy Indicator Based Information System–Public Health (IBIS-PH) was developed using open-source code, and states such as New Mexico and Kentucky adopted the system within the first few years of its development. However, over the last decade, efforts toward WDQS implementation have stalled, and progress has slowed. Public health is at risk of falling behind from a technological standpoint [21]. Despite the significant advances in technology, including faster processors, improved bandwidth, and lower cost of storage, state agencies commonly face data sharing barriers between organizations within their state [21]. The purpose of this study was to investigate the barriers faced by state agencies to the implementation of WDQSs from the perspectives of Behavioral Risk Factor Surveillance System (BRFSS) coordinators from each state. We aimed to understand how challenges and perceptions of systems differ between states that have implemented a WDQS and states that have not. We hypothesized that the perceptions and challenges reported by the BRFSS coordinators would vary across states because of the coordinators’ significance in releasing health data and their role in technology decisions in their states [14].

Methods

Study Design and Sampling

We designed and administered a web-based questionnaire to BRFSS coordinators from all 50 states and the District of Columbia (n=51). The BRFSS, established in 1984, is a health-related telephone survey that collects state data from US residents in all 50 states regarding their health-related risk behaviors, chronic health conditions, and use of preventive services. The BRFSS coordinator in each state is responsible for gathering information about health behaviors and is responsible for the management and oversight of the BRFSS survey [22,23]. First, we queried the coordinators on how states disseminate their data. In January 2015, each prospective participant was sent an invitation letter by US Mail. We found the names and contact information of all the BRFSS coordinators on the US Centers for Disease Control and Prevention website [24]. The letter provided details of the study and indicated that participation was confidential, as the results would not be disclosed to anyone except the study staff. The letter specified that the coordinators’ participation was voluntary and that they could withdraw at any point of the survey. Each participant was sent a follow-up email containing a link to the survey with a secure user name and password within 1-2 weeks upon receipt of the letter.

We queried the participants on the basic characteristics of their data dissemination, including how their health data are disseminated, which technology is used, and which types of data are available. We asked, “Does your state present health data over the internet?” and participants answered yes or no. We also queried if their state presents data using an interactive WBQS, which types of data are publicly available (eg, BRFSS, births, deaths), and finally, which path they took to develop their software (eg, in-house, adopted from an outside vendor). We also asked participants to rate the level of challenges they face regarding the dissemination of their data on a 4-point Likert scale (1, not at all challenging; 2, not very challenging; 3, somewhat challenging; 4, very challenging). A Likert scale is a set of statements (items) offered for a hypothetical situation under study, in which participants are asked to show their level of agreement (eg, strongly agree, somewhat agree). We presented 2 to 4 questions for three different categories: cost, staffing and support, and other challenges. We then asked the coordinators how they would rate various system-related aspects of their health data systems. They were presented a list of items and rated each item on a 5-point Likert scale (1, poor; 2, fair; 3, good; 4, very good; 5, excellent). For the system aspects, 3 to 4 questions were presented for each category, including website performance, data quality, and accessibility and support.

Data Analysis

We obtained descriptive statistics and profile characteristics for the participants from each state. We also compared the means of the ratings of participants from states that have implemented a WDQS and from states that have not implemented a WDQS. The means were compared using two-sample t tests between states with and without a WDQS, with P<.05 used as the level of significance.

Ethics Approval and Consent to Participate

Ethics approval for the study protocol was received from the Human Subjects Protection Office at the University of Connecticut Health Center.

Results

Profile characteristics at the state level can be found in Table 1. A total of 43 of the 51 coordinators completed the survey, for an overall response rate of 84%. Of the 43 participants, 42 completed the web-based survey, while 1 participant completed the survey over the telephone. Of the 43 states, 41 (95%) had some form of health data available on the web, while 28 (65%) reported having implemented a WDQS. Data available included BRFSS (33/43, 77%), births (31/43, 72%), deaths (30/43, 70%), lead screening (5/43, 12%), and hospitalizations (4/43, 9%).

We report the mean ratings for the challenges faced and system aspects in Table 2. On a 4-point Likert scale, participants rated the cost of system development (mean score 3.33), consultants/vendors (mean score 3.05), and the cost of hardware/software (mean score 2.89) as the most challenging. The overall mean score for all challenges faced was 2.68 on the 4-point Likert scale. Participants reported the lack of political support (mean score 1.77) and issues with data privacy (mean 2.55) as less of a challenge. We analyzed the mean results of the reported challenges between states with and without a WDQS (Table 2). Participants from states without a WDQS reported the cost of hardware and software to be a greater challenge than those from states with a WDQS (mean score 3.55, 95% CI 3.09-3.78, vs mean score 2.63, 95% CI 2.25-3.00; P=.01). System aspects were rated higher (P=.01) for the standardization of vocabulary by participants from states with a WDQS (mean 3.32, 95% CI=2.94-3.69) versus those from states without a WDQS (mean=2.85, 95% CI 2.47-3.22).

Discussion

To our knowledge, this is the first national study to investigate barriers faced by state agencies to the dissemination of their health data using informants in key roles. The findings revealed that BRFSS coordinators rated their systems more favorably in states where a WDQS was implemented. Interestingly, despite the high cost of technology, staffing, implementation, and maintenance of technology-based systems and other factors, BRFSS coordinators from states that implemented a WDQS perceived their systems more favorably. We hypothesize that these findings are indications of a favorable assessment of the cost-benefit ratio of implementation of technology-based systems relative to low-cost health data systems. Adequate staffing and funding for state health data systems is lacking, which has impeded or slowed progress or halted data dissemination efforts in these states [25,26]. Our findings are more important than ever, given the reliance of society on trusted, reliable, and accurate public health data [27,28].

Prior research has indicated that organizations are reluctant to share their data due to organizational, technical, and political barriers [15]. In the current study, respondents reported that state agencies are generally willing to share their data and do not perceive political barriers as a significant challenge to data sharing. However, these findings should be taken with caution, as there may be bias because these perspectives were based exclusively on the experience of BRFSS coordinators, which may not be representative of that of other key stakeholders across states. Respondents also reported lack of interoperability between systems, as data may be transmitted in formats that are incompatible with the originating system. These findings are in line with prior work, in which it was reported that departments lack adequate staffing and resources to profile, “cleanse,” and manipulate these data so they are usable [15,29]. If data are not usable, they have limited utility and do not create significant opportunities for public health research. According to the latest Public Health Workforce Interests and Needs Survey (PH WINS), a nationally representative survey of the public health workforce, state agencies lack adequate trained staff who are able to handle and interpret these data [30,31]. Public health agencies are lacking workers in areas such as data-informed decision-making, health informatics, and data quality, which are essential in the dissemination of public health data [30-32].

Our findings should be interpreted with certain limitations in mind. First, the results may not be generalizable beyond the perspective of the BRFSS coordinator in each state. As each state may have multiple stakeholders who have a vested interest in the WDQS, the opinion of the BRFSS coordinator may not be representative of the consensus from that state. Secondly, our study may reflect bias, as BRFSS coordinators in states with a WDQS may rate their systems higher due to the additional investments states have made in this technology. Third, because the study includes a small number of participants, there is insufficient statistical power to detect small differences in ratings among states with and without a WDQS. Fourth, as BRFSS coordinators from 7/51 states (14%) did not respond to the survey, there may be systematic bias related to the missing information from these states. The reasons that the BRFSS coordinators from those states refused to participate are also unknown. Fifth, questions may be interpreted differently from one state to the next. Web-based expertise and technical maturity may also vary from one state to another, depending on their experience. Finally, although measures such as quality, timeliness, satisfaction, and access were assessed for multiple constructs, their definitions were not presented in the survey. Respondents may have interpreted these measures differently, potentially resulting in bias. For example, the definition of “quality” may be perceived differently from one state to the next. Despite these limitations, the current study is, to our knowledge, the first to compare system ratings and assessments of challenges to presenting health data to the public among states with more primitive versus more advanced data systems. Directions for future research include more comprehensive efforts to evaluate the utility of WDQSs, as evidence of their usefulness and their potential impact on public health may help justify the additional expenditures required. Additionally, it is recommended that state agencies aim toward collaboration and investigate open-source software options. This model has been successful in the clinical setting. For example, open-source software has been adopted by several hospitals and clinics. A similar model can be applied for future WDQS development, as states should aim to collaborate and work toward building robust systems that are easy adoptable. In summary, it is important to design systems that facilitate access to local health data; these data provide information regarding health challenges at the subpopulation level, which will ultimately help guide future public health research. These problems have been at the forefront during the COVID-19 pandemic [33,34] and should be urgently addressed moving forward.

Data Availability

Data and materials are available upon request.