Mobile Phone Syndromic Surveillance for Respiratory Conditions in an Emergency (COVID-19) Context in Colombia: Representative Survey Design

Introduction

Surveillance systems that provide real-time and accurate information are increasingly critical for decision-making in public health, as demonstrated by the COVID-19 pandemic [1]. Commonly used indicators for monitoring the COVID-19 pandemic were the number of laboratory-confirmed cases, the number of persons hospitalized or in intensive care, and the number of COVID-19 deaths [2,3]. However, these indicators are highly dependent on the health systems’ capacity, might fail to represent real-time conditions, are often mediated by other factors such as health care access, and are less likely to detect less severe cases [4]. These challenges become more important in low- and middle-income countries (LMICs) where traditional surveillance tools are scarce. For this reason, new strategies to complement traditional epidemiologic surveillance are needed [5]. The World Health Organization (WHO) has recommended enhancing traditional epidemiologic surveillance systems with other tools such as syndromic surveillance, where individuals can self-report symptoms related to infection [5], improving the timeliness and coverage, at lower costs [6-9].

Mobile phones have the potential to be an efficient tool to perform syndromic surveillance [1,4,10] because of their widespread use. For example, in Colombia, it has been estimated that there are 133 mobile telephone subscriptions for every 100 people [11]. In addition, it has been identified that surveys using mobile phone technology have lower costs than household surveys [9]. Finally, mobile phone syndromic surveillance is easy to deploy in emergency contexts where social distancing is required.

Mobile phone–based syndromic surveillance systems used during the COVID-19 pandemic have been mainly passive and limited to mobile phone apps that require smartphone technology linked to contact-tracing apps [1,4,10,12-14], which could be limited in LMIC where the number of smartphones with internet access is limited [10]. Other studies in LMIC have used the interactive voice response (IVR) as an active surveillance system on behavior, exposure, knowledge, and perception related to COVID-19, but not as a syndromic surveillance system [6,15,16]. To our knowledge, there have been no published experiences of an active syndromic surveillance system on COVID-19 disease or other health emergencies using IVR as a data collection tool. We hypothesize that syndromic surveillance is a feasible option to conduct syndromic surveillance in respiratory emergencies. This study aims to assess the feasibility of a national active syndromic surveillance system for COVID-19 disease in Colombia.

Methods

Data and Surveillance Instrument

IVR surveys were developed and cognitive testing was performed [17]. Participant phone numbers were obtained through random digit dialing [18], which included a prefix ranging from 300 to 323 (which are the prefixes for all mobile phone numbers used in Colombia), followed by 7 digits randomly selected. Random digit dialing is a technique used in previous research to yield a random sample of phone numbers when a previous database does not exist or cannot be accessed for privacy reasons [19]. Respondents who were aged ≥18 years and provided consent, were considered eligible and could participate in a 10-minute survey that comprised the following topical modules: sociodemographic status, respiratory symptoms, previous COVID-19 infection and vaccination status, preferences about COVID-19 vaccination and information on COVID-19. On survey completion, respondents were given an airtime incentive of COL $4000 (around US $1) [20].

Sampling

After testing, we deployed 2 pilots of IVR syndromic surveys with different levels of representativeness. The objective of the first pilot was to assess the feasibility of the IVR system to carry out syndromic surveys, as well as determine population profiles at the national level more likely to respond to this tool. The second pilot was aimed at determining whether quota sampling was effective in improving population representativeness for less populated regions. Following previous work [21-23], the 5 different regions were defined as follows: Caribbean (including Córdoba, Sucre, Bolívar, Magdalena, Atlántico, La Guajira, Cesar, and San Andrés y Providencia) with around 12 million inhabitants, Pacific region (including Chocó, Valle del Cauca, Cauca, and Nariño) with a population around 8.5 million, Amazonas river region (including Amazonas, Putumayo, Caquetá, Vaupés, Guaviare, and Guainía) with about 1.1 million population, Orinoco River region (including Meta, Vichada, Casanare, and Arauca) with 2 million population, and the Central region (including Bogotá, Cundinamarca, Huila, Tolima, Boyacá, Santander, Norte de Santander, Caldas, Risaralda, Antioquia, and Quindío) with around 29 million population [24].

The sample size for the first pilot was calculated using the STEPS (WHO Stepwise Approach to Noncommunicable Disease Risk Factor Surveillance) survey calculator as used in previous research [9,16,25,26]. The STEPS survey is a recommended calculation for repeated cross-sectional, population-based household surveys (see equation 1). The assumptions for the sample size are a 95% CI (z score=1.96), margin of error of 0.05, and baseline prevalence of 0.34 [27], yielding 345 complete surveys to be deployed to have nationally representative surveys.

The second pilot instead focused on producing values that are representative of 5 regions in Colombia through automated strata sampling [28]. To achieve this, we recalculated the sample size with the prevalence of symptoms obtained from the previous survey (0.06), yielding a total of 87 complete surveys for each of the 5 regions, keeping all other parameters the same.

The deployment of the surveys took place between March 12, 2022, and March 15, 2022, and April 23, 2022, and May 13, 2022, respectively. The second pilot took longer because of the quota sampling, where many calls were deemed ineligible if the quota for that region had been already filled.

Outcomes—Disposition Codes

Disposition codes were assessed using the following nomenclature. I stands for complete interview. The respondents were aged ≥18 years, consented, and answered all applicable questions. P stands for incomplete interview. The respondents were aged ≥18 years, consented, answered demographics questions, and some questions from the other modules, but did not finish the information on the COVID-19 module (did not reach the airtime incentive question). B stands for break-off the interview. The respondents were aged ≥18 years, consented, and answered the demographics questions. R stands for refuse after listening to the consent. U stands for unknown as these were phone numbers that did not answer or did answer but did not reach the consent and their eligibility status could not be ascertained. Respondents can be deemed ineligible for three reasons: (1) line not registered, (2) they might report being aged younger than 18 years, or (3) in pilot 2 they might be from a region whose quota sample was already met [29].

Survey rates are defined as follows: (1) contact rate measures the proportion of all cases in which some eligible mobile phone user was reached by the survey (I+P+B+R/I+P+B+R+U). (2) Response rate measures the proportion of both complete and incomplete interviews divided by all known eligible mobile phone users and users with unknown eligibility (I+P/I+P+B+R+U). (3) Refusal rate measures the proportion of mobile phone users that do not consent or break off the survey after consent divided by all known eligible users and users with unknown eligibility (B+R/I+P+B+R+U). (4) Cooperation rate: measures the proportion of complete interviews divided by all known eligible mobile phone users. (I/I+P+B)

An outline for the disposition codes can be seen in Figure 1.

‎

Outcomes—Demographic Characteristics

Demographic characteristics of the respondents were compared to the National Quality of Life Survey 2021 (NQLS). Importantly, we did not use statistical tests to compare the 2 databases given that the sources of data are different in structure (Table 1).

All statistical analyses were conducted using Stata (version 14; StataCorp LLC).

Ethical Considerations

This study was approved by the Institutional Review Board of the Johns Hopkins Bloomberg School of Public Health (17868), and the Ethics Committee of the Public Health Institute at Universidad Javeriana under filing number 4 of the session conducted on August 13, 2021.

Results

Disposition Codes and Survey Rates for Pilots 1 and 2

Pilot 1

In total, 55,000 phone calls were made for the first pilot. The contact rate was 1.64% (900 out of 55,000), the response rate was 0.73% (403 out of 55,000), and the cooperation rate was 80.48% (367 of 456; see Tables 2 and 3).

We met the sample size with 367 surveys for a nationally representative sample. The average age was 36.36 years (SD 14.06 years, range 18 to 78 years); the population with the highest participation in the sample were aged 18 to 29 (n=139, 38%) years and 30 to 39 (n=88, 24%) years, followed by 40 to 49 age group (n=70, 19%). More than half were women (n=209, 57%), had not completed a college degree (n=301, 82%), and lived in an urban area (n=268, 73%).

We found that 6% (22 out of 367) of respondents reported that someone in their household had had any COVID-19–related symptoms in the past 3 days. It was primarily the respondents who had any COVID-19–related symptoms (n=11, 52%). Likewise, 44% (n=17) of the respondents reported having had a confirmed or suspected case of COVID-19 in the past. Further, 90% (n=329) of respondents reported having received at least one dose of the COVID-19 vaccine, 54% (n=177) reported being fully vaccinated but without the booster, and 27% (n=88) stated being fully vaccinated with the booster. Of the 10% (38/367) who had not been vaccinated, they cited concerns about vaccine safety (n=10, 27%). Additionally, the top 3 means by which respondents learned about the COVID-19 vaccine were other networks or the internet (n=180, 49%), Facebook (n=77, 21%), and friends or relatives (n=43, 12%). Finally, most respondents reported that they clearly understood the information on COVID-19 and trusted the information they received (Table 4).

Pilot 2

In total, 588,891 phone calls were made for the second pilot. The contact rate was 1.16% (6777 out of 588,891), the response rate was 0.08% (474 out of 588,891), and the cooperation rate was 82.45% (451 out of 547; see Tables 2 and 3).

We met the sample size for the 5 regions in Colombia (Caribbean region: n=90, Pacific region: n=91, Amazon region: n=88, Orinoco River region: n=93, and Central region: n=89). In total, for the 5 regions, a sample of 451 responses was reached. All results are presented in Table 5.

For the 5 regions, we found that between 6% (5/90) to 12% (11/93) of respondents reported that someone in their household had had any COVID-19–related symptoms in the past 3 days. For the 5 regions, more than 50% (374/451) of the respondents reported that they did not have confirmed or suspected COVID-19 in the past. The COVID-19 vaccination rate varied across regions, from 78% (69/88) in Amazonas to 96% (86/90) in the Caribbean. Most regions reported close to 50% (221/451) full vaccination but without the booster. Of those who had not been vaccinated, the main reason cited was about vaccine safety in all regions. Additionally, the top 3 main sources of information about COVID-19 vaccination were other networks or the internet, Facebook, and friends or relatives (336/451, 74%; Table 6).

Comparison of NQLS With Mobile Phone Survey Samples

Pilot 1

We found that demographic variables have a similar distribution between the mobile phone surveys and NQLS. Even though respondents are more likely to be younger and female, differences do not seem to be meaningful (we are not making statements on significance because we cannot test the differences between points estimated in both surveys). Regarding geographical variables, we found that in both IVR and NQLS samples, a quarter of respondents stated living in rural areas. Importantly, IVR respondents tend to be more educated than the general population (Table 5).

Pilot 2

Overall, for the 5 regions, we found that demographic variables have a similar distribution to pilot 1 results. When comparing pilot 2 results to NQLS, we found that in regions with lower access to mobile phone technology, population groups are less likely to be represented in IVR surveys (older, less educated, and rural) than in regions with higher access to mobile phone technology. For example, the central region concentrates around 60% of the population of the country. In this region, the share of respondents reporting living in rural areas was 9 percentage points higher than its share of the population. Meanwhile, in the Orinoco and Amazon river regions (each with around 1.5% of the population), the share of respondents reporting living in urban areas was 3 percentage points higher than its share of the population.

Discussion

In this study, we conducted 2 different pilots to assess the feasibility of a national active syndromic surveillance system for COVID-19 in Colombia. In both pilots, we found that it is possible to deploy a rapid active syndromic surveillance system for respiratory disease at the national level in Colombia with fair contact rates (between 1% and 2%) and excellent cooperation rates (above 80%) when compared to other studies [6]. Evidence from Burkina Faso, a low-income country, has reported similar cooperation rates using IVR [30] in a nonemergency context. In another study, an IVR survey implemented during the COVID-19 pandemic reported refusal rates in Ecuador and Sri Lanka of 5.1% and 2%, respectively, whereas our refusal rate was close to 1% (6799/509,267). Another study conducted in 5 LMICs reported similar contact, response, cooperation, and refusal rates [8].

Sampling and operational characteristics allow public health officials to monitor changes in the prevalence of respiratory symptoms over time to detect and prepare for changes in respiratory disease prevalence. An additional benefit of this technology is that it does not need face-to-face interaction reducing exposure to survey personnel, can be deployed faster in challenging or low-density environments [26], and is generally less costly [9]. However, this system has 2 main limitations. On the demand side, it is clear that respondents are more likely to be younger, female, and more educated than the average population implying that they might be overrepresented, as it has been found in other research using IVR [16,31]. To reduce selection bias, measures must be taken to improve participation from less represented groups [28]. It is possible that over time and as younger cohorts grow, issues around digital literacy and access to cell phone numbers will subside.

On the supply side, it is also clear that connectivity and mobile phone availability might play a role in survey participation [31]. To reduce selection bias associated with this effect, the second pilot included quotas by region, revealing similar patterns to those found in the first pilot (more likely to find younger, female, and more educated respondents), but still providing closer demographic estimates to those found in external surveys. The strategy of using quotas can be further expanded to smaller areas of particular interest such as municipalities, if needed.

This study is limited mainly by the availability of government data to compare results, particularly related to COVID-19 and other cases of respiratory disease, and vaccination status by region. The available information about behavior and COVID-19 in Colombia is reported on the Johns Hopkins COVID Behaviors Dashboard, which is at the national level. For example, in March 2022, the Dashboard indicated that in Colombia, about 40% of the unvaccinated wanted to get a vaccine, and our data at the national level (pilot 1) found that that was about 44% (17/38). Likewise, the Dashboard reported an ever–COVID-19 prevalence of 43%, and our data found an ever-prevalence (confirmed or suspected) of 44% (161/367), proving similar estimates with very different data sources.

This evaluation shows that using IVR as a surveillance system may be useful and feasible for conducting syndromic surveillance amid a health emergency. This study informs new areas of expansion of this work: including developing quotas for demand-side variables that affect response rates including sex, age, and educational level as well as sentinel systems so people can report on the status of their households and neighborhoods to expand the network of surveillance.