Real-time Prediction of the Daily Incidence of COVID-19 in 215 Countries and Territories Using Machine Learning: Model Development and Validation

Introduction

COVID-19, a highly infectious disease with serious clinical manifestations, was first reported in China in late 2019 and spread to other countries within weeks [1,2]. It was announced as a public health emergency of international concern by the World Health Organization (WHO) on January 30, 2020, and declared a pandemic on March 11, 2020 [3]. As of August 21, 2020, COVID-19 has been reported in 215 countries and territories, with 22,876,009 confirmed cases and 797,289 deaths [4]. The severity of the pandemic is variable in individual countries and territories. In some regions, severe outbreaks, occurred and case numbers took a long time to decrease. In other regions, it took more time for the spread of the disease to affect people. Even in places where the spread has been brought under control, there have been second or third waves of outbreaks, with variable severities of morbidities and mortalities.

Prediction of the incidence of COVID-19 in individual countries and territories is extremely important to provide reference for governments, health care providers, and the general public to prepare management measures to combat the disease. In the early months of 2020, prediction was useful to inform the countries and territories at risk of outbreak to take action for prevention. Currently, although the COVID-19 outbreak has already occurred in almost all regions globally, prediction has merits in monitoring the severity of spreading and recovery and assessing the likelihood of a secondary or tertiary epidemic. COVID-19 may become a seasonal or persistent epidemic in future years, like influenza. Thus, predicting and monitoring COVID-19 activity is needed at present and in the future. As a typical example, Google search data can predict the incidence of influenza effectively [5].

There are reported studies on the prediction of the incidence of COVID-19. A susceptible-exposed-infectious-recovered metapopulation model was used to simulate the epidemic in major cities in China [1]. Our previous study showed that data from internet search engines and social media platforms could predict the incidence of COVID-19 in China [6]. Later, data mining and machine learning techniques were used to predict the incidence of COVID-19 using data from the previous incidence or internet search volumes [7,8]. In these studies, only a few countries were investigated or the world was considered as one region. Furthermore, as the pandemic continues, increasing amounts of data are available for modeling and calibration. The actual situation may be different from previous predictions. For example, Tuli et al [9] predicted that the total number of new cases would reach 97% of the total expected cases on August 14 in the United States; however, actually, there were still tens of thousands of new cases daily. Therefore, an up-to-date and evolving prediction method is needed.

In this study, we aim to develop an efficient and novel methodology for real-time prediction of COVID-19 activity based on the previous daily incidence of COVID-19 and infoveillance data (search volume data via Google Trends) in all individual countries and territories worldwide.

Methods

Data Source

Two sets of data were collected. The first data set was the search volume data obtained from the Google Trends service. We collected the Google search volumes of 28 candidate features related to COVID-19 from January 1 to July 26, 2020, in 215 countries and territories. We used 14 terms, including coronavirus, pneumonia, and Covid-19; 6 symptom-related terms [10] (cough, diarrhea, fatigue, fever, nasal congestion, and rhinorrhea); and 5 prevention-related terms (hand washing, hand sanitizer, mask, social distance, and social isolation. There were two main reasons for selecting these 14 terms for Google Trends data collection: (1) Previous studies [6,10-13] have shown that the internet search data for these terms were correlated with the incidence of COVID-19. (2) These 14 terms are related to COVID-19 in different aspects, such as topics, symptoms, and prevention; therefore, they can reflect its incidence. Two types of Google Trends data were retrieved. The first type was the data of interest over time, defined as the search interest relative to the highest point for the specific term, region, and time interval. Values were calculated on a scale from 0 to 100, where 100 is the peak popularity for the time period. The second type of data, data of interest by region, was retrieved by setting the region to “worldwide” for the given term and time. The values were calculated on a scale from 0 to 100, where 100 is the location with the most popularity as a fraction of total searches in that location. It is notable that we included low search volume regions to obtain data for more regions.

The second data set was the daily number of new COVID-19 cases from January 10 to August 16, 2020, in 215 countries and territories, obtained from the WHO website [14]. For consistency, the values of the daily number of new COVID-19 cases in the 215 countries and territories were normalized to a scale from 0 to 100. An example showing the search volumes of the terms coronavirus and Covid-19 and the daily incidence of COVID-19 in the United Kingdom is shown in Figure 1.

‎View this figure

Correlation Analysis and Feature Engineering

We conducted two types of correlation analyses in each country or territory to find the right combination of input features. The first analysis was the Spearman correlation between the Google volume data of each feature and daily new confirmed cases with n days of lag (n=7, 14, 21, 28). We obtained the average and maximum Spearman correlation coefficients for different lag days between each Google volume data feature and daily newly confirmed cases in the 215 countries and territories (Multimedia Appendix 1). Each Google volume data feature was related to the daily new confirmed cases in the 215 countries and territories. The top related item was the interest over time in coronavirus. We found a better correlation with the 21-day lag than with the other days of lag. Therefore, we combined 28 Google volume data features with daily new confirmed cases of 21-day lag and formed 29 input features.

We calculated the Spearman correlation among 28 independent Google Trends volume data features in the 215 countries and territories to assess the independence of the features. We obtained the average results of the Spearman correlation coefficients between the top 10 cross-correlation Google volume data features in the 215 countries and territories. The highest correlation coefficient, between coronavirus interest over time and interest by region, was 0.59, which is lower than the criterion of strong correlation (0.8). Additionally, the cross-correlation between the remaining 18 Google volume data features was less than 0.1. Therefore, the correlation analysis results showed that the 28 Google volume data features were independent.

Cluster Analysis

The internet search patterns and the management of COVID-19 vary among countries and territories. We used a hierarchical clustering technique to cluster the 215 countries and territories into several groups, and we built a model for each group. Two types of data were used for clustering. One type was the related metrics of the importance scores of 29 input features resulting from the random forest regression algorithm in 215 countries and territories. The other type was the correlation of the development trend of daily new confirmed cases in the 215 countries and territories. Finally, the 215 countries and territories were clustered into 8 groups (Multimedia Appendix 2). There were 150 countries and territories in the first group, 38 in the second group, 11 in the third group, 7 in the fourth group, 6 in the fifth group, and 1 country or territory each in the sixth, seventh, and eighth groups.

In addition, to explore the internal relationships between the countries and territories in each cluster, two types of Spearman correlation coefficients were calculated. The first was the Spearman correlation between the daily new confirmed cases among the countries in the first five groups, which contain more than one country or territory. The second was the Spearman correlation between the search volume of the term coronavirus among the countries in the first five groups. These two correlations were also calculated in the sixth, seventh, and eighth groups with a single country or territory. The average Spearman correlation coefficients are shown in Table 1. There were two findings. The first was a correlation between the incidence of COVID-19 among the countries and territories in each group, especially in the first group, which contains 150 countries and territories. The second was that the correlation of Google search data among countries and territories in the first five groups was high. These findings also showed the reliability of our clustering method.

Modeling and Evaluation

We built a model for each group of countries/territories separately according to the clustering results and produced a 7-day-ahead incidence prediction of COVID-19 for the 215 countries and territories. The random forest regression algorithm has many decision trees; therefore, it has good robustness and a strong ability to resist overfitting. In addition, the random forest regression (RFR) algorithm gave the importance score of the features, which was helpful for feature selection. Therefore, the RFR algorithm was used to forecast the daily new confirmed cases of COVID-19 over the time series data set, which consisted of Google Trends data and the incidence of COVID-19 with 21-day lag provided by the WHO. Python 3.7.6 was used for the modeling and evaluation.

To quantitatively evaluate the performance of the models, we calculated four metrics: the mean absolute error (MAE), root mean square error (RMSE), Pearson correlation coefficient, and Spearman correlation coefficient.

where p_i, t_i, and M represent the predicted value, true value, and total number of days; X and Y are two independent time series; and σ_X and σ_Y are the standard deviations of X and Y, respectively. x_i and y_i are the ith value of X and Y and and are the mean values of X and Y, respectively, and n is the length of the time series. The P values of the Pearson correlations were calculated to assess the statistical significance.

Optimization and Validation

Several experiments were conducted to optimize and validate the results. First, to evaluate the role of each input feature in the prediction, we calculated the importance score of each input feature. To validate the features included in this study, a series of ablation studies, in which the top n features (n=1, 2, ..., 29) were used as inputs in turn, were conducted on different groups of countries/territories according to the clustering results. The Pearson correlation coefficient was averaged for the 8 groups of countries/territories based on the sum of the input features (Figure 2). According to the results, the top 13, 10, 10, 25, 2, 5, 1, and 3 features were used for the respective 8 groups in further experiments. The models using previous daily incidence with and without Google Trends data as input features were compared (Table 2). Adding Google search volume features as input improved the MAE, RMSE, Pearson correlation, and Spearman correlation by 0.28, 0.88, 0.06, and 0.08, respectively, in terms of average results in 215 countries and territories. These results confirmed that the Google Trends data played a positive role in predicting the incidence of COVID-19.

Second, to reduce the influence of random noise, we used multiple-step forecasting (5-step, 6-step, 7-step, 8-step, 9-step, and 10-step), which used the data over the past n days (n=5, 6, 7, 8, 9, 10) to predict the daily new confirmed cases of COVID-19. The average quantitative results of the different steps of the second period in the 215 countries and territories are shown in Table 3. The performance of the 7-step prediction was slightly better than that obtained using other numbers of steps in terms of 4 of 5 quantitative metrics in our study. Therefore, we selected the 7-step prediction.

Third, the proposed RFR algorithm was compared with two other mainstream algorithms. One was decision tree regression (DTR), which is a traditional machine learning algorithm. The other was long short-term memory (LSTM), which is a deep learning algorithm. Especially, LSTM, an artificial recurrent neural network, was used as a 3-layer model in our study. TensorFlow and Keras were used as frameworks for training the LSTM models. Table 4 shows the average quantitative results of the different methods for the 215 countries and territories. The random forest regression algorithm achieved better performance in terms of all quantitative metrics compared to DTR and LSTM.

Fourth, we trained the models with as much data as possible. Therefore, we updated the models and repeated the experiments at regular intervals. The effectiveness of the models at 3 time periods was compared.

Fifth, we predicted the incidence of COVID-19 2 days and 7 days ahead. Table 5 shows the average quantitative results of different time windows of the second period in the 215 countries and territories. We found the predictive power of the 7-day window to be less accurate than that of the 2-day window. It was noted that the 2-day window prediction may not be sufficiently informative for policy making, and the 7-day window would be more useful to enable a responsible authority to respond proactively and impose interventions in time. Therefore, we used the 7-day window for the prediction of the incidence of COVID-19.

‎View this figure

Results

We produced 7-day-ahead and real-time COVID-19 forecasts for 215 countries and territories. Our final models performed well in predicting the daily new confirmed cases of COVID-19 in most of the countries and territories examined. Of the 215 countries and territories, 198 (92.1%) had MAEs <10 (Figure 3a), and 187 (87.0%) had Pearson correlation coefficients >0.8 (Figure 3b). In all 215 countries/territories, the mean MAE was 5.42 (range 0.26-15.32), the mean RMSE was 9.27 (range 1.81-24.40), the mean Pearson correlation coefficient 0.89 (range 0.08-0.99) and mean Spearman correlation coefficient 0.84 (range 0.21-1.00), with P<.001 in most of the countries and territories. Detailed results of the individual countries and territories are listed in Multimedia Appendix 3. The average results for each group are shown in Table 6. Generally, the models performed well in the first, second, third, and fifth groups, which contained 150, 38, 11, and 6 countries and territories, respectively. The Pearson correlation coefficients were >0.80 in most of these countries and territories. In contrast, the performance was poorer in groups 4, 6, 7, and 8. The Pearson correlation coefficients were <0.80 in most of these countries.

The performance of the models in 3 different time periods is shown in Table 7. As the training data increased, the overall prediction performance of the model improved. In addition, we randomly selected 4 countries from the 150 countries in the first group for further analysis. The predicted and actual daily new confirmed cases of the 3 different time periods in these 4 countries are shown in Figure 4. For example, our models can predict the increase in daily new confirmed cases in China from July 18 to July 31 and the decrease of daily new confirmed cases in China from August 3 to August 16. Our models were able to predict the daily new confirmed case numbers consistently.

‎View this figure

‎View this figure

Discussion

Principal Results

In this study, we have established a method to obtain a 7-day-ahead prediction of COVID-19 activity by combining the previous incidence of COVID-19 and Google search volume data at the country or territory level in the real world. The models performed well in most countries. In a total of 215 countries and territories, the mean MAE, RMSE, Pearson correlation and Spearman correlation values were 5.42, 9.27, 0.89, and 0.878, respectively. The P value was <.001 in most countries and territories, which shows the significance of the Pearson correlation coefficient. The results show that the longer the training time and the greater the amount of available data, the higher the prediction accuracy.

Comparison With Prior Work

Our study has several advantages compared to other reported studies on similar topics. First, we investigated 215 individual countries and territories. Other studies only investigated a single country [15] or several countries [16], or they considered the world as one whole region [11,17]. Our method performed well in most of the countries and territories. Second, we used clustering methods to pool data with similar patterns together. The advantage of large amounts of data improved the performance of the models. Our results show that the groups with a large number of countries and territories had better results compared to those with a limited number of countries. There has been a report on clustering of provinces in China [12], but not clustering of countries. In another study, countries were grouped into 4 clusters; however, this clustering was not used for prediction [18]. Third, we combined data regarding both previous incidence of COVID-19 and Google Trends search volumes for prediction. Both of these data sources have been used individually in forecasting COVID-19 incidence. Our results showed that adding the Google Trends data improved the performance of the models developed from the previous incidence data. Fourth, we used machine learning methods for modeling. Many previous studies [13,19,20] indicated that Google Trends data are correlated with the incidence of COVID-19. Machine learning has been used to manage some large data sets and develop models to predict the incidence of COVID-19 in only a few countries [20]. Our models can produce useful and reliable predictions in most countries. Some studies used epidemical models such as susceptible-infected-recovered-dead (SIRD) for prediction. However, the spread of COVID-19 is affected by many factors, including public health management, traveling, and other social activities, which are not taken into account in the SIRD model. In Iran, COVID-19 was expected to subside at the beginning of May 2020 according to the SIRD model. However, at the time of writing this manuscript, thousands of new cases were still being reported daily in Iran [21]. Fifth, our models used the most up-to-date data and were updated three times. As more data become available, the performance of the models can be further improved. Some models developed several months ago may have severe errors. For example, Tuli et al [9] predicted that the total number of new cases would reach 97% of the total expected cases on August 14, 2020, in the United States [9]; however, tens of thousands of new cases were reported daily in July and August.

The high accuracy of our models enables real-time forecasting of the short-term trends of COVID-19, not only during the outbreak but also during the recovery period and subsequent second or third epidemic in individual countries. The methods can also be adapted to predict the incidence of subregions. Real-time digital surveillance of COVID-19 is provided, which would save time and resources in data collection. As the COVID-19 pandemic is changing rapidly, digital health systems should provide an effective solution to address the challenges to public health and consequential socioeconomical complications with high efficiency. The epidemic of COVID-19 is likely to persist in the future, and it may even become a seasonal infectious disease like influenza. Therefore, accurate surveillance and prediction of its activity would help governments, health care providers and the general population to take appropriate actions to compact the disruptive effects of COVID-19 [5].

Limitations

We recognize some limitations of the current study. First, the models did not predict well in some countries, which were clustered into independent groups and had limited periods of data for training. Second, the occurrence of COVID-19 in some countries would have increased the awareness of other countries, especially those with a close relationship, consequently resulting in a large Google search volume for terms related to COVID-19. Third, there is a limitation in applying our models in countries and territories where Google is not the mainstream search engine. Thus, predictions of COVID-19 incidence in regions with limited internet access or prohibited Google access may not be accurate. Fourth, COVID-19 and influenza share some common symptoms and even prevention methods. Therefore, the Google Trends data of these terms may not be able to differentiate COVID-19 and influenza. Further studies are needed to develop an algorithm to differentiate these two infectious diseases using Google Trends.

Conclusions

In this study, we integrated the Google Trends data and previous daily incidence of 215 individual countries and territories using the techniques of features engineering, country clustering, and machine learning. We are able to achieve prediction 7 days ahead of time of the daily incidence of COVID-19 in real time in most countries and territories.