Comparing the performance of time series models with or without meteorological factors in predicting incident pulmonary tubercul

Description of the PTB notification rate and meteorological factors

The annual PTB notification rates between 2005 and 2017 of Xuzhou, Nantong, and Wuxi was 56.41/100 000, 59.93/100 000, and 57.10/100 000, respectively. The range of annual notification rates for PTB was 31.54/100 000 to 78.96/100 000 in Xuzhou, 35.42/100 000 to 92.63/100 000 in Nantong, and 43.65/100 000 to 87.13/100 000 in Wuxi. The description of the monthly meteorological factors in the three cities between 2005 and 2017 is listed in Additional file 1: Table S1.

The ARIMA model

The monthly number of PTB cases showed a long-term downward trend and seasonal fluctuations, with a peak in March to April and a trough in December to January (in Xuzhou) or January to February (in Nantong and Wuxi) (Fig. 3). Therefore, we applied one ordinary difference and one seasonal difference to make the series stationary (d = D = 1). Then, we initially identified the parameters of the ARIMA model (p, q, P, and Q) to construct alternative models for each city according to the ACF and PACF plots of the stationary series (Additional file 1: Figure S1, a1–a3, and b1–b3). We determined the optimal ARIMA model to be ARIMA (1,1,1)(0,1,1)₁₂ for Xuzhou and ARIMA (0,1,1)(0,1,1)₁₂ for Nantong and Wuxi because (1) they had the smallest normalized BIC, (2) their residual series were demonstrated to be white noise, and (3) the parameters were all significant (P < 0.05) (Additional file 1: Table S2, c1–c3, and d1–d3 of Additional file 1: Figure S1). PTB cases in 2018 were predicted by the optimal ARIMA model and are listed in Table 1.

Table 1 The monthly number of pulmonary tuberculosis cases in the three cities in 2018 predicted by the ARIMA, ARIMAX, and RNN models

The ARIMAX model

The time series plots of the six meteorological factors in the three cities between 2005 and 2017 are shown in Additional file 1: Figure S2. The optimal ARIMA models for the MAT, MAP, MAS, MAH, MP and MST were ARIMA (0,0,0)(0,1,1)₁₂, ARIMA (0,0,0)(0,1,1)₁₂, ARIMA (0,1,1)(0,1,1)₁₂, ARIMA (1,0,0)(2,1,0)₁₂, ARIMA (0,0,0)(0,1,1)₁₂, and ARIMA (0,1,1)(0,1,1)₁₂ for Xuzhou, ARIMA (1,0,1)(0,1,1)₁₂, ARIMA (0,0,1)(0,1,1)₁₂, ARIMA (0,1,1)(1,1,0)₁₂, ARIMA (1,1,1)(1,1,0)₁₂, ARIMA (0,0,0)(0,1,1)₁₂, and ARIMA (1,0,1)(0,1,1)₁₂ for Nantong, and ARIMA (0,0,0)(2,1,0)₁₂, ARIMA (0,0,1)(0,1,1)₁₂, ARIMA (0,1,1)(0,1,1)₁₂, ARIMA (0,1,2)(0,1,1)₁₂, ARIMA (0,0,0)(0,1,1)₁₂, and ARIMA (1,1,1)(0,1,1)₁₂ for Wuxi, respectively. We then estimated the correlation between PTB and each meteorological factor at different lag times. The CCF plots showed that PTB was positively correlated with MAS (2-month lag), MAH (1-month lag) and MP (2-month lag) and negatively correlated with MST (1-month lag) in Xuzhou. PTB was positively correlated with MAT (0-month lag), MAP (1-month lag) and MAS (2-month lag) in Nantong and was positively correlated with MST (0-month lag) and negatively correlated with MAH (0-month lag) (P < 0.05) in Wuxi (Fig. 4). We incorporated different combinations of significant meteorological factors as external variables into the optimal ARIMA model to construct alternative ARIMAX models (Table 2). Finally, we determined the optimal ARIMAX model to be ARIMA (1,1,1)(0,1,1)₁₂ with MP (2-month lag) for Xuzhou, ARIMA (0,1,1)(0,1,1)₁₂ with MAP (1-month lag) for Nantong and ARIMA (0,1,1)(0,1,1)₁₂ with MAH (0-month lag) for Wuxi. PTB cases in 2018 were predicted by the optimal ARIMAX model and are listed in Table 1.

Table 2 Alternative ARIMAX models for the three cities

The RNN model

We compared the MAPE of each RNN model with different parameters using the testing set to identify the appropriate parameters. The RNN5 model had the smallest MAPE with the testing set in each city (Table 3). The number of PTB cases in the current month in Xuzhou was positively correlated with MAS one month prior (P < 0.01), with MAS two months prior (P < 0.01), and with MAS three months prior (P < 0.01) and negatively correlated with MST two months prior (P < 0.01), with MAT three months prior (P < 0.01), with MP three months prior (P < 0.05), and with MST three months prior (P < 0.01). The number of PTB cases in the current month in Nantong was negatively correlated with MAS one month prior (P < 0.05), MAH one month prior (P < 0.05), MAS two months prior (P < 0.01), MAH two months prior (P < 0.01), MAS three months prior (P < 0.01), and MAH three months prior (P < 0.05). The number of PTB cases in the current month in Wuxi was positively correlated with MAT one month prior (P < 0.01), MAS one month prior (P < 0.01), MST one month prior (P < 0.05), MAS two months prior (P < 0.01), and MAS three months prior (P < 0.01) and negatively correlated with MAP one month prior (P < 0.01), MAH one month prior (P < 0.05), MAT three months prior (P < 0.05), and MAH three months prior (P < 0.05) (Additional file 1: Table S3). Then, we constructed the RNN6-RNN9 models by incorporating significant meteorological factors into the RNN5 model. The detailed composition of the nine RNN models is listed in Additional file 1: Table S4. We determined the optimal RNN model to be RNN8 for Xuzhou and RNN7 for Nantong and Wuxi since they had the smallest MAPE with the testing set after three training cycles (Table 3). Additional file 1: Figure S3 shows the epoch-error plots of the optimal RNN models after three training cycles. The downward trend in the error of the models with the training set was no longer significant after reaching the set number of epochs, indicating that the training epochs were appropriate. Finally, we chose the RNN8 model after the first training in Xuzhou and the RNN7 model after the second training in Nantong and Wuxi (Table 3). PTB cases in 2018 were predicted by the optimal RNN model and are listed in Table 1.

Table 3 Alternative recurrent neural network models for the three cities

Evaluating the performance of three models

As shown in Table 4, the ARIMAX model is slightly superior to the ARIMA and RNN models in Xuzhou, significantly superior to the ARIMA and RNN models in Nantong, and slightly superior to the ARIMA and significantly superior to the RNN models in Wuxi. Generally, the ARIMAX model showed the best performance.

Table 4 Evaluation of the performance of the ARIMA, ARIMAX, and RNN models in predicting the monthly number of pulmonary tuberculosis cases in the three cities in 2018