Performance of a Full-Coverage Cervical Cancer Screening Program Using on an Artificial Intelligence– and Cloud-Based Diagnostic System: Observational Study of an Ultralarge Population

Introduction

Cervical cancer can be largely prevented through early intervention [1]. However, it remains the fourth leading cause of cancer-related deaths in women worldwide. The Global Cancer Statistics for 2020 reported approximately 600,000 new cases of and 340,000 deaths from cervical cancer [2]. In response to this significant health challenge, the World Health Organization (WHO) launched the Global Strategy for the Elimination of Cervical Cancer in 2020, highlighting 3 key interventions: human papillomavirus (HPV) vaccination, cervical cancer screening, and cervical disease treatment [3]. The target for coverage of cervical cancer screening in specific populations is to reach 70% and above. In 2021, WHO updated its cervical cancer screening guidelines, providing recommendations to different countries and regions [4]. However, many countries with the highest burden of disease face limitations in achieving sufficient screening coverage [5]. In low- and middle-income countries (LMICs), only 9%-11% of women have been screened, in contrast to the 84% screening rate in high-income countries [6].

The national cervical cancer screening program implemented by the Chinese government in 2009 has played a crucial role in cervical cancer prevention and control (Multimedia Appendix 1). However, there still exists a considerable gap between the current cervical cancer screening rate and the target set by the WHO [7,8]. The economic burden of cervical cancer disease accounts for 18.2% of the global burden [2], and its incidence and mortality rates are 11.4% and 3.4%, respectively [9], with both being the highest in the central region, including Hubei province. The screening rate for Chinese women aged 35-64 years is approximately 36.8%, with lower rates observed in rural areas compared to urban areas [10]. Affordability and service accessibility are still the key factors influencing Chinese women’s use of cervical cancer screening services, followed by education, health insurance, occupation, and family income [11]. Organized free screening can significantly increase their willingness to be screened. While China has already screened 130 million individuals, there are still 300 million people to be screened. Screening technology is a critical factor affecting the increase of screening coverage. Currently, cytology screening is the primary method used in China’s national screening program. However, the shortage of pathologists in primary health institutions has limited the widespread adoption of cytology screening [12]. Studies have also indicated higher cervical cancer mortality rates in rural areas compared to urban areas [9,13]. Therefore, the selection of appropriate technology to rapidly enhance screening coverage, particularly in regions with limited health resources, is an urgent issue in China today.

Artificial intelligence (AI) technology has the potential to address the issue of insufficient pathologists in primary health institutions and accelerate the improvement of screening coverage rates. Therefore, this technology has been implemented in the Hubei government’s organizational screening for 5 years. Now, Hubei province has become the first in China to implement full-coverage of cervical cancer screening for women of appropriate age by adopting AI technology, namely completing cervical cancer screening for 12.67 million urban and rural women in 3 years [14]. AI technology has a wide range of applications in cancer screening with favorable diagnostic performance [15-19]. A few studies conducted on Chinese populations have explored the use of AI technology in cervical cancer screening and diagnosis [11,20,21], comparing its compliance to manual reading, identifying high-grade lesions, and developing AI algorithms [22-24]. Further, 1 recent study compared AI technology with traditional cytology screening and HPV testing, demonstrating comparable cost-effectiveness of AI technology versus HPV testing [25]. Another study focused on the application of AI technology in population screening, including the roles and costs of health care workers, as well as measures to overcome challenges in online screening [26]. Although previous studies have reported results of internal quality control, such as compliance rates between AI and manual groups, evidence of external quality control is lacking. Furthermore, the screening efficiency of AI for ultralarge populations, the speed of improving coverage, the quality of screening, and the responsibilities and costs of the different screening organizations involved are not known. The performance of HPV testing for screening in the population has been reported [27-29], but the performance of AI technology in real-world screening is lacking and there is an urgent need to understand the performance of AI technology in full-coverage screening.

We described the governing and implementing agencies of the full-coverage screening program and their responsibilities, as well as how AI technology made the program a reality. More importantly, this study aimed to evaluate the performance of an AI and cloud diagnostic system in the first full-coverage cervical cancer screening program in China. The evaluation encompassed 4 key dimensions: accessibility of screening services, screening efficiency, diagnostic quality, and cost of different institutions. The assessment provided real-world evidence of the adoption of AI technology to improve screening coverage for other provinces in China and LMICs.

Methods

Full-Coverage Screening Program

The full-coverage screening program consists of 2 interconnected systems, a management system led by the provincial health committee and an implementation system led by the maternal and child health hospitals at the county and district levels (Figure 1). These 2 systems work collaboratively to ensure the successful implementation and coordination of the screening program. The health commission is responsible for program development, implementation guidance, project quality control, and evaluation. The cervical cancer screening implementation body consists of primary screening institutions, sampling institutions, and third-party testing institutions. In the implementation of the screening program, the key technology that makes the full-coverage plan a reality is the application of the AI and cloud diagnostic platform.

‎

AI and Cloud Diagnostic System

Principles of AI-Assisted Liquid-Based Cytology

The DNA content of cervical epithelial exfoliated cells is correlated with cervical carcinogenesis. In the AI-assisted diagnostic process, images of cell samples are captured, and the cell nuclei within these images are segmented. For accurate diagnosis, normal cells’ nuclei within a single sample group are the reference in the AI’s algorithm. The segmented nuclei are then quantified based on their cellular DNA content, which ranges from 0 to 255 pixels. Then, by analyzing the pixels, the AI obtains the information on the cell nucleus DNA, and finally, the parameters such as the DNA content of the cell nucleus in this sample are counted to figure out the type of cell lesion. The specific algorithms involved in AI diagnosis are reported in previous studies [24].

Screening Procedure

Step 1: Sample Acquisition

Women can access free cervical cancer screening services at primary health institutions by presenting their ID cards (Figure 2). At the same time, they are required to sign an informed consent form. Labels with QR codes are automatically produced through identification of the ID card. The staff of the primary health institutions uses a special cervical brush to collect cervical exfoliated cells, which are preserved in sample preservation liquids. Cell samples are transported to the Landing laboratory, a government-appointed testing facility. The information of the whole process from sample collection to diagnostic report is available through a QR code.

‎

Step 2: AI-Assisted Liquid-Based Cytology Diagnosis

Laboratory technicians make slides of the samples. The slides are scanned and the cell images are uploaded to the cloud platform. Trained and validated algorithms within the AI diagnostic engine analyze the cellular features present in the images. The results of the AI diagnosis are reviewed by cytotechnicians who identify negative cases and retain suspicious or positive cases for further evaluation.

Step 3: Quality Control

To ensure accuracy, all suspicious or positive cases, as well as a randomly selected 10% of negative cases, are mixed. The pathologist then diagnoses these cases and diagnostic reports will be issued. The entire process from AI reading to cytotechnicians review and pathologist review belongs to the internal quality control of our AI-assisted diagnostic system. Importantly, pathologists have the flexibility to work from any location, including their homes or even abroad, as long as they can connect to the cloud platform using their electronic devices.

Step 4: Results Notification

Diagnostic reports with negative screening results are sent to the primary health institutions in a centralized file bag. For positive screening results, the diagnostic reports are sent to the primary health institutions in a separate packet. Facility staff are responsible for notifying examinees. They can also obtain the information by scanning the receipt with a QR code through WeChat, which is the fastest way.

Step 5: Follow-Up Management

Based on the results of cytological screening feedback, primary health institutions follow up on examinees with abnormal initial screening results. Those with normal cytology results will be screened once every 3 years. If the cytology results are abnormal or suspicious, colposcopy is performed directly. If the colposcopy is negative, then a cytology reexamination is needed after 1 year. If colposcopy is positive, a histopathologic biopsy is performed and follow-up treatment is provided for those patients who require therapy.

Quality Control Process

Quality control consists of internal quality control and external quality control (Figure 2). Internal quality control (for details see step 3) is completed by 4 pathologists with senior titles from the landing laboratory, and the frequency of rechecking is daily. Besides, the internal diagnostic quality control is performed using a fully digital process, involving quality control in making slides, staining, and diagnosis. The external quality control is to invite pathologists from tertiary care hospitals to randomly select slides for a recheck regularly, including to recheck the results of cytology diagnosis and pathology diagnosis. That is, external quality control is a reexamination of internal quality control. Quality control adheres to the “Quality Assurance and Quality Control Guidelines for Cervical Cancer Screening” and the “Cervical Cancer Screening Quality Assessment Manual (2022 Edition).”

Implementation Stage

The application of the AI and cloud diagnostic system has progressed through 3 stages.

Phase 1: Pilot Phase

This phase was mainly used in some municipal hospitals to improve the efficiency of institutional diagnosis and save the expenditure on institutional labor costs.

Phase 2: Provincial Organizational Screening Phase

In 2017, it was first applied in the cervical screening program organized by the provincial government, with a cumulative screening quantity of 1.46 million completed in the first 3 years.

Phase 3: Full-Coverage Program Phase

AI-assisted diagnostics was applied in the provincial government’s full-coverage screening program. The technology was also being expanded in some countries in the Belt and Road Initiative, such as Pakistan.

Data Collection

We collected data on screening quantity, screening time, diagnostic quality, and cost. The screening data were collected from the initiation of the full-coverage screening program until January 31, 2023, with a total of 1,704,461 individuals and 18,261 histopathology biopsies. The screening program followed a closed-loop management process, encompassing screening, follow-up, and treatment for all examinees. We clarified the time and workload of AI-assisted diagnosis through the Wuhan landing cloud intelligence laboratory. Additionally, the daily workload of pathologists was estimated based on guidelines and literature studies, such as the Guidelines for Construction and Management of Pathology Departments (Trial) and literature studies. To evaluate the diagnostic quality, an external quality assessment was organized. The sample size was 220, including 170 randomly selected cytology samples and 50 histopathology samples. Data on the cost of full-coverage screening were obtained from experts of the Health Commission of Hubei Province.

Ethical Considerations

This study did not expose women’s privacy or identifying information, nor did it result in associated economic or social risks. Women participated in cervical cancer screening voluntarily and signed an informed consent form before being screened. The consent included that their screening information could be used for big data research. This study was conducted in compliance with the Declaration of Helsinki principles and approved by the ethics committee of Tongji Medical College, Huazhong University of Science and Technology (number [2023]IEC(A235)).

Results

Accessibility

Cervical cancer screening services were extended to all administrative districts, including 13 prefecture-level administrative regions and 4 directly administered county-level administrative regions in Hubei Province, and a total of 1,704,461 women had been screened (Table 1). The number of county administrative districts covered by the screening program has increased from 83 in 2020 to 103 at present, of which 39 are in urban areas and 64 are in rural areas. The completed screening coverage rate was 13.45% (Multimedia Appendix 2). Among them, Huanggang city had the highest coverage rate of 18.91%.

The screening project was officially launched in September 2022, with 1.54 million individuals screened in 5 months. Before the launch of the project, there were screening pilots in some areas in July (n=155,103) and August (n=9417). The amount of screening was limited, except for Wuhan, where a total of 153,747 individuals were screened in July. A screening peak was rapidly formed after the launch, including a screening volume of about 498,909 in November. The COVID-19 outbreak affected accessibility to screening services, most significantly in December (n=295,199) and January (n=87,760).

The highest active participation after the launch of this project was among rural women with 67.54% (n=1,147,839) of the total screening population. This was followed by urban nonunion women with 27.91% (n=474,420; Table 2). In total, 160,376 (94.07%) of these screenings were normal. Further, 199,473 (5.85%) were cytologically positive, indicating a need for referral for colposcopy. The positive rate of AI-assisted cytological screening was 5.84%. In the pathological examination performed in the project, 1863 individuals were examined for precancerous lesions and 93 individuals were examined for cervical cancer.

Screening Efficiency

Based on real-world screening volumes, full-coverage programs could be achieved in approximately 1 year by using AI technology (Table 3). For the number of pathologists involved in full coverage, AI technology was 87.5 (35,000/400) times more efficient than manual reading of slides. Based on the screening capacity of primary health institutions [12], it would take approximately 16 years to complete full coverage of the program’s screening population. This would be even longer if repeat screening every 3 years is taken into account.

Diagnostic Quality

A total of 220 individuals were reassessed for quality, including 170 cytology slides and 50 histopathology slides (Multimedia Appendix 3). According to the criteria in the cervical cancer quality assessment handbook [30], the specimen satisfaction rate, the compliance rate of negative and positive smear rechecks, and the recheck rate of pathological examination results all met the quality requirements (Table 4). The AI-assisted diagnosis had a 99% compliance rate for positive smears and a 100% compliance rate for negative smears when compared to the cytologist’s diagnosis. All negative cases and CIN2+ (cervical intraepithelial neoplasia) cases were exact matches, with a small number of deviations mainly in low-grade lesions. Compared to pathologists, AI diagnostic systems were less likely to miss or underestimate the severity of cervical lesions.

Cost

The per capita cost of screening in the full-coverage program is CN ¥49 (US $6.89; Table 5), while the price of health services in primary health institutions is about CN ¥160 (US $22.48). The primary health institutions and third-party testing institutions would receive CN ¥19 (US $2.67) and CN ¥27 (US $3.79) per case, respectively. The primary health institutions are responsible for only cytology sampling, colposcopy examination, and histopathology sampling throughout the screening process. Ultimately the primary health institutions can obtain the cost of services 2, 6, and 7 in Table 5. Third-party testing laboratories provide consumables and bear the costs of sample transportation, subsequent diagnostics, management of information systems, etc. The staff of the Women’s Union were involved in publicity and advocacy, so they received CN ¥3 (US $0.42) per case.

Discussion

Principal Findings

The diagnostic system based on an AI and cloud platform can achieve coverage of the whole population for cervical cancer screening, especially in areas with insufficient health resources. Cervical cancer screening services were extended to all administrative districts, the number of which has increased from 83 in 2020 to 103 at present. With the voluntary participation of women, a total of 170,461 individuals were screened between July 2022 and January 31, 2023, achieving a coverage rate of 13.45%. Rural women accounted for the largest proportion of screenings at 67.54% (1,147,839/1,699,591). In the screening program, 1,603,376 (94.07%) were diagnosed as normal, 1612 (1612/1,704,461, 0.09%) samples failed, with 1863 (1863/18,261, 10.20%) cases of precancerous lesions and 93 (93/18,261, 0.51%) cases of cervical cancer. The positive rate of AI-assisted cytological screening was 5.84%. Based on the maximum screening quantity of AI-assisted LBC in the real-world, full-coverage programs could be achieved in approximately 1 year, which was 87.5 times more efficient than manual reading of slides. Compared to real-world screening capacity, AI technology could accomplish full-coverage screening amounts at least 15 years ahead of schedule. The review compliance rates of positive and negative smears and pathological biopsies were 96% and above, meeting national quality control requirements. From a public payment perspective, the cost of this program was CN ¥49 (US $6.89) per person, with the primary screening institution and the third-party testing institute receiving CN ¥19 (US $2.67) and CN ¥27 (US $3.79), respectively. In short, AI-assisted diagnosis has proven to be accessible, efficient, reliable, and cost-effective, and plays an important role in facilitating improvements in screening coverage, especially in areas with insufficient health resources. To our knowledge, this study reports the largest sample of AI-assisted cervical cancer screening and is the first to evaluate the performance of AI technology in a full cervical cancer screening program. AI technology plays an important role in promoting screening coverage and opening up new pathways for the elimination of cervical cancer in China and other LMICs.

AI technology has greatly enhanced the accessibility of screening services, especially for rural women. Improving the equity and accessibility of screening services is an important issue for the elimination of cervical cancer [31]. China mainly adopts cytology screening, and the lack of pathologists limits the large-scale application of this technique, especially in remote or rural areas. The full-coverage screening covered all county-level administrative regions in China, with the largest number of rural women screened at 1,147,839 (67.54%), far more than urban women. This is on the one hand because screening services do increase the accessibility of screening services for rural women, and on the other hand, because of the long-term implementation of the screening program in rural areas, rural women have a high level of recognition and willingness to participate in the program. Previous studies have also shown that tissue-based screening can increase participation in screening [32,33]. In addition, according to the geographical characteristics of the county, the implementers adopted the approach of going to the village for screening or organizing women to be screened at the township health center for remote areas to ensure the convenience and accessibility of screening services. AI technology can not only solve the screening problem in remote or poor areas of China but also solve the cervical cancer screening problem in Pakistan by establishing laboratories and exporting equipment abroad. This technology will benefit more Belt and Road Initiative countries in the future.

AI-assisted LBC has greatly enhanced screening efficiency and accelerated the achievement of WHO’s screening goals. With a real-world maximum daily diagnostic workload of 35,000 cases, the AI system will achieve full cervical cancer screening coverage in less than a year, 2 years earlier than expected. Further, the daily diagnostic volume can be continuously increased, which can be achieved by adding high-throughput scanners and technical staff. This screening was completed in about 6 months with a coverage of 13.45% (1,704,461/12,672,278) of women in Hubei, which was huge progress. A survey of the screening capacity of primary health institutions found that some women may not be able to access screening services once in their lifetime [12]. In contrast, AI screening efficiency in the real world was approximately 16 times more efficient than the screening efficiency of primary health institutions. The coverage of screening has a significant impact on cervical cancer incidence and mortality [34-36], so the high efficiency of AI technology also accelerates the process of achieving the goal of cervical cancer elimination.

Diagnostic quality is an important evaluation indicator, including the consistency between AI and manual reading, the failure rate, and the positive cytology screening rate. The national cervical cancer cytology screening positivity rate is 4%, while HPV testing is as high as 10.1% [28]. The cytology screening misdiagnosis rate was lower than that of HPV testing and the leakage rate was higher than that of HPV testing. The positive rate of AI-assisted cytological screening was 5.84%, which was higher than traditional manual reading. This may be attributed to the improved diagnostic performance of AI-assisted cytological screening.

The total number of samples screened this time was about 1.7 million, with only 0.9% (1612/1,704,461) failure samples. Compared to the 3.3% [23] and 2.91% [26] in the previous study, the full-coverage program demonstrated better sample pass rates with the largest sample sizes. This may be due to the continuous training of technicians and strict quality control during the implementation of the full-coverage program. In a previous study with a sample of about 0.7 million, the overall agreement rate between AI and manual reading was 94.7% [23], which belonged to internal quality control. In the census population, AI was significantly more sensitive than manual reading for CIN2+/CIN3+ respectively, while the corresponding specificity was slightly reduced when compared to cytotechnicians. In suspicious populations, AI-assisted reading has similar sensitivity in detecting CIN2+/CIN3+, but is more specific than manual reading [22]. Our study adds another aspect regarding the rate of agreement, namely the consistency of external quality assessments. External quality control was a reevaluation of the outcomes of internal quality control. Additionally, the results of the external quality evaluation showed that the compliance rates for cytology screening and histopathology were 99%-100% and 96%, respectively. This all met the requirements for quality control of cervical screening in China and the WHO.

AI-assisted LBC has a favorable price advantage to support large-scale screening. Price is a very important consideration for the government to purchase this screening service and provide it free to women, especially for the population group. The per capita cost of this full-coverage screening program is CN ¥49 (US $6.89). In comparison, hospitals may charge almost 3 to 4 times (about CN ¥160 [US $22.48]) as much. CN ¥3 (US $0.42) of the project cost is for advocacy and mobilization. However, the cost of the project will continue to decrease over time with the widespread use of social media and the increased willingness to participate. Previous studies have also confirmed that AI-assisted diagnosis is more economical than HPV testing [25]. The affordability of screening services is an important factor affecting the availability of screening services [8,37]. Low cost offers the possibility of achieving screening in large-scale populations, especially in poor areas.

Policy Implication

The full-coverage screening program in Hubei province is the first provincial coverage program and could provide useful lessons for other cities and regions in China, as well as other LMICs, to rapidly increase screening coverage. First, AI-assisted diagnosis technology has a lower cost and reliable quality to reach as many people as possible with limited government funding and to enhance screening coverage. Second, AI technology can address the challenges of primary health institutions, especially the accessibility of screening services in remote or poor areas. Thus, coverage of vulnerable populations will be achieved and equity in screening services will be improved. Finally, AI technology has tremendous capacity advantages that can rapidly improve screening coverage and shorten the process of achieving the target of 70% screening coverage.

Unlike mass population-based screening, precision screening can be implemented in remote, poor, and financially resource-limited areas, even in LMICs. The target population for precision screening may come from areas with high incidence or mortality of cervical cancer, remote and inaccessible areas, or areas with low screening coverage. The cytology screening approach of AI-assisted diagnosis not only addresses screening in areas with insufficient health resources, but more importantly, through the screening program, improves the screening capacity of staff in the area, follow-up management capacity, and health service providers’ treatment capacity, forming a virtuous cycle of service provision and capacity enhancement.

Limitation

Women’s participation in screening is a voluntary selection process. Thus, patients who were diagnosed with a positive screening result did not necessarily choose a county or district health service again when they needed further screening or treatment. This is mainly reflected in the absence of pathological results of screened women in the AI information system. This part of the population often chooses to go to the best municipal hospitals for further screening or treatment. This to some extent wastes health resources and makes it more difficult to manage the follow-up of screened women. The challenge of inducing a rational order for medical treatment is also a problem that China’s health care reform is addressing. However, the AI-assisted diagnosis system has facilitated the follow-up management of women’s screening and the overall improvement of women’s health, thanks to the powerful information management system of the AI-assisted diagnosis system, which can keep track of the implementation of the screening program in the whole region and the information of all the test results of screened women.

Conclusion

Our AI-assisted diagnostic system can complete cervical cancer screening in a short period with low cost and high quality, and make up for the shortage of pathologists in primary health institutions. It has been proven to perform well in large-scale screening and is an effective pathway to address the lack of health resources, especially in areas with insufficient pathologists. AI diagnostics will help China achieve its goal of eliminating cervical cancer ahead of schedule, and also provide a reference for other LMICs and WHO screening guidelines.