Detection of Sleep Apnea Using Wearable AI: Systematic Review and Meta-Analysis

Introduction

Background

Sleep apnea refers to a health condition where airflow either ceases or decreases episodically during sleep [1]. According to the American Academy of Sleep Medicine, sleep apnea is categorized as a sleep disorder wherein an individual experiences challenges pertaining to breathing when they are asleep [2]. Primarily, there are 3 kinds of sleep apnea. First, obstructive sleep apnea (OSA) is the consequence of issues with the operation of the upper respiratory tract and is considered a chronic breathing disorder associated with sleep [3]. By contrast, a condition where signals required to regulate breathing muscles are not generated or transmitted is referred to as central sleep apnea (CSA). Complex or mixed sleep apnea is a condition that involves a combination of both OSA and CSA [4]. It often begins as OSA and evolves into CSA [4].

According to global estimates, approximately 936 million adults aged between 30 and 69 years experience OSA [5]. A systematic review showed that the global prevalence of OSA is between 9% and 38% [6]. In the United States alone, the number of people struggling with sleep apnea may exceed 30 million, as per the American Medical Association [7]. Furthermore, studies showed that >80% of sleep apnea cases remain undiagnosed [7-10]. If not diagnosed and treated, sleep apnea may result in severe health issues, such as mood disorders [11-13], cardiovascular diseases [14,15], cognitive deterioration [16,17], increased risk of road accidents [18,19], and all-cause mortality [20,21]. Therefore, the timely detection of sleep apnea for prompt initiation of treatment is imperative.

Conventionally, polysomnography is a comprehensive diagnostic test used in the field of sleep medicine to evaluate and monitor various physiological parameters during sleep to help diagnose sleep disorders, such as sleep apnea [22]. Despite being considered the gold standard for diagnosing sleep apnea, it does have some disadvantages and limitations: (1) it is relatively expensive; (2) access to sleep laboratories may be limited, particularly in certain geographic areas; (3) it can be inconvenient for patients, as they must spend a full night in a sleep laboratory with numerous sensors and electrodes attached to their body; (4) the physiological parameters recorded using polysomnography may not fully reflect the individual’s typical sleep behavior due to a first-night effect in a sleep laboratory, where sleep patterns are different from those at home due to the novelty of the environment; and (5) it is a subjective process, as analyzing polysomnography data depends on sleep clinicians’ experience [22-24]. Hence, there is a dire need to develop and integrate automated technologies and tools that are more efficient and capable of addressing the challenges posed by the current system of diagnosing sleep apnea.

One of the promising solutions that have been used to address the limitations of polysomnography is wearable artificial intelligence (AI), which refers to the integration of AI algorithms and techniques into wearable devices (eg, smartwatches, fitness trackers, and smart glasses) to collect and analyze data (eg, heart rate [HR], respiration rate, and oxygen saturation) to offer various functionalities and insights. Sleep apnea can be efficiently detected with wearable AI due to its convenience, accessibility, affordability, objectivity, and real-time monitoring capabilities. Various types of wearable devices can be used for gathering biomarkers associated with sleep apnea: on-body devices (worn directly on the body or skin), near-body devices (worn close to the body but not touching the skin), in-body devices (implanted within the body), and electronic textiles (clothes with built-in technology). Wearable AI can be used for (1) detecting apnea events in respiration, (2) identifying the type of apnea events in respiration (hypopnea, OSA, CSA, and mixed), (3) detecting patients with sleep apnea, and (4) estimating the severity of sleep apnea.

Research Problem and Aim

In the last decade, numerous investigations have been carried out to evaluate the effectiveness of wearable AI in detecting sleep apnea. Consolidating the results of these studies can contribute to forming more conclusive judgments regarding the effectiveness of wearable AI in detecting sleep apnea. Previous literature reviews attempted to summarize the evidence, but they were constrained by the following limitations. First, most previous reviews were literature reviews rather than systematic reviews [22-28]. Second, many reviews concentrated solely on OSA rather than considering all types of sleep apnea [22,23,25-29]. Third, some reviews focused on a specific type of data, such as HR variability [2,25] and electrocardiography [1,2,25], for sleep apnea detection. Fourth, main databases, such as Embase [1,2,22-29], ACM [1,2,22-29], IEEE [22-25,27-29], and Scopus [1,2,22-25,28], were not incorporated in the searches of previous reviews. Fifth, all prior reviews focused on the performance of various sensors rather than specifically addressing wearable devices [1,2,22-29]. Sixth, one of the reviews focused on non-AI tools for detecting sleep apnea [29]. Seventh, the risk of bias was not taken into account in most of the reviews [1,2,22-28]. Finally, none of these reviews used statistical techniques (eg, meta-analysis) to aggregate findings from previous studies [1,2,22-29]. Hence, this review aimed to bridge all these identified gaps with a focus on examining the performance of wearable AI when it comes to both the detection and prediction of sleep apnea, thereby making it the first of its kind in this field.

Methods

Overview

This review was undertaken and reported in line with the PRISMA-DTA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Diagnostic Test Accuracy) guidelines [30]. Multimedia Appendix 1 provides this review’s PRISMA-DTA checklist. Its protocol has been registered with the PROSPERO (CRD42023495554).

Search Strategy

On December 7, 2023, a comprehensive search was performed across the following electronic repositories: MEDLINE (via Ovid), Embase (via Ovid), ACM Digital Library, Scopus, IEEE Xplore, and Google Scholar. MEDLINE and Embase were chosen due to their reputation as authoritative sources for biomedical and health sciences literature. ACM Digital Library and IEEE Xplore were selected for their status as leading repositories for publications in computing, information technology, electrical engineering, and electronics. Scopus was included because of its comprehensive coverage of scientific literature across multiple disciplines, including health sciences, engineering, computer science, and social sciences. Google Scholar was incorporated, as it indexes scholarly literature from diverse sources and serves as a valuable supplementary tool for identifying relevant studies and gray literature. We set an autoalert to run the search query biweekly for 3 months, concluding on March 6, 2024. Because Google Scholar returned a massive number of results, this review assessed only the first 100 results (equivalent to 10 pages). To identify additional relevant studies, we examined the references cited in the studies already included (backward reference list checking) and studies that had cited the included studies (forward reference list checking).

Relevant literature reviews were assessed, and 2 experts holding doctoral degrees in digital health and health informatics were consulted to compile and collate search terms [31]. The final search query combined three categories of search terms: (1) terms related to AI (eg, “artificial intelligence,” “machine learning,” and “deep learning”), (2) terms associated with wearable devices (eg, “wearable,” “smartwatch,” and “smartband”), and (3) terms linked to sleep apnea (eg, “sleep apnea” and “sleep aponea”). The Boolean operators “OR” and “AND” were used to combine terms within the same category and across different categories, respectively. The specific search query used for searching each database is detailed in Multimedia Appendix 2 for reference.

Study Eligibility Criteria

This review included studies that used AI algorithms to detect sleep apnea or predict its occurrence by leveraging data derived from wearable devices. The research articles deemed suitable for inclusion in this review were those that concentrated on individuals diagnosed with or suspected of having any type of sleep apnea. No limitations were imposed based on age, gender, or ethnicity. In addition, for inclusion in this review, studies were required to evaluate the performance of AI algorithms in detecting or predicting apnea events in respiration, identifying types of apnea events in respiration, detecting patients with sleep apnea, or estimating the severity of sleep apnea. The studies had to provide the confusion matrix or performance metrics (eg, accuracy, sensitivity, and specificity). Studies using AI solely for detecting sleep quality, sleep stages, or other sleep disorders or forecasting the outcomes of sleep apnea interventions were excluded. This review included studies that gathered data using, at a minimum, on-body wearable devices. Conversely, research papers exclusively relying on the following devices for data collection were not considered: nonwearable devices; handheld devices (eg, mobile phones); near-body wearable devices; in-body wearable devices; wearable devices physically connected to nonwearable devices; and wearable devices necessitating expert oversight, such as those demanding precise electrode placement. This review included only peer-reviewed journal articles, conference papers, and dissertations, without restrictions on study setting, study design, reference standard (ie, ground truth), year of publication, or country of study. However, papers not published in English were excluded from consideration. The decision to exclude studies not written in English was based on practical considerations related to resource constraints and the accessibility of non-English literature. While including studies in languages other than English may enhance the comprehensiveness of the review, it can also pose challenges in terms of language translation, interpretation, and the synthesis of findings. Furthermore, English is widely recognized as the dominant language of scholarly communication in many scientific disciplines, including health care and biomedical research. We have transparently acknowledged its implications for the review’s scope and findings in the Limitations section. We also excluded studies that fell into the categories of editorials, reviews, protocols, posters, conference abstracts, and research highlights. The decision to exclude these publication types was primarily guided by the need to maintain the focus and rigor of our review process. While editorials, reviews, and research highlights provide valuable insights into and perspectives on a topic, they typically do not present original research findings or empirical data that meet the objectives of our study. Similarly, protocols, posters, and conference abstracts often offer preliminary or incomplete results that may not undergo peer review or provide sufficient detail for a comprehensive analysis. This helps maintain the quality and reliability of the evidence synthesized in our review while minimizing the risk of bias introduced by including non–peer-reviewed or preliminary findings.

Study Selection

The study selection process comprised 3 key steps. Initially, the EndNote (version X9; Clarivate) software was used to eliminate any duplicate papers from the initial pool. Subsequently, 2 reviewers assessed the titles and abstracts of the remaining studies, separately deciding on their inclusion. Finally, the reviewers independently scrutinized the full texts of the remaining articles. Any discrepancies were deliberated upon and resolved through discussion. The level of agreement between the reviewers was substantial, indicated by a κ score of 0.92 for the evaluation of titles and abstracts and 0.95 for the examination of full texts.

Data Extraction

Initially, 5 studies were used to develop and test the data extraction form shown in Multimedia Appendix 3. Independently, 2 reviewers used Excel (Microsoft Corp) to extract metadata from the studies, participants’ characteristics, wearable devices’ specifications, and AI algorithms’ features. In addition to the previously mentioned extracted data, we collected the highest performance score for each metric, algorithm, and measured outcome. When studies provided raw data or confusion matrices, we calculated all possible performance metrics (eg, accuracy, specificity, and sensitivity). In case of the unavailability of such data, we attempted to obtain them by reaching out to the studies’ first and corresponding authors. Any discrepancies between the 2 reviewers were addressed through discussion between them.

Risk of Bias and Applicability Appraisal

To evaluate the quality of the studies included in our review, we adapted the Quality Assessment of Studies of Diagnostic Accuracy-Revised (QUADAS-2) [32] tool to align with our review’s specific objectives. This adaptation involved substituting some of the original criteria, which were not applicable to our context, with more relevant criteria from the Prediction Model Risk of Bias Assessment Tool [33]. We modified the QUADAS-2 tool to encompass 4 main domains tailored to our review: “participants,” “index test” (focused on AI algorithms), “reference standard” (representing the ground truth), and “analysis.” For each domain, we developed 4 targeted questions aligned with our review’s objectives. In addition, our evaluation assessed the practical applicability of the results derived from the first 3 domains. To optimize our adapted tool, we initially tested it on 5 studies for fine-tuning purposes. The included studies were independently evaluated by 2 reviewers using the modified QUADAS-2 tool (Multimedia Appendix 4). Any differences in their assessments were discussed and resolved through consensus.

Data Synthesis

We used both narrative and statistical techniques to synthesize the data extracted from the included studies. In our narrative synthesis, we used textual descriptions and tabulated summaries to elucidate the characteristics of the included studies, encompassing study metadata, wearable devices, and AI techniques. As for the statistical approach, a meta-analysis was carried out when at least 2 different studies presented enough data to perform the analysis. We conducted conventional meta-analyses for results associated with the following outcomes, given that they were extracted from different unique studies (ie, independent effect sizes): identification of types of apnea events in respiration, detection of patients with sleep apnea, and estimation of the severity of sleep apnea. Specifically, DerSimonian-Laird random-effects models [34] using the Freeman-Tukey double arcsine transformation [35,36] were performed to pool the extracted results. This method considers variations arising from sampling and accounts for heterogeneity in estimates. The analysis was carried out using the meta toolkit within R (version 4.2.2; The R Foundation) [37].

We also performed multilevel meta-analyses for results related to the detection of apnea events in respiration, as certain results originated from the same study (ie, dependent effect sizes) [34,38]. Multilevel meta-analyses were used to address this dependency in effect sizes, thereby minimizing the risk of type I errors. These analyses were carried out using the metafor toolkit within R (version 4.2.2) [35].

When applicable, subgroup meta-analyses were conducted to explore how different factors might influence the effectiveness of wearable AI [34,38]. These factors included AI algorithms, the type of algorithm (ie, machine learning [ML] vs deep learning), the number of participants, the type of sleep apnea, the status of the wearable device (ie, commercial vs noncommercial), the placement of the wearable device, data set size, data type, ground truth, and validation method. We considered differences in results between subgroups to be statistically significant if the statistical probability (P value) was <.05.

To assess how consistent the studies were in their findings (heterogeneity), we used 2 statistical tests. The first test is the Cochrane Q statistic, which indicates whether the observed differences in results could be due to chance alone. A P value <.05 indicates significant heterogeneity, meaning the results varied more than expected by chance. The second test is the I² statistic, which quantifies the proportion of observed variability due to real differences between studies rather than differences by chance [35,39]. Heterogeneity was considered insignificant when I² ranged from 0% to 40%, moderate when I² fell within the 30% to 60% range, substantial when I² ranged from 50% to 90%, or considerable when I² extended from 75% to 100%.

Results

Search Results

As depicted in Figure 1, a total of 615 citations were retrieved when the above-identified databases were searched. Of the retrieved citations, 161 (26.2%) duplicates were removed using EndNote X9, leaving 454 (73.8%) studies. Further, 362 (79.7%) studies were removed after screening the titles and abstracts of these 454 studies. After retrieving and reading the full text of all the remaining 92 (20.3%) studies, it was determined that 57 (62%) of these studies were ineligible for inclusion. The main reasons for exclusion were that they did not use wearable devices (23/92, 25%), did not use AI algorithms (11/92, 12%), did not focus on sleep apnea (6/92, 7%), were irrelevant publication types (16/92, 17%), or were not written in English (1/92, 1%). We identified 3 additional studies relevant to this review through backward reference list checking. In total, 38 studies were included in this review [40-77], and 27 (71%) of them were eligible for meta-analyses [40,41,45-49,52-55,57,58,61-64,66,68,69,71-77].

‎

Characteristics of the Included Studies

As displayed in Table 1, the number of studies has varied over the years, with the highest number reached in 2020 (11/38, 29%). While the included studies were conducted in 16 different countries, the studies were predominantly from the United States (9/38, 24%). Most of the studies were journal articles (29/38, 76%), but conference papers also made a substantial contribution (9/38, 24%). The average number of participants across studies was 155.9 (SD 374.9). The number of participants ranged from 4 to 2252. The mean age of participants was identified in 25 (66%) of the 38 included studies and ranged from 25.6 to 61.1 years, with an average of 47.3 (SD 9.3) years. Across 25 studies reporting the proportion of female participants, female participants constituted an average of 37.4% of the total participants, ranging from 12% to 65%. A total of 20 studies reported the BMI, which ranged from 22.1 to 38.7 kg/m², with an average of 28.6 (SD 3.81) kg/m². About two-thirds (25/38, 66%) of studies did not focus on a specific type of sleep apnea. The characteristics of each included study are listed in Multimedia Appendix 5.

Features of Wearable Devices

Commercial wearable devices constituted the majority of wearable devices in the included studies (24/38, 63%; Table 2). The most mentioned wearable device in the included studies was the Belun Ring (3/38, 8%). Wearable devices are placed on various body parts, with the chest (16/38, 42%), wrist (11/38, 29%), and abdomen (9/38, 24%) being the most common locations. Wearable devices were worn for 1 full night (6-8 hours) in 29 studies (76%). The features of wearable devices in each included study are shown in Multimedia Appendix 6.

Features of AI

Classification was the dominant problem-solving approach used in the included studies (38/38, 100%; Table 3). Various AI algorithms were used in the included studies, with convolutional neural networks (CNNs) being the most common (14/38, 37%). Among the 38 included studies, most studies (n=37, 97%) used AI to detect the current sleep apnea, whereas 3 (8%) studies used wearable AI to predict sleep apnea before its occurrence. The mean data set size reported in 28 (74%) studies was 60,554 (SD 133,059), with the range spanning from 12 to 561,480. Most studies (36/38, 95%) used closed-source data, while only 2 (5%) of 38 studies used open-source data. Data were gathered through wearable devices in all studies (38/38, 100%), via self-reported questionnaires in 3 (8%) studies, and using nonwearable devices (eg, smartphones) in 2 (5%) studies. Respiration data (eg, respiratory rate and respiratory efforts; 25/38, 66%) and HR data (eg, HR, HR variability, and interbeat interval; 21/38, 55%) were the most frequently used data for developing the models in the included studies. The number of features reported in 21 (55%) of the 38 studies ranged from 3 to 212, with an average of 44.3 (62.5). Most studies used polysomnography as the ground truth assessment method (26/38, 68%), followed by the wearable device (8/38, 21%) and the context of the experiment (eg, performing different patterns of breathing; 4/38, 11%). In 28 studies that reported the assessor of the ground truth, sleep technicians were the most common assessors (23/38, 61%), followed by sleep physicians (8/38, 21%). American Academy of Sleep Medicine guidelines were followed in 84% (32/38) of studies to assess the ground truth. Train-test split was the most common approach used in the included studies to validate the performance of AI models (20/38, 53%), followed by k-fold cross-validation (17/38, 45%). The included studies used wearable AI to detect apnea events in respiration (24/38, 63%) and patients with sleep apnea (15/38, 40%) and to identify the severity of sleep apnea (21/38, 55%) and types of apnea events in respiration (8/38, 21%). The features of AI in each included study are described in Multimedia Appendix 7.

Results of Risk-of-Bias Appraisal

Nearly half of the included studies (17/38, 45%) reported comprehensive details to determine whether an appropriate consecutive or random sample of eligible participants was used. Over half of the studies (22/38, 58%) avoided inappropriate exclusions. A substantial majority, 30 (79%) out of 38 studies, ensured a balanced number of patients across subgroups. In addition, around two-thirds (25/38, 66%) of the studies reported a sufficient sample size. Consequently, a little less than half of the studies (16/38, 42%) were assessed as having a low risk of bias in the “selection of participants” domain, as shown in Figure 2. In terms of matching participants to the predefined requirements in the review question, a low level of concern was identified in nearly 40% (15/38, 40%) of the included studies, as shown in Figure 3.

‎

A substantial majority of the included studies comprehensively detailed their AI models, with 34 (89%) out of 38 studies providing thorough descriptions. Almost all, 35 (92%) out of 38 studies, clearly reported the features (predictors) used. Moreover, an overwhelming majority, 36 (95%) out of 38 studies, ensured that these features were sourced without prior knowledge of the outcome data. Consistency in feature assessment across participants was observed in 35 (92%) out of 38 studies. Consequently, the potential for bias in the “index test” domain was assessed as low in the vast majority of the studies (32/38, 84%), as shown in Figure 2. In addition, 32 (84%) out of 38 studies were found to have minimal concerns regarding the alignment between the model’s predictors and the review question’s criteria, as illustrated in Figure 3.

‎

In most of the included studies (32/38, 84%), the outcome of interest, specifically sleep apnea, was consistently assessed using appropriate methodologies. Nearly all studies (37/38, 97%) defined and determined the outcome in a uniform manner for all participants. An overwhelming majority of the studies (36/38, 95%) determined the outcome without prior knowledge of the predictor information. In a substantial portion of the studies (33/38, 87%), the diagnostic test was conducted for an appropriate duration to ensure accurate results. As a result, the potential for bias in the “reference standard” domain was deemed low in the vast majority of the studies (32/38, 84%), as shown in Figure 2. In addition, the same number of studies (32/38, 84%) showed minimal concerns regarding any discrepancies between the outcome’s definition, timing, or determination and the review question’s criteria, as indicated in Figure 3.

Finally, a significant majority of the studies (34/38, 89%) ensured the inclusion of all enrolled participants in the data analysis. A substantial number of these studies (32/38, 84%) executed proper data preprocessing. Similarly, a high proportion (34/38, 89%) adopted suitable measures to evaluate the performance of their models. Nearly half of the studies (17/38, 45%) demonstrated an appropriate split among training, validation, and test sets. However, the risk of bias in the validation methods used by the remaining studies remained unclear due to insufficient information being provided. Consequently, slightly more than half of the studies (20/38, 53%) were deemed to have a low risk of bias in the “analysis” domain, as indicated in Figure 2. A detailed breakdown of the “risk of bias” and “applicability concerns” for each domain in every study is available in Multimedia Appendix 8.

Results of the Studies

As mentioned earlier, meta-analyses were carried out to pool results related to 4 outcomes: detection of apnea events in respiration, identification of types of apnea events in respiration, detection of patients with sleep apnea, and estimation of the severity of sleep apnea. The following subsections present the results of the meta-analyses for each outcome.

Apnea Events in Respiration

Accuracy

We conducted meta-analyses of 36 estimates of accuracy derived from 2,702,305 respiratory events across 17 (45%) of the 38 studies (Table 4). The pooled mean accuracy of these estimates was 0.893 (95% CI 0.82-0.94). The meta-analyzed evidence exhibited considerable statistical heterogeneity (P<.001; I²=100%). Further, Table 4 shows that there is a statistically significant difference in the pooled mean accuracy between subgroups in the “algorithms” group (P<.001) and “type of algorithms” group (P=.02), whereas no statistically significant difference (P>.05) was found in the pooled mean accuracy between subgroups in the remaining groups.

Sensitivity

As shown in Table 5, meta-analyses were carried out on 22 estimates of sensitivity derived from 872,443 respiratory events across 15 (39%) of the 38 studies. The pooled mean sensitivity of these estimates was 0.793 (95% CI 0.67-0.87). The meta-analyzed evidence has considerable statistical heterogeneity (P<.001; I²=100%). With regard to subgroup analyses, there was no statistically significant difference in the pooled mean sensitivity between subgroups in all groups.

Specificity

Meta-analyses were performed to pool 22 estimates of specificity derived from 1,699,503 respiratory events across 15 (39%) of the 38 studies (Table 6). The pooled mean specificity of these estimates was 0.946 (95% CI 0.88-0.98). There was considerable statistical heterogeneity (P<.001; I²=100%) in the meta-analyzed studies. We also found a statistically significant difference in the pooled mean specificity between subgroups in the “status of wearable device” group (P=.01), while there was no statistically significant difference (P>.05) in the pooled mean specificity between subgroups in the rest of the groups.

Type of Apnea Events in Respiration

We conducted meta-analyses of 6 estimates of accuracy derived from 637,250 respiratory events across 6 (16%) of the 38 studies (Table 7). The pooled mean accuracy of these estimates was 0.815 (95% CI 0.64-0.94). The meta-analyzed studies exhibited considerable statistical heterogeneity (P<.001; I²=100%). In addition, there was a statistically significant difference in the pooled mean accuracy between subgroups in the “data type” group (P=.001), while no statistically significant difference (P>.05) was found in the pooled mean accuracy between subgroups in the remaining groups.

Patients With Sleep Apnea

Accuracy

We carried out meta-analyses of 13 estimates of accuracy derived from 2015 participants across 13 (34%) of the 38 studies (Table 8). The pooled mean accuracy of these estimates was 0.869 (95% CI 0.81-0.92). The meta-analyzed estimates showed considerable statistical heterogeneity (P<.001; I²=100%). Further, there was a statistically significant difference in the pooled mean accuracy between subgroups in the “type of sleep apnea” group (P=.049). However, no statistically significant difference (P>.05) was found in the pooled mean accuracy between subgroups in the remaining groups.

Sensitivity

As shown in Table 9, meta-analyses were carried out on 13 estimates of sensitivity derived from 1580 participants across 13 (34%) of the 38 studies. The pooled mean sensitivity of these estimates was 0.938 (95% CI 0.89-0.97). The meta-analyzed evidence has considerable statistical heterogeneity (P<.001; I²=82%). With regards to subgroup analyses, there was no statistically significant difference in the pooled mean sensitivity between subgroups in all groups except for the “placement of wearable device” group (P<.001).

Specificity

Meta-analyses were performed to pool 13 estimates of specificity derived from 436 participants across 13 (34%) of the 38 studies (Table 10). The pooled mean specificity of these estimates was 0.752 (95% CI 0.63-0.86). There was considerable statistical heterogeneity (P<.001; I²=78%) in the meta-analyzed studies. Our subgroup meta-analyses showed that there was no statistically significant difference in the pooled mean specificity between subgroups in all groups.

Severity of Sleep Apnea

Accuracy

We performed meta-analyses of 9 estimates of accuracy derived from 1661 participants across 9 (24%) of the 38 studies (Table 11). The pooled mean accuracy of these estimates was 0.651 (95% CI 0.54-0.75). The meta-analyzed studies exhibited considerable statistical heterogeneity (P<.001; I²=93%). In addition, there was a statistically significant difference in the pooled mean accuracy between subgroups in “type of sleep apnea” group (P=.03) and “data type” group (P=.01), while no statistically significant difference (P>.05) was found in the pooled mean accuracy between subgroups in the remaining groups.

Correlation Coefficient

As shown in Table 12, meta-analyses were carried out on 12 estimates of correlation coefficient (r) derived from 1266 participants across 12 (32%) of the 38 studies. The pooled mean r of these estimates was 0.877 (95% CI 0.82-0.92). The meta-analyzed evidence has considerable statistical heterogeneity (P<.001; I²=82%). With regard to subgroup analyses, there was a statistically significant difference in the pooled mean r between subgroups in the “placement of wearable device” group (P<.001) and the “data type” group (P<.001). However, no statistically significant difference (P>.05) was found in the pooled mean r between subgroups in the remaining groups.

Discussion

Principal Findings

This systematic review investigated how well wearable AI performs in detecting sleep apnea. Overall, the findings indicate that wearable AI demonstrated a performance that is deemed acceptable, although not optimal, for detecting sleep apnea. Specifically, wearable AI was able to correctly classify apnea events and nonapnea events in 89.3% of respiratory events. This performance was notably higher when using CNN in particular or deep learning algorithms in general. The superiority of CNN architectures can be attributed to their ability to capture the localized dependencies inherent in apnea patterns through convolution kernels. The meta-analyses conducted in this review revealed that wearable AI performed better in detecting nonapnea respiratory events (94.6%) compared to apnea respiratory events (79.3%). This could be linked to the training of AI models using an unrepresentative sample, wherein the number of nonapnea respiratory events (n=1,699,503) was approximately twice as high as the number of apnea respiratory events (n=872,443). This highlights the challenge of applying data balancing techniques for heterogeneous and time-dependent measurements, particularly evident in longitudinal recordings as observed in apnea studies.

Although the sensitivity of wearable AI in detecting apnea events in respiration remained unaffected by any moderating factors, the specificity was influenced by the status of the wearable device, where noncommercial devices exhibited higher specificity than commercial devices. This can be because all studies that used noncommercial wearable devices applied deep learning algorithms, whereas more than one-third (6/16, 38%) of studies that used commercial wearable devices applied ML algorithms (eg, random forest, AdaBoost, and k-nearest neighbors). Introducing scalable AI models, such as deep learning models, into commercial apnea detection applications presents challenges due to their computational expense and resource requirements, thereby complicating market penetration and impacting profit margins. However, recent advancements in tiny ML models and edge AI implementations offer potential solutions to mitigate these challenges. This review also demonstrated that wearable AI was able to correctly differentiate between different types of apnea events (eg, apnea, hypopnea, obstructive apnea, and central apnea) in 81.5% of respiratory events, and this performance was not influenced by any moderators. This can be attributed to the lack of studies (≤4) in all subgroup analyses related to this outcome.

In this review, wearable AI demonstrated 86.9% accuracy in correctly identifying patients with and patients without sleep apnea. This performance was notably higher when the wearable AI was used for detecting sleep apnea in general (92%) rather than OSA in particular (82.3%). This difference may be attributed to the fact that approximately 83% (5/6) of studies focusing on general sleep apnea detection used respiration data to develop the AI models. By contrast, only 29% (2/7) of studies concentrating on OSA detection incorporated respiration data. Given that respiration data are widely acknowledged as the most crucial indicator of sleep apnea, this disparity in use may explain the varying performance levels observed in the review.

Unlike apnea event detection, wearable AI exhibited superior performance in identifying patients with sleep apnea (93.8%) compared to those without sleep apnea (75.2%). This could be associated with the training of AI models using an unrepresentative sample, wherein the number of patients with sleep apnea (1580) was >3 times higher than the number of patients without sleep apnea (436). The specificity of wearable AI in detecting sleep apnea was not affected by any moderator, while its sensitivity was higher when wearable devices were placed on both the chest and abdomen in comparison with other placements (wrist or fingers). This moderation effect could be attributed to the fact that all studies that placed wearable devices on both the chest and abdomen focused on detecting sleep apnea in general, while 6 (86%) out of 7 studies that placed wearable devices in other places focused on detecting OSA in particular. Further, all studies that placed wearable devices on other body parts used commercial wearable devices, whereas only 1 of the studies that placed wearable devices on both the chest and abdomen used commercial wearable devices.

Our meta-analyses also revealed that wearable AI accurately differentiated between various levels of sleep apnea severity (normal, mild, moderate, and severe) in 65.1% of cases. This performance was higher when the wearable AI was used for detecting the severity of sleep apnea in general rather than OSA in particular. This could be linked to the fact that all studies that aimed to detect OSA used commercial devices that were placed on either fingers or wrists, while two-thirds of the studies that focused on sleep apnea in general used noncommercial devices that were placed on both the abdomen and chest. This performance was also higher when the model was developed using both respiration and oxygen saturation data in comparison with using a combination of body movement, HR, and oxygen saturation data. This could be associated with the fact that all studies using the combination of body movement, HR, and oxygen saturation data focused on the detection of OSA using commercial devices placed on fingers, while all studies using both respiration and oxygen saturation data focused on detecting any sleep apnea type using noncommercial devices (in 2, 67% out 3 studies) placed on the abdomen and chest.

Finally, the accuracy of wearable AI in estimating the severity of sleep apnea (ie, the apnea-hypopnea index score) reached 87.7%. This accuracy was higher when the wearable device was placed on the chest and when using both respiration and HR data or a combination of HR, oxygen saturation, and body movement.

Research and Practical Implications

Our analysis revealed that wearable AI shows promise in identifying sleep apnea, distinguishing its type, and gauging its severity; however, it is not yet ready for widespread use in clinical practices for 3 reasons. First, its current performance falls below the optimal level. Second, only 9 (24%) of the 38 studies were judged to have a low risk of bias in all domains. Third, heterogeneity between studies was considerable in most meta-analyses. Therefore, we cannot propose the use of wearable AI as a replacement for traditional sleep assessments (eg, polysomnography and home sleep apnea testing), but we recommend that wearable AI be used in conjunction with these assessments, taking into account factors such as cost-effectiveness and practical challenges in real-world implementation.

Among all wearable devices used in the included studies, only 1 was specifically designed for diagnosing sleep and monitoring sleep health and obtained clearance from the US Food and Drug Administration. This may be due to a shortage of such wearable devices in the market or a scarcity of studies evaluating them. We urge manufacturers of wearable devices to extend their focus beyond evaluating sleep quality and incorporate AI into their devices for identifying sleep apnea, its various types, and its severity. Further, researchers should pay more attention to such wearable devices in their future studies. The main challenge of conducting such studies is the cost of such wearable devices.

Our meta-analyses indicate that the performance of wearable AI was notably higher when using CNN in particular or deep learning algorithms in general. Therefore, we recommend that manufacturers of wearable devices and researchers prioritize these techniques during the development of devices intended for the detection of sleep apnea. However, obtaining large, high-quality, and standardized data sets for training and validating CNN or deep learning models can be challenging.

Our meta-analysis suggests the need for implementing AI on the edge through specially crafted tiny ML modules with federated learning protocols. Such an approach not only enhances performance metrics but also addresses critical considerations regarding resource efficiency, latency reduction, and privacy preservation inherent in commercial apnea detection systems. However, implementing AI on wearable devices, especially with tiny ML modules, poses challenges related to hardware constraints, such as limited processing power, limited memory, and high energy consumption. Further, ensuring that AI algorithms can run efficiently on resource-constrained devices without compromising performance is a significant challenge. Implementing federated learning protocols for edge devices introduces additional complexities related to communication, synchronization, and security. Designing robust federated learning frameworks that can effectively train AI models across distributed devices while preserving data privacy and security requires careful consideration and expertise.

Most studies included in this review focused on the application of wearable AI for the detection of existing sleep apnea, its type, or its severity, rather than the anticipation of its occurrence. Foreseeing the onset of sleep apnea in the future is as pivotal as, if not more pivotal than, recognizing the current sleep apnea state, as it can pave the way for the development and implementation of proactive interventions. Consequently, we encourage researchers to undertake additional investigations into the capacity of wearable AI to predict future instances of sleep apnea. Such studies collect longitudinal data over an extended period to train and validate predictive models accurately. However, obtaining continuous and comprehensive sleep data from individuals over time can be challenging due to factors such as participant compliance, dropout rates, and the need for long-term monitoring.

In this review, only a single study evaluated the effectiveness of wearable AI in identifying CSA. In addition, only 7 (18%) of the 38 studies investigated the capability of wearable AI to differentiate between different types of sleep apnea. More research is urgently needed to evaluate the performance of wearable AI in these crucial areas. Our study also suggests that more open-source data sets with prepared manual labels for different types of sleep apnea are needed. Collecting large-scale, comprehensive, and well-annotated data from individuals with CSA poses challenges due to the rarity of CSA cases. Furthermore, identifying informative features and physiological signals from wearable devices that can distinguish between different types of sleep apnea is challenging due to the overlap in the clinical presentation and physiological characteristics of different types of sleep apnea, particularly in cases of mixed sleep apnea where both obstructive and central events occur concurrently.

Merely 3 (8%) of the 38 included studies used self-reported data and nonwearable device data alongside wearable device data for the detection of sleep apnea. The inclusion of self-reported data (eg, data regarding demographics, BMI, medical history, family history, and medications) and nonwearable device data (eg, data collected via mobile phones, smart pillows, smart mattresses, voice recorders, and Internet of Things) has the potential to enhance the efficacy of wearable AI in identifying sleep apnea. Hence, manufacturers and researchers are encouraged to take these types of data into consideration, alongside wearable device data, when developing wearable AI for the diagnosis of sleep apnea. However, challenges arise in transferring nonwearable data to wearable devices and the potential impact on the performance of wearable devices in terms of processing speed, memory use, energy consumption, synchronization, and security.

A few studies in this review compared the performance of wearable devices worn on different parts of the body (eg, wrist, abdomen, and chest) and developed wearable AI for not only detecting but also intervening in sleep apnea. This points to a crucial gap in research, which urges further investigation into the different performances of wearable AI with different placements and integrated treatment delivery via wearable AI for sleep apnea management.

Among the 38 studies in our review, 11 (29%) were excluded from the meta-analyses due to insufficient details crucial for their conduct (eg, confusion matrices and the number of apnea and nonapnea cases). They also did not provide multiple performance measures (eg, accuracy, sensitivity, and specificity), which are essential for estimating the necessary information. It is recommended that researchers include these specific details in their reports to facilitate the conduct of meta-analyses by other researchers. However, we acknowledge that the space constraints imposed by journals and conference proceedings may present a challenge for researchers seeking to include more comprehensive details in their reports.

Limitations

Our review intentionally excluded studies involving (1) nonwearable devices, near-body wearable devices, in-body wearable devices, wearable devices wired to nonwearable devices, and wearable devices requiring an expert for their placement on users; (2) wearable AI in detecting other sleep disorders (eg, insomnia, narcolepsy, and restless legs syndrome); and (3) wearable AI in predicting outcomes of sleep apnea interventions or detecting sleep quality or sleep stages. Therefore, our findings are specifically applicable to wearable AI for sleep apnea detection and may not be generalizable to the excluded devices, disorders, or outcomes. Our findings are based on studies conducted in only 16 countries. Further, while most studies were carried out in hospitals, only 4 (11%) of the 38 studies were conducted in health care centers. Therefore, extrapolating our results to broader populations and clinical settings requires caution. This limitation acknowledges the need for further reviews in these broader areas.

Another limitation of this review is the likelihood of an overestimation or underestimation of the results of our meta-analyses due to 2 reasons. First, some relevant studies could have been overlooked, as our search was confined to English-language publications, and we did not explore other widely used databases, such as CINAHL and Web of Science. Secondly, 11 of the 38 studies in this review were excluded from the meta-analyses, as they did not report details required for meta-analyses.

Conclusions

Our review underscores the potential of wearable AI in identifying sleep apnea, differentiating its type, and gauging its severity. However, wearable AI is not yet ready for integration into routine clinical practices due to its suboptimal performance. Therefore, until further evidence demonstrates an ideal performance, we suggest the concurrent use of wearable AI with traditional sleep apnea assessments (eg, polysomnography and home sleep apnea testing), rather than a complete substitution. Manufacturers need to develop certified commercial wearable devices that can easily detect sleep apnea, predict its occurrence, and deliver proactive interventions. CNN in particular or deep learning algorithms in general should be prioritized during the development of wearable AI for the detection of sleep apnea. Further studies are needed to assess the performance of wearable AI in detecting CSA and distinguishing it from other types of sleep apnea. Researchers should consider incorporating self-reported and nonwearable device data alongside wearable data to enhance the efficacy of wearable AI in detecting sleep apnea. Additional research is required to evaluate the varying performance of wearable devices with different placements. Researchers should also report sufficient details about their findings to enable other researchers to conduct meta-analyses effectively.