Association Between the Digital Clock Drawing Test and Neuropsychological Test Performance: Large Community-Based Prospective Cohort (Framingham Heart Study)

Introduction

The Clock Drawing Test (CDT) is a widely used neuropsychological test to screen cognitive impairment and dementia because of its ease of administration and clinical assessment capability [1,2]. The test is typically administered by specifying a time, for example, ten past eleven, and asking patients or participants to draw a clock showing that time (the command condition), followed by asking patients or participants to copy a predrawn clock image (the copy condition). Both test conditions require multiple cognitive domains. The command test condition requires intact attention, auditory comprehension, semantic memory, executive function, and visuoconstructional abilities, whereas the copy test condition relies primarily upon visuospatial, attention, and executive function skills [3-6]. Keen observation of the process by which drawings are produced is key to the evaluation of the type and severity of cognitive impairment [7,8]. Multiple manual scoring systems have been created to objectively quantify test performance. However, none of these scoring systems can capture the full breadth of cognitive skills used in completing the test [3].

Recently, a digital version of the CDT (dCDT) that uses a digital ballpoint pen and smart paper was developed as an alternative to the standard clock drawing scoring systems [9,10]. The digital pen can record its position with a timestamp and has excellent precision in capturing all graphomotor, spatial, and temporal information [11-13]; however, the characterization of these features and their correlations with standard neuropsychological tests has yet to be examined in a large community-based setting.

The objective of this investigation was to examine the association between dCDT features and cognitive functions in the Framingham Heart Study (FHS) cohort. We also investigated the association between dCDT features and cognitive status (clinically diagnosed mild cognitive impairment compared to those with normal cognition).

Methods

Study Sample

The FHS is a community-based prospective cohort study that was established in 1948. Three generations of participants have been enrolled. Details about the FHS cohort have been previously published [14-16]. This study included participants who completed at least one dCDT and neuropsychological assessment. Participants with prevalent dementia (n=23) or who had not been reviewed by the expert panel (n=138) were excluded. The Boston University Medical Campus Institutional Review Board approved the study procedures and protocols. Written informed consent was obtained from all participants.

The dCDT

Since October 2011, FHS participants who have come for their regular neuropsychological test visit were simultaneously administered the dCDT using a digital pen. The test was jointly developed by the Massachusetts Institute of Technology and the Lahey Hospital and Medical Center with the collaboration of the Clock Sketch Consortium [9-11,17]. Participants used the digital pen (Anoto Inc), to draw a clock on smart paper with a faint dot pattern (Figure 1). The digital pen functions as a regular ballpoint pen does but also measures the pen’s position 80 times per second at a spatial resolution of 0.002 inches with a built-in camera [10,11]. Drawings are automatically classified into different categories, such as numbers, hands, and lines. For quality control, an external rater can replay and deconstruct each drawing to ensure appropriate classification. It typically takes 1 to 2 minutes to classify clocks drawn by healthy or mildly impaired individuals. Any classification errors can be corrected by the rater using a user-friendly drag-and-drop interface. Additional time may be required for classification of more complex clock images, however, most tasks are completed within 5 minutes.

More than 100 dCDT features have been derived to measure the entire drawing process, including capturing all strokes and their corresponding latencies. These features reflect a range of cognitive functions related to drawing efficiency, simple motor operations, information processing speed, and spatial reasoning (Multimedia Appendix 1, Table S1). The rank-based inverse normal transformation was later applied to all dCDT features to reduce the distribution skewness.

‎View this figure

Neuropsychological Assessment

A neuropsychological test protocol, measuring multiple cognitive domains of verbal memory, visual memory, attention and concentration, executive function, abstract reasoning, language, and visuoperceptual organization, was administered to all FHS participants [18]: Wechsler Memory Scale [19] Logical Memory—Immediate Recall, Delayed Recall, and Recognition; Visual Reproduction—Immediate Recall, Delayed Recall, and Recognition; Paired Associate Learning—Immediate Recall, Delayed Recall, and Recognition; the Wechsler Adult Intelligence Scale [20] Digit Span—Forward, Backward, and Similarities; Boston Naming Test—30-item version [21]; Trail Making Test A and B [22]; Hooper Visual Organization Test [23]; Verbal Fluency and Verbal Fluency—Animal [24,25]. All tests were administered by trained raters.

Ascertainment of Mild Cognitive Impairment

In addition to regular research center visits, FHS participants underwent neuropsychological assessments every 4 to 5 years [26,27]. For participants with possible cognitive impairment, regular neuropsychological tests were conducted every 1 to 2 years and neurological exams were performed on a subset of participants. When potential cognitive impairment decline was present, a clinical review was conducted by a panel with at least one neurologist and one neuropsychologist. Mild cognitive impairment diagnosis was determined by the review panel, which required that the patient exhibit evidence of a decline in cognitive performance in one or more cognitive domains, have no records indicating functional decline, and did not meet criteria for dementia. Although the Clinical Dementia Rating scale [28] was not formally applied, the panel used the Clinical Dementia Rating scoring scale (0-3) to quantify the severity of impairment; for mild cognitive impairment, a rating of 0.5 was given.

Statistical Analyses

Linear regression models were used to assess the association between each dCDT feature and neuropsychological tests. The models were adjusted for age and sex. Bonferroni correction was used to adjust for multiple testing, and significant associations were claimed if P<.05/n_t, where n_t was the number of tests performed; therefore, tests were significant if P<4.8x10^–4, with α=.05 and 105 tests. A composite score was also created for each neuropsychological test based on dCDT features that were significantly associated with the test. The score for sample i is defined as

where m is the number of dCDT features significantly associated with the neuropsychological test, β_j is the estimate of effect size for feature j, and V_ij is the normalized dCDT feature j for sample i. The score represented a weighted combination of all dCDT features for the neuropsychological test. The associations between the composite score and each neuropsychological test were also tested with linear regression models adjusted for age and sex.

The association between neuropsychological tests and dCDT composite scores with mild cognitive impairment was assessed by logistic regression models adjusted for age, sex, and education. Age was treated as a continuous variable, whereas sex was treated as a dichotomous variable. Education was treated as a categorical variable (eg, no high school degree, high school degree, some college, and college graduate). The difference between groups was assessed with the Wilcoxon rank-sum test for continuous variables or the chi-square test for dichotomous and categorical variables. Bonferroni correction was used to adjust for multiple tests, and associations were significant if P< 2.8×10^–3, given that 18 neuropsychological tests were used. All statistical analyses were performed using R software (version 4.0.3, The R Project).

Results

As shown in Table 1, our study sample included 2,062 participants (age: mean 62, SD 13 years; 51.6% women; and 43.4% received college-level education or higher). Among them, 36 participants had been diagnosed with mild cognitive impairment. As expected, participants with mild cognitive impairment were generally older and had worse cognitive performance than those in the normal cognition group.

A total of 105 distinct dCDT features were derived from each dCDT drawing. Associations between each individual dCDT feature and 18 neuropsychological tests assessing different cognitive functions are shown in Table S1 (Multimedia Appendix 1). In addition, dCDT features that were significantly associated different cognitive domains were summarized in Table S2 (Multimedia Appendix 1). On average, each neuropsychological test was associated with 50 dCDT features after adjusting for multiple testing.

The weighted composite scores built from significant dCDT features for each neuropsychological test were all significantly associated with their corresponding neuropsychological tests (Table 2).

Eight neuropsychological tests were significantly associated with mild cognitive impairment (P<2.8×10^–3), including Visual Reproduction—Delayed Recall, Visual Reproduction—Immediate Recall, Visual Reproduction—Recognition, Paired Associate Learning—Immediate Recall, Paired Associate Learning—Delayed Recall, Digit Span—Backward, Trail Making Test B, and Logical Memory—Delayed Recall (Table 3). All dCDT composite scores were significantly associated with mild cognitive impairment (P<2.8x10^–3).

Discussion

Neuropsychological tests have been widely used in the assessment of cognitive performance. All 18 neuropsychological tests, for the assessment of multiple cognitive domains, were significantly associated with an average of 50 dCDT features (range 13 to 86 features), and dCDT composite scores were significantly associated with mild cognitive impairment compared to normal cognition.

The CDT examines a wide range of cognitive abilities [5]. The command condition requires that an individual first understand the verbal command, recall all clock related attributes from semantic memory, understand the visuospatial relationships between clock features, and execute the command using necessary mental planning and visuoconstructional abilities. For the copy condition, successful performance requires that an individual recognize the visuospatial attributes in the model to be copied and then marshal the necessary executive abilities to execute output in an organized fashion. However, the standard pencil-and-paper CDT for dementia assessment is usually subjective and time intensive. Given only a limited number of features can be scored, the standard CDT has relatively inferior sensitivity and variable specificity for mild or questionable dementia [2]. On the other hand, the dCDT provides comprehensive and objective assessment of multiple cognitive domains with far greater time efficiency [29,30]. It was reported that the total completion time of dCDT was positively correlated with cognitive functions, whereas the post–clock face latency and pre–first-hand latency were negatively associated with the working memory and processing speed [31]. Our study extended this work by including more than 100 dCDT features and assessing their association with 18 different neuropsychological tests. The composite scores built from dCDT features were significantly associated (all P<.001) with multiple neuropsychological tests, such as Trail Making Test A, Trail Making Test B, Hooper Visual Organization Test, and Visual Reproduction subtests. Our results suggested that dCDT composite scores represent better surrogates for their corresponding neuropsychological tests than individual dCDT features. The results also underscore the psychometric characteristics of the dCDT for measuring multiple cognitive domains, such as attention, executive function, visuoperceptual organization, and visual memory, findings which are consistent with those of prior studies [1,5].

A complete neuropsychological test protocol must be administered by a trained rater and interpreted by a neuropsychological expert, which takes at least 45 minutes; therefore, financial and medical resource requirements limit the application of a complete neuropsychological test protocol in general clinics. In contrast, the dCDT is much more convenient, and the test generally takes less than 2 minutes. It is also worth noting that some patients with mild cognitive impairment may show normal performance in some of cognitive domains of neuropsychological test, demonstrating reduced sensitivity of these neuropsychological tests in detecting mild cognitive impairment for some patient groups [32]. As dCDT composite scores were derived from a combination of multiple features associated with neuropsychological tests, they have the potential to identify more subtle cognitive impairment than individual neuropsychological tests. In an earlier study, Dion et al [31] analyzed 202 older adults without dementia and found that participants with mild cognitive impairment tended to take more time to complete the entire test—more “Think than Ink” (ie, percentage of time thinking vs percentage of time drawing)—and drew smaller clock face areas than those drawn by participants with normal cognition. In another study consisting of 138 patients with mild cognitive impairment and amnesia, 106 patients with mild Alzheimer disease, and 137 normal cognition participants; a tablet-based dCDT provided a slightly higher diagnostic accuracy for patients with mild cognitive impairment and amnesia than the CERAD (Consortium to Establish a Registry for Alzheimer's Disease) total score (81.5% vs 77.5%) [33]. The dCDT features have also been used to differentiate between other neurological diseases, such as memory impairment disorders, vascular cognitive disorders, and Parkinson disease [17].

Several study limitations merit consideration. First, this was a cross-sectional study, which cannot reveal temporal relationship between dCDT performance and mild cognitive impairment. It would be interesting to perform longitudinal analysis to investigate early cognitive markers of dCDT features that predict future cognitive decline. Second, only a moderate number of patients with mild cognitive impairment were included. The number of patients with dementia was even smaller, and therefore, patients with dementia were excluded. Third, neuropsychological tests were used to diagnose mild cognitive impairment, which possibly caused some circularity and overestimated diagnostic performance of neuropsychological tests. On the other hand, neither dCDT features nor derived composite scores were used for the mild cognitive impairment diagnosis, which reduced the bias of potential overestimation. Fourth, due to the increasing exposure to digitalized clock displays, a recent study found that some participants drew digital clocks instead of analog clocks required by the test [34]. It is thus important to continue to explore novel cognitive assessment strategies to better capture new features from different neuropsychological tests to avoid potential bias caused by this new technology trend. Finally, yet importantly, FHS participants were mostly of European ancestry and English speakers, therefore, the applicability of these findings to populations of other race and ethnicity is unknown. Notwithstanding these limitations, our study had several strengths. We studied the association between dCDT and a standard epidemiologic neuropsychological test protocol with community-based FHS study data. FHS data have been collected consistently with rigorous quality control and clinical diagnosis by consensus review. Notably, unlike tablet-based apps, the digital pen used in our study offers an almost identical user experience as that of a traditional ballpoint pen; no extra training is needed, which is particularly important for older adult participants who might be unfamiliar with new digital technologies. The performance was thus less likely distorted [10,11].

Associations between dCDT features and standard neuropsychological test data, as well as composite scores from dCDT features as an alternative to neuropsychological tests for the classification of mild cognitive impairment, from more than 2000 participants from a large community-based cohort suggest the potential of dCDT as a cost-effective and easy-to-administer tool for general practitioners, with potential for use in low-resource countries or regions where clinical dementia expertise is limited.