This paper is in the following e-collection/theme issue:

New Methods (902) Diagnostic Tools in Mental Health (220) Cognitive and Neurorehabilitation (236) Methods and New Tools in Mental Health Research (424) Cognitive Training for the Elderly (82) Dementia and Cognitive Decline (419) Neurology and Neurosciences (148)

Published on in Vol 24, No 4 (2022): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/34513, first published .
Associations Between the Digital Clock Drawing Test and Brain Volume: Large Community-Based Prospective Cohort (Framingham Heart Study)

Associations Between the Digital Clock Drawing Test and Brain Volume: Large Community-Based Prospective Cohort (Framingham Heart Study)

Associations Between the Digital Clock Drawing Test and Brain Volume: Large Community-Based Prospective Cohort (Framingham Heart Study)

Authors of this article:

Jing Yuan1, 2 Author Orcid Image ;   Rhoda Au2, 3, 4, 5, 6, 7 Author Orcid Image ;   Cody Karjadi2, 3 Author Orcid Image ;   Ting Fang Ang2, 3, 6 Author Orcid Image ;   Sherral Devine2, 3 Author Orcid Image ;   Sanford Auerbach3, 4 Author Orcid Image ;   Charles DeCarli8 Author Orcid Image ;   David J Libon9 Author Orcid Image ;   Jesse Mez3, 4, 7 Author Orcid Image ;   Honghuang Lin10 Author Orcid Image
Article Authors Cited by (3) Tweetations (4) Metrics

    Original Paper

    1Department of Neurology, Peking Union Medical College Hospital, Peking Union Medical College, Chinese Academy of Medical Sciences, Beijing, China

    2Department of Anatomy and Neurobiology, Boston University School of Medicine, Boston University, Boston, MA, United States

    3Framingham Heart Study, Boston University School of Medicine, Boston University, Boston, MA, United States

    4Department of Neurology, Boston University School of Medicine, Boston University, Boston, MA, United States

    5Department of Epidemiology, Boston University School of Public Health, Boston University, Boston, MA, United States

    6Slone Epidemiology Center, Boston University School of Medicine, Boston University, Boston, MA, United States

    7Alzheimer’s Disease Research Center, Boston University, Boston, MA, United States

    8Department of Neurology and Center for Neuroscience, University of California, Davis, Sacramento, CA, United States

    9Department of Geriatrics and Gerontology and Department of Psychology, New Jersey Institute for Successful Aging, Rowan University, School of Osteopathic Medicine, Stratford, NJ, United States

    10Division of Clinical Informatics, Department of Medicine, University of Massachusetts Chan Medical School, Worcester, MA, United States

    Corresponding Author:

    Honghuang Lin, PhD

    Division of Clinical Informatics

    Department of Medicine

    University of Massachusetts Chan Medical School

    55 Lake Ave North, S7-749

    Worcester, MA, 01655

    United States

    Phone: 1 774 455 4881

    Email: Honghuang.Lin3@umassmed.edu


    Background: The digital Clock Drawing Test (dCDT) has been recently used as a more objective tool to assess cognition. However, the association between digitally obtained clock drawing features and structural neuroimaging measures has not been assessed in large population-based studies.

    Objective: We aimed to investigate the association between dCDT features and brain volume.

    Methods: This study included participants from the Framingham Heart Study who had both a dCDT and magnetic resonance imaging (MRI) scan, and were free of dementia or stroke. Linear regression models were used to assess the association between 18 dCDT composite scores (derived from 105 dCDT raw features) and brain MRI measures, including total cerebral brain volume (TCBV), cerebral white matter volume, cerebral gray matter volume, hippocampal volume, and white matter hyperintensity (WMH) volume. Classification models were also built from clinical risk factors, dCDT composite scores, and MRI measures to distinguish people with mild cognitive impairment (MCI) from those whose cognition was intact.

    Results: A total of 1656 participants were included in this study (mean age 61 years, SD 13 years; 50.9% women), with 23 participants diagnosed with MCI. All dCDT composite scores were associated with TCBV after adjusting for multiple testing (P value <.05/18). Eleven dCDT composite scores were associated with cerebral white matter volume, but only 1 dCDT composite score was associated with cerebral gray matter volume. None of the dCDT composite scores was associated with hippocampal volume or WMH volume. The classification model for differentiating MCI and normal cognition participants, which incorporated age, sex, education, MRI measures, and dCDT composite scores, showed an area under the curve of 0.897.

    Conclusions: dCDT composite scores were significantly associated with multiple brain MRI measures in a large community-based cohort. The dCDT has the potential to be used as a cognitive assessment tool in the clinical diagnosis of MCI.

    J Med Internet Res 2022;24(4):e34513

    doi:10.2196/34513

    Keywords

    Clock Drawing Test; digital; neuropsychological test; cognitive; technology; Boston Process Approach; neurology; Framingham Heart Study; dementia; Alzheimer 


    The Clock Drawing Test (CDT) is a widely used assessment tool able to screen for impairment associated with mild cognitive impairment (MCI) [1] and dementia [2-5]. The most common instruction for clock drawing asks participants to draw the face of a clock, put in all of the numbers, and set the hands for 10 past 11. This is followed by asking participants to copy a model of a clock [3,6]. Multiple cognitive domains are involved in completion of the test, including graphomotor ability, attention, syntactic comprehension, visual and semantic memory, executive function, and visuoconstructional ability [6]. Previous studies found that CDT performance evaluated by various manual scoring systems was correlated with a variety of cortical and subcortical areas on magnetic resonance imaging (MRI), without showing consistent and specific brain localization [7-9].

    A significant innovation is the introduction of a digital CDT (dCDT), which was jointly developed by the Massachusetts Institute of Technology and Lahey Hospital & Medical Center, in collaboration with the Clock Sketch Consortium [10-13]. In addition to retaining the classic features of the traditional CDT, including easy, inexpensive, noninvasive administration, the dCDT captures more than 100 latencies and graphomotor features for subtle cognitive changes, which would be difficult or impossible without this technology [12-14]. A recent study reported that the DCTClock test (a commercial version of the dCDT) showed excellent discrimination between diagnostic groups of normal cognition (NC) and MCI or early Alzheimer dementia [14]. The test was also associated with amyloid β and tau burden in positron emission tomography scans among NC participants [14].

    In our prior study [15], we validated the psychometric characteristics of the dCDT against standard paper-pencil neuropsychological (NP) tests in the community-based Framingham Heart Study (FHS) [15]. The composite scores derived from dCDT features were significantly associated with both NP test results and mild cognitive impairment [15]. Nonetheless, it is unclear whether dCDT composite scores are associated with MRI measures of atrophy, which have been used as important markers for cognitive impairment and cerebral small vessel disease injuries [16-20].

    The objectives of this study were to investigate the associations between dCDT features and MRI measures in the FHS, and to assess the diagnostic potential of dCDT features as a surrogate for standard NP tests in the differentiation of MCI from NC that is validated against MRI measures of neurodegeneration.


    Study Sample

    The FHS is a community-based prospective cohort study that was established in 1948. Three generations of participants have been enrolled and followed up every 2 to 8 years. Details on the FHS cohorts can be found in previous publications [21-23]. As part of an ancillary study, participants were invited to undergo NP assessment and brain MRI scans regularly. The dCDT has been administered as part of the NP assessment along with other NP tests since 2011. This study included participants who completed at least one dCDT and had a contemporary MRI within 6 months (98.2% had MRI scans on the same day as the dCDT). Participants with prevalent dementia and stroke were excluded (n=84). We further excluded participants who were flagged as having possible cognitive impairment, but were not reviewed by an expert panel (n=139).

    Ethics Approval

    The Boston University Medical Campus Institutional Review Board approved the study procedures and protocols. Written informed consent was obtained from all participants. All the data used for this study could be requested through the FHS research website [24].

    Standard NP Tests and the dCDT

    In the FHS, standard NP tests were administered and scored to produce traditional quantitative measures of cognitive performance [25]. These tests included the Wechsler Memory Scale [26] Logical Memory—Immediate Recall, Delayed Recall, and Recognition; Visual Reproduction—Immediate Recall, Delayed Recall, and Recognition; and Paired Associate Learning—Immediate Recall, Delayed Recall, and Recognition; the Wechsler Adult Intelligence Scale [27] Digit Span—Forward and Backward and Similarities; the Boston Naming Test—30 item version [28]; the Trail Making Test A and B [29]; the Hooper Visual Organization Test [30]; and Verbal Fluency (FAS & FAS-Animal) [31,32]. These NP tests measure multiple cognitive domains of verbal memory, visual memory, attention and concentration, executive function, abstract reasoning, language, and visuoperceptual organization. For all tests, higher scores indicate better performance with the exception of the Trail Making Test A and B, where a shorter completion time indicates better cognitive performance.

    Since October 2011, FHS participants have been administered the dCDT with a digital pen during their regular NP test visit by trained examiners [10-13]. The digital pen measures the pen position 80 times per second at a spatial resolution of ±0.002 inches [10]. After the dCDT test, the pen was connected to a computer with preinstalled software that automatically extracted the drawing information and classified each pen stroke and associated latencies. An external rater manually examined each drawing to ensure appropriate classification of strokes using a user-friendly drag-and-drop interface to correct any classification error. A total of 105 dCDT features were derived and used from the entire drawing process, which captured the strokes, latencies, and spatial relationships as the measures of drawing efficiency, simple motor functioning, information processing, and spatial reasoning (Multimedia Appendix 1). As reported in our previous study [15], we further derived composite scores based on dCDT features significantly associated with 18 NP tests, which represented a weighted combination of dCDT features that were previously associated with a specific NP test.

    Acquisition and Measurement of MRI Variables

    The MRI protocol in the FHS has been described previously [33]. Briefly, participants were imaged using a Siemens 1.5T field strength machine (Siemens Medical) and a 3-dimensional T1-weighted coronal spoiled gradient-recalled echo sequence. All images were transferred to and processed by the University of California Davis Medical Center for centralized processing. Segmentation of brain structural MRI was performed by semiautomated procedures, and the complete information can be obtained elsewhere [33]. In brief, gray matter, white matter, and cerebrospinal fluid segmentation were performed using an expectation-maximization algorithm after skull stripping and intensity inhomogeneity correction. The hippocampus was segmented by a multi-atlas hippocampal segmentation algorithm [34]. White matter hyperintensity (WMH) was segmented on a combination of FLAIR and 3D T1 images using a modified Bayesian probability structure [35]. Total cerebral cranial volume (TCV) was determined by outlining the intracranial vault lying above the tentorium and was used for correcting the head size. The primary MRI measure was total cerebral brain volume (TCBV). Secondary measures were cerebral white matter volume, cerebral gray matter volume, hippocampal volume, and WMH volume. All MRI measures were corrected for head size by calculating the percentage of these volumes over the TCV. The percent of WMH/TCV was log-transformed for normality. The WMH volume was used both as a continuous variable and as a dichotomous variable (large WMH volume versus no or minimal WMH volume), similar to a previous study [36]. A large WMH volume (WMH-Large) was defined as a volume more than one standard deviation higher than the age-specific mean value.

    Case Ascertainment of MCI

    The detailed cognitive ascertainment procedure and quality control in the FHS have been described previously [37,38]. Briefly, participants received NP assessments on average every 4 to 5 years, in addition to their regular research center examinations. For participants with possible cognitive impairment, more frequent NP tests were conducted on average every 1 to 2 years, and neurological examinations were performed on a subset of these participants. A clinical review was triggered when there was an indication of a potential cognitive impairment and/or decline, and was conducted by a panel consisting of at least one neurologist and one neuropsychologist. The review panel determined the MCI diagnosis, which required evidence of a decline in cognitive performance in one or more cognitive domains, no records indicating functional decline, and not meeting the criteria for dementia. Although the Clinical Dementia Rating (CDR) scale [39] was not formally applied, the review panel used the CDR scoring scale (0-3) to quantify the severity of impairment; all MCI cases were given a rating of 0.5.

    Statistical Analysis

    The dCDT composite scores were derived by linear regression models based on dCDT features significantly associated with 18 NP tests, which represented a weighted combination of dCDT features that were previously associated with a specific NP test [15]. The scores and all MRI measures were normalized with a mean of 0 and standard deviation of 1. Linear regression models were used to assess the associations of dCDT composite scores with MRI measures, adjusting for age, sex, and education. The same models were also used to test the association between individual dCDT features and MRI measures. In the sensitivity analysis, the models were additionally adjusted for vascular risk factors, including hypertension, diabetes, smoking, and prevalent atrial fibrillation. Bonferroni correction was used to adjust for multiple testing in the linear regression models. Significant associations were claimed if P<.05/N, where N was the number of tests performed.

    Logistic regression models were also constructed to classify MCI using 4 different sets of predictors. The analysis was restricted to participants who were 65 years or older at the time of NP testing (only 1 case was diagnosed before 65 years). Model 1 included only age, sex, and education. Model 2 included age, sex, education, and MRI measures (TCBV, cerebral white matter volume, cerebral gray matter volume, and hippocampal volume). Model 3 included age, sex, education, and dCDT composite scores. Model 4 included age, sex, education, MRI measures, and dCDT composite scores. Model performance was assessed by the area under the curve (AUC). All statistical analyses were performed using R software version 4.0.3 (R Core Team) [40].


    As shown in Table 1, our study sample included 1656 participants (mean age 61 years, SD 13 years; 50.9% women; 42.9% received college-level education or higher) who were free of dementia or stroke when NP tests were conducted. Among them, 23 participants were diagnosed with MCI at the time of or before their NP tests but had not progressed to the threshold of clinical dementia diagnosis. As expected, participants with MCI were generally older and had worse cognitive performance, smaller cerebral volume measures, and larger WMH volume on MRI than those in the NC group.

    Table 1. Clinical characteristics of the study sample.
    VariableTotal (N=1656)MCIa (n=23)NCb (n=1633)
    Age (years), mean (SD)61 (13)78 (7)61 (13)
    Women, n (%)843 (50.9)14 (60.9)829 (50.8)
    Education, n (%)



    		No high school187 (11.3)3 (13.0)184 (11.3)

    		High school319 (19.3)6 (26.1)313 (19.2)

    		Some college440 (26.6)3 (13.0)437 (26.8)

    		College and higher710 (42.9)11 (47.8)699 (42.8)
    MRIc measuresd, mean (SD)



    		Total cerebral cranial volume (cm3)1251 (129)1252 (104)1251 (129)

    		Total cerebral brain volume (%)77.0 (2.9)72.8 (2.5)77.0 (2.9)

    		Cerebral white matter volume (%)36.3 (2.4)34.3 (2.7)36.3 (2.4)

    		Cerebral gray matter volume (%)40.4 (2.1)37.4 (2.2)40.4 (2.1)

    		Hippocampal volume (%)0.54 (0.05)0.50 (0.05)0.54 (0.05)

    		Log (white matter hyperintensity volume) (%)−2.33 (1.48)−0.59 (1.38)−2.35 (1.47)
    Neuropsychological test scores, mean (SD)



    		LMie13 (3)10 (4)13 (3)

    		LMde12 (4)8 (4)12 (4)

    		LMre10 (1)9 (2)10 (1)

    		VRif9 (3)5 (2)9 (3)

    		VRdf8 (3)3 (3)8 (3)

    		VRrf3 (1)2 (1)3 (1)

    		PASig15 (3)11 (3)15 (3)

    		PASdg9 (1)7 (2)9 (1)

    		PASrg10 (1)9 (2)10 (1)

    		DSfh7 (1)6 (1)7 (1)

    		DSbh5 (1)4 (1)5 (1)

    		Trails Ai (s)31 (13)44 (14)31 (13)

    		Trails Bi (s)83 (65)219 (159)82 (61)

    		Similarities17 (3)14 (5)17 (3)

    		Hooper Visual Organization Test26 (3)23 (4)26 (3)

    		Boston Naming Test—30 item version26 (7)24 (7)26 (7)

    		FASj41 (12)32 (12)41 (12)

    		FAS-Animalj19 (6)15 (5)19 (6)

    aMCI: mild cognitive impairment.

    bNC: normal cognition.

    cMRI: magnetic resonance imaging.

    dAll MRI measures were corrected for head size by calculating the percentage of the volumes over the total cerebral cranial volume above the tentorium. The percentage of white matter hyperintensity volume/total cerebral cranial volume was log transformed.

    eWechsler Memory Scale Logical Memory—Immediate Recall (LMi), Delayed Recall (LMd), and Recognition (LMr).

    fVisual Reproduction—Immediate Recall (VRi), Delayed Recall (VRd), and Recognition (VRr).

    gPaired Associate Learning—Immediate Recall (PASi), Delayed Recall (PASd), and Recognition (PASr).

    hWechsler Adult Intelligence Scale Digit Span—Forward (DSf) and Backward (DSb).

    iTrail Making Test A (Trails A) and B (Trails B).

    jVerbal fluency test (FAS and FAS-Animal).

    As shown in Table 2, all dCDT composite scores (n=18) were significantly associated with TCBV after adjusting for multiple testing. Better dCDT performance was associated with larger TCBV. The effect size of composite scores for visual memory and visuoperceptual organization tended to be slightly higher than other dCDT composite scores.

    Table 2. Association between digital Clock Drawing Test composite scores and total cerebral brain volume.
    dCDTa composite scorebEffect sizeStandard errorP valuec
    dCDT_LMid0.0790.0173.2×10−6
    dCDT_LMdd0.0830.0171.1×10−6
    dCDT_LMrd0.0610.0172.9×10−4
    dCDT_VRie0.0980.0171.4×10−8
    dCDT_VRde0.0990.0171.2×10−8
    dCDT_VRre0.0930.0176.4×10−8
    dCDT_PASif0.0790.0173.5×10−6
    dCDT_PASdf0.0840.0178.8×10−7
    dCDT_PASrf0.0730.0171.7×10−5
    dCDT_DSfg0.0710.0172.5×10−5
    dCDT_DSbg0.0670.0176.5×10−5
    dCDT_Trails Ah−0.0740.0171.2×10−5
    dCDT_Trails Bh−0.0800.0173.0×10−6
    dCDT_SIMi0.0840.0179.4×10−7
    dCDT_HVOTj0.0940.0176.1×10−8
    dCDT_BNT30k0.0750.0179.9×10−6
    dCDT_FASl0.0830.0171.5×10−6
    dCDT_FAS-Animall0.0620.0172.2×10−4

    adCDT: digital Clock Drawing Test.

    bThe models were adjusted for age, sex, and education.

    cBonferroni correction was used to adjust for multiple testing, and significant associations were claimed if P<.05/18 (2.8×10−3), where 18 is the number of tests performed. All P values were significant.

    dWechsler Memory Scale Logical Memory—Immediate Recall (LMi), Delayed Recall (LMd), and Recognition (LMr).

    eVisual Reproduction—Immediate Recall (VRi), Delayed Recall (VRd), and Recognition (VRr).

    fPaired Associate Learning—Immediate Recall (PASi), Delayed Recall (PASd), and Recognition (PASr).

    gWechsler Adult Intelligence Scale Digit Span—Forward (DSf) and Backward (DSb).

    hTrail Making Test A (Trails A) and B (Trails B).

    iSIM: Similarities.

    jHVOT: Hooper Visual Organization Test.

    kBNT30: Boston Naming Test—30 item version.

    lVerbal fluency test (FAS and FAS-Animal).

    As shown in Table 3, cerebral white matter volume was significantly associated with 11 dCDT composite scores involving the comprehensive cognitive domains of verbal memory, visual memory, abstract reasoning, language, and visuoperceptual organization (all with P<2.8×10−3). In contrast, cerebral gray matter volume was only associated with the visual memory composite score (dCDT_Visual Reproduction—Immediate Recall, β=.059; P=2.6×10−3). Hippocampal volume was not significantly associated with any dCDT composite scores. As shown in Table 4, none of the dCDT composite scores was significantly associated with WMH volume.

    Table 3. Association of digital Clock Drawing Test composite scores with cerebral white matter, gray matter, and hippocampal volumes.
    dCDTa composite scorebCerebral white matter volumecCerebral gray matter volumecHippocampal volumec

    		Effect sizeStandard errorP valuedEffect sizeStandard errorP valuedEffect sizeStandard errorP valued
    dCDT_LMie0.0710.0231.9×10−3f0.0420.0192.8×10−20.0330.0252.0×10−1
    dCDT_LMde0.0750.0231.1×10−3f0.0450.0192.0×10−20.0380.0251.3×10−1
    dCDT_LMre0.0520.0222.0×10−20.0340.0197.1×10−20.0190.0254.6×10−1
    dCDT_VRig0.0830.0234.2×10−4f0.0590.0202.6×10−3f0.0540.0263.8×10−2
    dCDT_VRdg0.0840.0233.0×10−4f0.0580.0203.1×10−30.0540.0263.9×10−2
    dCDT_VRrg0.0800.0235.6×10−4f0.0540.0205.5×10−30.0520.0264.3×10−2
    dCDT_PASih0.0700.0232.3×10−3f0.0440.0192.2×10−20.0350.0251.7×10−1
    dCDT_PASdh0.0780.0237.3×10−4f0.0440.0192.4×10−20.0410.0261.1×10−1
    dCDT_PASrh0.0670.0233.2×10−30.0360.0196.2×10−20.0470.0256.5×10−2
    dCDT_DSfi0.0630.0235.7×10−30.0410.0193.4×10−20.0260.0253.1×10−1
    dCDT_DSbi0.0590.0239.0×10−30.0370.0195.0×10−20.0230.0253.7×10−1
    dCDT_Trails Aj−0.0630.0236.0×10−3−0.0450.0191.8×10−2−0.0460.0257.0×10−2
    dCDT_Trails Bj−0.0690.0232.9×10−3−0.0470.0191.4×10−2−0.0470.0256.7×10−2
    dCDT_SIMk0.0710.0232.0×10−3f0.0490.0191.2×10−20.0340.0261.8×10−1
    dCDT_HVOTl0.0830.0234.0×10−4f0.0520.0208.0×10−30.0480.0266.4×10−2
    dCDT_BNT30m0.0710.0231.9×10−3f0.0360.0195.9×10−20.0280.0252.7×10−1
    dCDT_FASn0.0710.0232.1×10−3f0.0470.0191.5×10−20.0400.0261.2×10−1
    dCDT_FAS-Animaln0.0540.0231.7×10−20.0340.0197.1×10−20.0210.0254.0×10−1

    adCDT: digital Clock Drawing Test.

    bThe models were adjusted for age, sex, and education.

    cAll magnetic resonance imaging measures were the percentage of the volumes over the total cerebral cranial volume above the tentorium.

    dBonferroni correction was used to adjust for multiple testing, and significant associations were claimed if P<.05/18 (2.8×10−3), where 18 is the number of tests performed.

    eWechsler Memory Scale Logical Memory—Immediate Recall (LMi), Delayed Recall (LMd), and Recognition (LMr).

    fSignificant.

    gVisual Reproduction—Immediate Recall (VRi), Delayed Recall (VRd), and Recognition (VRr).

    hPaired Associate Learning—Immediate Recall (PASi), Delayed Recall (PASd), and Recognition (PASr).

    iWechsler Adult Intelligence Scale Digit Span—Forward (DSf) and Backward (DSb).

    jTrail Making Test A (Trails A) and B (Trails B).

    kSIM: Similarities.

    lHVOT: Hooper Visual Organization Test.

    mBNT30: Boston Naming Test—30 item version.

    nVerbal fluency test (FAS and FAS-Animal).

    Table 4. Association between digital Clock Drawing Test composite scores and white matter hyperintensity volume.
    dCDTa composite scorebWhite matter hyperintensity volumecWhite matter hyperintensity-Larged

    		Effect sizeStandard errorP valueeEffect sizeStandard errorP valuee
    dCDT_LMif0.0400.0183.0×10−2−0.1200.0731.1×10−1
    dCDT_LMdf−0.0410.0182.5×10−2−0.1300.0738.2×10−2
    dCDT_LMrf−0.0350.0185.0×10−2−0.1100.0711.3×10−1
    dCDT_VRig−0.0420.0192.3×10−2−0.1500.0743.7×10−2
    dCDT_VRdg−0.0430.0192.0×10−2−0.1500.0743.7×10−2
    dCDT_VRrg−0.0430.0192.2×10−2−0.1600.0742.9×10−2
    dCDT_PASih−0.0360.0185.0×10−2−0.1000.0731.5×10−1
    dCDT_PASdh−0.0400.0183.0×10−2−0.1300.0738.0×10−2
    dCDT_PASrh−0.0200.0182.7×10−1−0.0350.0736.3×10−1
    dCDT_DSfi−0.0420.0182.2×10−2−0.1600.0723.0×10−2
    dCDT_DSbi−0.0360.0184.7×10−2−0.1000.0721.5×10−1
    dCDT_Trails Aj0.0370.0184.4×10−20.1600.0733.1×10−2
    dCDT_Trails Bj0.0390.0183.4×10−20.1500.0743.6×10−2
    dCDT_SIMk−0.0390.0183.5×10−2−0.1300.0736.6×10−2
    dCDT_HVOTl−0.0400.0193.4×10−2−0.1400.0745.7×10−2
    dCDT_BNT30m−0.0370.0184.5×10−2−0.1200.0739.8×10−2
    dCDT_FASn−0.0410.0182.5×10−2−0.1600.0743.2×10−2
    dCDT_FAS-Animaln−0.0350.0184.8×10−2−0.1100.0711.3×10−1

    adCDT: digital Clock Drawing Test.

    bThe models were adjusted for age, sex, and education.

    cThe white matter hyperintensity volume was the percentage over the total cerebral cranial volume above the tentorium and was log transformed.

    dThe large white matter hyperintensity volume was defined as that more than one standard deviation higher than the age-specific mean value.

    eBonferroni correction was used to adjust for multiple testing, and significant associations were claimed if P<.05/18 (2.8×10−3), where 18 is the number of tests performed.

    fWechsler Memory Scale Logical Memory—Immediate Recall (LMi), Delayed Recall (LMd), and Recognition (LMr).

    gVisual Reproduction—Immediate Recall (VRi), Delayed Recall (VRd), and Recognition (VRr).

    hPaired Associate Learning—Immediate Recall (PASi), Delayed Recall (PASd), and Recognition (PASr).

    iWechsler Adult Intelligence Scale Digit Span—Forward (DSf) and Backward (DSb).

    jTrail Making Test A (Trails A) and B (Trails B).

    kSIM: Similarities.

    lHVOT: Hooper Visual Organization Test.

    mBNT30: Boston Naming Test—30 item version.

    nVerbal fluency test (FAS and FAS-Animal).

    Further detailed analysis for cortical gray matter and specific brain areas (frontal, parietal, temporal, and occipital cortical gray matter volumes) is shown in Multimedia Appendix 2. We found that cortical gray matter volume was associated with visual memory composite scores (P=1.4×10−3 for dCDT_Visual Reproduction—Immediate Recall and 1.7×10−3 for dCDT_Visual Reproduction—Delayed Recall), which was consistent with the results for cerebral gray matter volume shown in Table 3. For specific cortical regions, parietal and temporal cortical gray matter volumes were associated with dCDT composite scores, whereas no association was found for frontal and occipital cortical gray matter volumes. The results remained largely the same after additionally adjusting for vascular risk factors (Multimedia Appendix 3, Multimedia Appendix 4, Multimedia Appendix 5, and Multimedia Appendix 6).

    We also examined the associations between individual dCDT features and MRI measures. Among the 105 dCDT features, 17 of them were significantly associated with TCBV. However, only 2 features were associated with cerebral white matter volume and 4 features were associated with cerebral gray matter volume (Multimedia Appendix 7). The scattered associations suggest that, rather than individual features, the composite scores can better characterize the combinatory effects of multiple features.

    We then built 4 classification models of MCI for participants over 65 years old, each with a different set of predictors. As shown in Figure 1, the AUCs were 0.755, 0.840, 0.859, and 0.897 for Models 1-4, respectively. Model 4 that incorporated traditional risk factors (age, sex, and education), MRI measures, and dCDT composite scores showed superior performance compared with models consisting of solely MRI or dCDT composite scores with traditional risk factors (Models 1-3).

    ‎
    Figure 1. Performance of classification models to distinguish mild cognitive impairment from normal cognition. AUC: area under the curve.
    View this figure

    In this study, we investigated the potential of the dCDT as a cognitive assessment tool by studying the association of dCDT features with brain MRI structural measures in more than 1600 participants. We observed significant associations between dCDT composite scores and multiple MRI measures, including TCBV, cerebral white matter volume, and cerebral gray matter volume. However, no association was observed between dCDT composite scores and hippocampal volume or WMH volume. The combination of dCDT composite scores with MRI measures and clinical risk factors reached an AUC of 0.897 to distinguish MCI from normal cognition. Given that standard NP tests are time burdensome and labor intensive, the dCDT might be used as an alternative tool to screen for MCI in a large population [12,13,41,42].

    A previous study found that a smaller TCBV was associated with worse performance in Visual Reproduction—Immediate Recall, Visual Reproduction—Delayed Recall, Visual Reproduction—Recognition, the Trail Making Test A and B, and the Hooper Visual Organization Test [16]. Notably, this study found that global cerebrum atrophy was associated with several dCDT measures, suggesting a more extensive deficit in multiple cognitive domains, which might be used as an indicator of general “brain health.” Interestingly, dCDT performance tended to be better correlated with cerebral white matter volume, and parietal and temporal cortical gray matter volumes, suggesting that they may be the primary neuroanatomical basis for cognitive processes in the dCDT. While the cerebral cortex has long been considered as the major neuroanatomical basis of cognitive function, white matter is increasingly recognized as equally critical for cognition [43-45]. Cerebral white matter contains neural fibers that are the extensions of neurons into subcortical regions [46]. The neural network composed of cortical neurons and subcortical neural fibers is essential to maintain normal neural signal transmission and functional connectivity for cognitive processing [46]. Nonetheless, although there was a significant association between the dCDT and cerebral white matter volume, we did not find an association between dCDT performance and WMH volume. One possible reason is the exclusion of dementia patients in our sample, who generally have a larger WMH volume compared to that in individuals with normal cognition or MCI. Another reason may be that the dCDT performance was more closely associated with white matter atrophy rather than localized white matter injuries.

    Several limitations of our study, however, merit consideration. First, the cross-sectional design could not reveal a temporal relationship between dCDT features and MRI measures. Future longitudinal analysis will investigate whether dCDT features could serve as early cognitive markers for predicting incident brain structural changes or incident MCI or dementia. Second, only a moderate number of MCI cases were observed in this study, and the number of dementia cases was too small to study. The diagnostic value of the dCDT could be further validated when more incident cases are observed during longer follow-up periods in the FHS. In addition, besides the Harvard Aging Brain Study, future studies with larger sample sizes of MCI and early Alzheimer dementia in more diverse populations (population-based or hospital-based samples) are warranted to further test the diagnostic performance before extensive application of the dCDT as a substitute for classic NP tests in classification and selection criteria for clinical research and clinical trials. Third, standard NP test performance was used during the dementia diagnosis review process, whereas dCDT composite scores were built from standard NP tests. However, dCDT performance was not directly used during the review process. Finally, yet importantly, FHS participants were mostly non-Hispanic White and native English speakers. Therefore, the generalizability of our findings to other ethnic populations is unknown.

    Aside from these limitations, our study had several strengths. This is the first study to investigate the association between dCDT features and MRI measures in a population-based evaluation. All the NP tests and MRI measures were collected consistently with rigorous quality control. The MCI diagnosis was made by an experienced review panel, which capitalized on all available relevant data sources for consistent diagnosis. Further, given the current relatively widespread use of the paper-pencil clock drawing test as a cognitive screening tool in clinical practice, results from this study do support the potential use of a digital version in place of the paper-pencil version. This application may be of particular use in clinical settings where neuropsychological and neurological expertise is unavailable.

    In summary, we assessed the psychometric characteristics of the dCDT with brain volume measures by MRI. In combination with age, sex, education, and MRI measures, the dCDT improved the classification performance of MCI similar to standard NP tests. Our results suggest that the dCDT may be used as a cost-effective and easy-to-administer tool by general practitioners to assess subtle cognitive changes occurring in MCI or early dementia stages and in underdeveloped countries or regions where medical resources are limited.

    Acknowledgments

    We thank the Framingham Heart Study participants for their decades of dedication and the staff for their hard work in collecting and preparing the data. This work was supported by the Framingham Heart Study’s National Heart, Lung, and Blood Institute contract (N01-HC-25195; HHSN268201500001I), and National Institutes of Health grants from the National Institute on Aging (AG008122, AG016495, AG033040, AG054156, AG049810, AG062109, and U01AG068221) and Pfizer. This work was also supported by the Alzheimer’s Association Grant (AARG-NTF-20-643020) and American Heart Association Grant (20SFRN35360180). Support for JY was provided by the Chinese Academy of Medical Sciences Innovation Fund for Medical Sciences (2016I2M1004). The views expressed in this manuscript are those of the authors and do not necessarily represent the views of the National Institutes of Health or the US Department of Health and Human Services.

    Authors' Contributions

    Concept and design: HL, RA, and JY; Acquisition, analysis, or interpretation of data: HL, RA, JY, CK, AFAA, SD, and SA; Drafting of the manuscript: JY and HL; Statistical analysis: HL; Administrative, technical, or material support: AA; Supervision: HL and RA. All authors critically revised the manuscript for important intellectual content.

    Conflicts of Interest

    RA is a scientific advisor at Signant Health and a consultant at Biogen. There is no declaration from the other authors.

    ‎
    Multimedia Appendix 1

    Raw feature names and descriptions.

    DOCX File , 29 KB
    ‎
    Multimedia Appendix 2

    Association between digital Clock Drawing Test composite scores and cortical gray matter volumes.

    DOCX File , 38 KB
    ‎
    Multimedia Appendix 3

    Association between digital Clock Drawing Test composite scores and total cerebral brain volume after additionally adjusting for vascular risk factors.

    DOCX File , 18 KB
    ‎
    Multimedia Appendix 4

    Association of digital Clock Drawing Test composite scores with cerebral white matter, gray matter, and hippocampal volumes after additionally adjusting for vascular risk factors.

    DOCX File , 21 KB
    ‎
    Multimedia Appendix 5

    Association between digital Clock Drawing Test composite scores and white matter hyperintensity volume after additionally adjusting for vascular risk factors.

    DOCX File , 19 KB
    ‎
    Multimedia Appendix 6

    Association between digital Clock Drawing Test composite scores and cortical gray matter after additionally adjusting for vascular risk factors.

    DOCX File , 24 KB
    ‎
    Multimedia Appendix 7

    Association between individual digital Clock Drawing Test features and magnetic resonance imaging measures.

    DOCX File , 212 KB
    1. Ahmed S, Brennan L, Eppig J, Price CC, Lamar M, Delano-Wood L, et al. Visuoconstructional Impairment in Subtypes of Mild Cognitive Impairment. Appl Neuropsychol Adult 2016 Mar;23(1):43-52 [FREE Full text] [CrossRef] [Medline]
    2. Cosentino S, Jefferson A, Chute DL, Kaplan E, Libon DJ. Clock drawing errors in dementia: neuropsychological and neuroanatomical considerations. Cogn Behav Neurol 2004 Jun;17(2):74-84. [CrossRef] [Medline]
    3. Libon DJ, Malamut BL, Swenson R, Sands LP, Cloud BS. Further analyses of clock drawings among demented and nondemented older subjects. Arch Clin Neuropsychol 1996;11(3):193-205. [Medline]
    4. Agrell B, Dehlin O. The clock-drawing test. Age Ageing 1998;27(3):399-403. [CrossRef]
    5. Pinto E, Peters R. Literature review of the Clock Drawing Test as a tool for cognitive screening. Dement Geriatr Cogn Disord 2009 Feb 19;27(3):201-213 [FREE Full text] [CrossRef] [Medline]
    6. Libon D, Swenson R, Barnoski E, Sands L. Clock drawing as an assessment tool for dementia. Archives of Clinical Neuropsychology 1993 Sep;8(5):405-415. [CrossRef]
    7. Supasitthumrong T, Herrmann N, Tunvirachaisakul C, Shulman K. Clock drawing and neuroanatomical correlates: A systematic review. Int J Geriatr Psychiatry 2019 Feb;34(2):223-232. [CrossRef] [Medline]
    8. Matsuoka T, Narumoto J, Shibata K, Okamura A, Nakamura K, Nakamae T, et al. Neural correlates of performance on the different scoring systems of the clock drawing test. Neurosci Lett 2011 Jan 10;487(3):421-425. [CrossRef] [Medline]
    9. Thomann PA, Toro P, Dos Santos V, Essig M, Schröder J. Clock drawing performance and brain morphology in mild cognitive impairment and Alzheimer's disease. Brain Cogn 2008 Jun;67(1):88-93. [CrossRef] [Medline]
    10. Davis R, Penney D, Pittman D, Libon D, Swenson R, Kaplan E. The Digital Clock Drawing Test (dCDT) - I: Development of a new computerized quantitative system. 2011 Presented at: 39th Annual Meeting of the International Neuropsychological Society; February 2-5, 2011; Boston, MA   URL: http://rationale.csail.mit.edu/clocksketch/Publications/INS2011-dCDTI.pdf
    11. Penney D, Davis R, Libon D, Lamar M, Price C, Swenson R, et al. The Digital Clock Drawing Test (dCDT) - II: A new computerized quantitative system. 2011 Presented at: 39th Annual Meeting of the International Neuropsychological Society; February 2-5, 2011; Boston, MA   URL: http://rationale.csail.mit.edu/clocksketch/Publications/INS2011-dCDTII.pdf
    12. Davis R, Libon D, Au R, Pitman D, Penney D. THink: Inferring Cognitive Status from Subtle Behaviors. AIMag 2015 Sep 28;36(3):49-60. [CrossRef]
    13. Souillard-Mandar W, Davis R, Rudin C, Au R, Libon DJ, Swenson R, et al. Learning Classification Models of Cognitive Conditions from Subtle Behaviors in the Digital Clock Drawing Test. Mach Learn 2016 Mar;102(3):393-441 [FREE Full text] [CrossRef] [Medline]
    14. Rentz DM, Papp KV, Mayblyum DV, Sanchez JS, Klein H, Souillard-Mandar W, et al. Association of Digital Clock Drawing With PET Amyloid and Tau Pathology in Normal Older Adults. Neurology 2021 Apr 06;96(14):e1844-e1854 [FREE Full text] [CrossRef] [Medline]
    15. Yuan J, Libon DJ, Karjadi C, Ang AFA, Devine S, Auerbach SH, et al. Association Between the Digital Clock Drawing Test and Neuropsychological Test Performance: Large Community-Based Prospective Cohort (Framingham Heart Study). J Med Internet Res 2021 Jun 08;23(6):e27407 [FREE Full text] [CrossRef] [Medline]
    16. Seshadri S, Wolf PA, Beiser A, Elias MF, Au R, Kase CS, et al. Stroke risk profile, brain volume, and cognitive function: the Framingham Offspring Study. Neurology 2004 Nov 09;63(9):1591-1599. [CrossRef] [Medline]
    17. Debette S, Seshadri S, Beiser A, Au R, Himali JJ, Palumbo C, et al. Midlife vascular risk factor exposure accelerates structural brain aging and cognitive decline. Neurology 2011 Aug 02;77(5):461-468 [FREE Full text] [CrossRef] [Medline]
    18. Libon DJ, Preis SR, Beiser AS, Devine S, Seshadri S, Wolf PA, et al. Verbal memory and brain aging: an exploratory analysis of the role of error responses in the Framingham Study. Am J Alzheimers Dis Other Demen 2015 Sep;30(6):622-628 [FREE Full text] [CrossRef] [Medline]
    19. Fletcher E, Raman M, Huebner P, Liu A, Mungas D, Carmichael O, et al. Loss of fornix white matter volume as a predictor of cognitive impairment in cognitively normal elderly individuals. JAMA Neurol 2013 Nov 01;70(11):1389-1395 [FREE Full text] [CrossRef] [Medline]
    20. Hugenschmidt CE, Peiffer AM, Kraft RA, Casanova R, Deibler AR, Burdette JH, et al. Relating imaging indices of white matter integrity and volume in healthy older adults. Cereb Cortex 2008 Feb;18(2):433-442. [CrossRef] [Medline]
    21. Mahmood SS, Levy D, Vasan RS, Wang TJ. The Framingham Heart Study and the epidemiology of cardiovascular disease: a historical perspective. The Lancet 2014 Mar 15;383(9921):999-1008 [FREE Full text] [CrossRef] [Medline]
    22. Tsao CW, Vasan RS. Cohort Profile: The Framingham Heart Study (FHS): overview of milestones in cardiovascular epidemiology. Int J Epidemiol 2015 Dec;44(6):1800-1813 [FREE Full text] [CrossRef] [Medline]
    23. Wolf PA. Contributions of the Framingham Heart Study to stroke and dementia epidemiologic research at 60 years. Arch Neurol 2012 May;69(5):567-571 [FREE Full text] [CrossRef] [Medline]
    24. For Researchers. The Framingham Heart Study.   URL: https://framinghamheartstudy.org/fhs-for-researchers/ [accessed 2022-04-02]
    25. Au R, Seshadri S, Wolf PA, Elias MF, Elias PK, Sullivan L, et al. New norms for a new generation: cognitive performance in the framingham offspring cohort. Exp Aging Res 2004 Oct;30(4):333-358. [CrossRef] [Medline]
    26. Wechsler D, Stone C. Wechsler Memory Scale (WMS). New York, NY: The Psychological Corporation; 1948.
    27. Wechsler D. Wechsler Adult Intelligence Scale (WAIS). New York, NY: The Psychological Corporation; 1955.
    28. Kaplan E, Goodglass H, Weintraub S, Segal O. Boston Naming Test. Philadephia, PA: Lea & Febiger; 1983.
    29. Reitan RM. Validity of the Trail Making Test as an indicator of organic brain damage. PMS 1958 Dec;8(7):271. [CrossRef]
    30. Hooper H. The Hooper Visual Organization Test: Manual. Los Angeles, CA: Western Psychological Service; 1958.
    31. Benton A, Hamsher K, Sivan A. Multilingual Aphasia Examination. Iowa City, IA: University of Iowa; 1976.
    32. Goodglass H, Kaplan E. The Assessment of Aphasia and Related Disorders. Philadelphia, PA: Lea & Febiger; 1983.
    33. DeCarli C, Massaro J, Harvey D, Hald J, Tullberg M, Au R, et al. Measures of brain morphology and infarction in the framingham heart study: establishing what is normal. Neurobiol Aging 2005 Apr;26(4):491-510. [CrossRef] [Medline]
    34. Sanborn V, Preis SR, Ang A, Devine S, Mez J, DeCarli C, et al. Association Between Leptin, Cognition, and Structural Brain Measures Among "Early" Middle-Aged Adults: Results from the Framingham Heart Study Third Generation Cohort. J Alzheimers Dis 2020;77(3):1279-1289 [FREE Full text] [CrossRef] [Medline]
    35. DeCarli C, Fletcher E, Ramey V, Harvey D, Jagust WJ. Anatomical mapping of white matter hyperintensities (WMH): exploring the relationships between periventricular WMH, deep WMH, and total WMH burden. Stroke 2005 Jan;36(1):50-55 [FREE Full text] [CrossRef] [Medline]
    36. Au R, Massaro JM, Wolf PA, Young ME, Beiser A, Seshadri S, et al. Association of white matter hyperintensity volume with decreased cognitive functioning: the Framingham Heart Study. Arch Neurol 2006 Feb;63(2):246-250. [CrossRef] [Medline]
    37. Seshadri S, Wolf PA, Beiser A, Au R, McNulty K, White R, et al. Lifetime risk of dementia and Alzheimer's disease. The impact of mortality on risk estimates in the Framingham Study. Neurology 1997 Dec;49(6):1498-1504. [CrossRef] [Medline]
    38. Au R, Piers RJ, Devine S. How technology is reshaping cognitive assessment: Lessons from the Framingham Heart Study. Neuropsychology 2017 Nov;31(8):846-861 [FREE Full text] [CrossRef] [Medline]
    39. Hughes CP, Berg L, Danziger WL, Coben LA, Martin RL. A new clinical scale for the staging of dementia. Br J Psychiatry 1982 Jun;140:566-572. [CrossRef] [Medline]
    40. R: A language and environment for statistical computing.   URL: https://www.R-project.org/ [accessed 2022-04-02]
    41. Dion C, Arias F, Amini S, Davis R, Penney D, Libon DJ, et al. Cognitive Correlates of Digital Clock Drawing Metrics in Older Adults with and without Mild Cognitive Impairment. J Alzheimers Dis 2020;75(1):73-83 [FREE Full text] [CrossRef] [Medline]
    42. Müller S, Herde L, Preische O, Zeller A, Heymann P, Robens S, et al. Diagnostic value of digital clock drawing test in comparison with CERAD neuropsychological battery total score for discrimination of patients in the early course of Alzheimer's disease from healthy individuals. Sci Rep 2019 Mar 05;9(1):3543 [FREE Full text] [CrossRef] [Medline]
    43. Price CC, Jefferson AL, Merino JG, Heilman KM, Libon DJ. Subcortical vascular dementia: integrating neuropsychological and neuroradiologic data. Neurology 2005 Aug 09;65(3):376-382 [FREE Full text] [CrossRef] [Medline]
    44. Libon DJ, Price CC, Giovannetti T, Swenson R, Bettcher BM, Heilman KM, et al. Linking MRI hyperintensities with patterns of neuropsychological impairment: evidence for a threshold effect. Stroke 2008 Mar;39(3):806-813. [CrossRef] [Medline]
    45. Filley CM, Fields RD. White matter and cognition: making the connection. J Neurophysiol 2016 Nov 01;116(5):2093-2104 [FREE Full text] [CrossRef] [Medline]
    46. Fields RD. White matter in learning, cognition and psychiatric disorders. Trends Neurosci 2008 Jul;31(7):361-370 [FREE Full text] [CrossRef] [Medline]

    ‎
    AUC: area under the curve
    CDR: Clinical Dementia Rating
    CDT: Clock Drawing Test
    dCDT: digital Clock Drawing Test
    FHS: Framingham Heart Study
    MCI: mild cognitive impairment
    MRI: magnetic resonance imaging
    NC: normal cognition
    NP: neuropsychological
    TCBV: total cerebral brain volume
    TCV: total cerebral cranial volume
    WMH: white matter hyperintensity

    Edited by R Kukafka; submitted 28.10.21; peer-reviewed by MJ Brouillette, I Tarnanas; comments to author 20.12.21; revised version received 08.02.22; accepted 13.03.22; published 15.04.22

    Copyright

    ©Jing Yuan, Rhoda Au, Cody Karjadi, Ting Fang Ang, Sherral Devine, Sanford Auerbach, Charles DeCarli, David J Libon, Jesse Mez, Honghuang Lin. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 15.04.2022.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.