This paper is in the following e-collection/theme issue:

Mobile Health (mhealth) (2764) mHealth for Wellness, Behavior Change and Prevention (2742) Skin Cancer and Melanoma Prevention (110)

Published on in Vol 20, No 4 (2018): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/9392, first published .
Use of Smartphones for Early Detection of Melanoma: Systematic Review

Use of Smartphones for Early Detection of Melanoma: Systematic Review

Use of Smartphones for Early Detection of Melanoma: Systematic Review

Authors of this article:

Cédric Rat1, 2 Author Orcid Image ;   Sandrine Hild1 Author Orcid Image ;   Julie Rault Sérandour1 Author Orcid Image ;   Aurélie Gaultier3 Author Orcid Image ;   Gaelle Quereux2, 4 Author Orcid Image ;   Brigitte Dreno2, 4 Author Orcid Image ;   Jean-Michel Nguyen2, 3 Author Orcid Image
Article Authors Cited by (78) Tweetations (48) Metrics

    Review

    1Department of General Practice, Faculty of Medicine, University of Nantes, Nantes, France

    2Unit 1232 – Team 2, Centre de Recherche en Cancérologie, French National Institute of Health and Medical Research, Nantes, France

    3Department of Epidemiology and Biostatistics, Nantes University Hospital, CHU Nantes, Nantes, France

    4Oncodermatology Department, Nantes University Hospital, CHU Nantes, Nantes, France

    Corresponding Author:

    Cédric Rat, MD, PhD

    Department of General Practice

    Faculty of Medicine

    University of Nantes

    1 rue Gaston Veil

    Nantes, 44000 Nantes

    France

    Phone: 33 6 61 86 79 53

    Email: cedric.rat@univ-nantes.fr


    Background: The early diagnosis of melanoma is associated with decreased mortality. The smartphone, with its apps and the possibility of sending photographs to a dermatologist, could improve the early diagnosis of melanoma.

    Objective: The aim of our review was to report the evidence on (1) the diagnostic performance of automated smartphone apps and store-and-forward teledermatology via a smartphone in the early detection of melanoma, (2) the impact on the patient’s medical-care course, and (3) the feasibility criteria (focusing on the modalities of picture taking, transfer of data, and time to get a reply).

    Methods: We conducted a systematic search of PubMed for the period from January 1, 2007 (launch of the first smartphone) to November 1, 2017.

    Results: The results of the 25 studies included 13 concentrated on store-and-forward teledermatology, and 12 analyzed automated smartphone apps. Store-and-forward teledermatology opens several new perspectives, such as it accelerates the care course (less than 10 days vs 80 days), and the related procedures were assessed in primary care populations. However, the concordance between the conclusion of a teledermatologist and the conclusion of a dermatologist who conducts a face-to-face examination depended on the study (the kappa coefficient range was .20 to .84, median κ=.60). The use of a dermoscope may improve the concordance (the kappa coefficient range was .29 to .87, median κ=.74). Regarding automated smartphone apps, the major concerns are the lack of assessment in clinical practice conditions, the lack of assessment in primary care populations, and their low sensitivity, ranging from 7% to 87% (median 69%). In this literature review, up to 20% of the photographs transmitted were of insufficient quality. The modalities of picture taking and encryption of the data were only partially reported.

    Conclusions: The use of store-and-forward teledermatology could improve access to a dermatology consultation by optimizing the care course. Our review confirmed the absence of evidence of the safety and efficacy of automated smartphone medical apps. Further research is required to determine quality criteria, as there was major variability among the studies.

    J Med Internet Res 2018;20(4):e135

    doi:10.2196/jmir.9392

    Keywords

    smartphone; melanoma; screening; teledermatology; telemedicine; mobile app 


    Background

    The incidence of melanoma has increased in all Western countries over the last 30 years and has increased 3 to 5 times depending on the country [1,2], presently affecting 13.8 people in North America, 14.6 in northern Europe and 35.1 out of the 100,000 people in Australia [1,2]. Melanoma 5-year survival depends on the stage at the time of diagnosis, decreasing by 84% at a localized stage to 13% at the metastatic stage [2]. It is therefore essential that clinicians and policy makers concentrate their efforts to ensure early detection of the disease [3]. Numerous factors associated with a delayed diagnosis are patient-related [4-12]. Other factors are related to the opportunity to consult a dermatologist rather than a general practitioner [4,13]. However, various authors have reported difficulties in obtaining an appointment with a dermatologist [12,14-17].

    Many countries have tested the use of telemedicine in dermatology as a way to increase access to health care services when distance is a critical factor [18-26]. Telemedicine in dermatology can be based either on videoconferences or on store-and-forward teledermatology procedures. Videoconferences, which allow a patient and a dermatologist to be connected for a consultation, are time-consuming for the dermatologist and may require an expensive setup [20,27]. Store-and-forward teledermatology procedures are based on sending information and photographs to a dermatologist for a deferred medical opinion [20].

    Although smartphones have now revolutionized the daily life of physicians in all Western countries [19,24,28,29], one issue is to determine whether smartphone use has been assessed for store-and-forward teledermatology procedures. As various apps exist that provide scores, decision aids, and management advice, an alternative both to videoconferences and to store-and-forward teledermatology might be automated smartphone apps with no need of a dermatologist opinion. An issue to be addressed is whether such apps can help in the early diagnosis of melanoma.

    Objective

    We initiated a review focusing on the use of a smartphone in sustaining melanoma early detection (either store-and-forward teledermatology or automated apps). The aim was to report evidence on (1) the diagnostic performance of the procedures, (2) the impact on the patient’s medical-care course and delays before the dermatological consultation, and (3) the limitations of either store-and-forward teledermatology or automated apps.


    This review was conducted according to the key steps required for systematic reviews [30]. Considering that evidence might be sparse, the literature review was based on a broad scope and was not restricted to randomized controlled trials.

    Study Identification and Selection

    We conducted a systematic search of PubMed for the period from January 1, 2007 (launch of the first smartphone) to November 1, 2017. The keywords were as follows: [smartphone OR cell phone OR remote OR telemedicine] AND [dermatology OR skin disease OR melanoma OR skin neoplasm OR skin abnormalities]. We also searched the reference lists of reviews and studies identified during the initial search by hand. Abstracts and full texts were reviewed independently by 2 reviewers (SH and JR) for inclusion. Any disagreements on inclusion or exclusion were resolved by consensus, and a third reviewer (CR) was consulted to resolve any remaining disagreements.

    Inclusion and Exclusion Criteria

    In this manuscript, the term “smartphones” refers to mobile phones that have an internet data communication system and a digital camera (compared to mobile phones that would not have these two specific devices). The inclusion criteria for the studies included in this review were as follows: (1) photographs concerning pigmented suspicious lesions, (2) photographs taken or analyzed using a smartphone, (3) patients older than 18 years, (4) studies written in French or English, and (5) abstracts available.

    The exclusion criteria were (1) no use of a smartphone, (2) the research area was not related to melanoma early detection, (3) dermatology teleconsultation in the form of a videoconference, (4) study based on histology, (5) studies consisting of assessing patients or caregivers’ preferences through interviews or surveys, and (6) an editorial or a letter to the editor.

    Considering that evidence might be sparse, the literature review was based on a broad scope and the inclusion criteria were not restricted to a “Patients Intervention Comparison Outcome” presentation [30].

    Data Extraction and Synthesis

    Studies were critically appraised by 2 reviewers (SH and JR), and discrepancies were resolved by consensus. The studies were first classified by the type of procedure assessed (app or store-and-forward teledermatology). Then the following data were extracted, such as design, population of the sample, whether the study had been conducted in the context of primary care, whether it was a descriptive or comparative study, and main outcome measures (Tables 1 and 2).

    The main outcomes in studies focusing on store-and-forward teledermatology were either (1) the diagnostic concordance between the teledermatology procedure and the reference (the kappa coefficient of concordance is a measure of agreement between 2 raters, based on the following formula [(observed probability–expected probability)/(1–expected probability)]) or (2) the impact on the patient’s medical-care course and delays before dermatological consultation. The proportion of uninterpretable photographs was also reported.

    The main outcomes in studies focusing on apps were sensitivity (defined as the number of true positive assessments/number of all positive assessments), specificity (defined as the number of true negative assessments/number of all negative assessments), and accuracy (defined as the number of correct assessments/number of all assessments). The proportion of uninterpretable photographs was also reported. When necessary, the authors were contacted to obtain information not reported in the studies.

    Analysis of Bias

    We assessed the risk of bias using the quality assessment of diagnostic accuracy 2nd edition (QUADAS-2) [55]. The reporting of risk of bias focused on patient selection, index test, reference standard, and flow and timing. For each item, signaling questions helped to estimate whether the risk of bias was low or high. Unclear was used if no information was available. The applicability of the study intervention was also assessed using QUADAS-2, focusing on patient selection, index test, and reference standard.

    Table 1. Studies based on store-and-forward teledermatology procedures (design, patients, comparison, and outcome). N/A: not applicable. RCT: randomized controlled trial.
    AuthorsDesignPatients,
    n (%)    		Population of sampleComparisonMain outcomes
    Store-and-forward teledermatology without teledermoscopy



    		Boyce et al, 2011 [31]Prospective studyN/APatients at an elevated risk of melanomaFace to faceConcordance

    		Lamel et al, 2011 [32]Prospective study1 (0.7)Patients from a melanoma screening campaignFace to faceConcordance

    		Janda et al, 2014 [33]RCT1 (1)Patients at an elevated risk of melanomaFace to faceConcordance
    Store-and-forward teledermatology that included teledermoscopy


    		Ford et al, 2015 [34]Quasi-experiment22 (11.3)Patients recruited in primary care centersFace to faceSecondary care referral

    		Börve et al, 2013 [29]Prospective study12 (17)Patients referred for an excisionFace to faceaConcordance

    		Börve et al, 2015 [35]Quasi-experiment55 (3.52)Patients recruited in primary care centersTeledermatology versus paper referralDelays

    		Hue et al, 2016 [36]Descriptive study1 (0.3)Patients from a melanoma screening campaignNo comparisonFeasibility and delays

    		Janda et al, 2013 [37]Descriptive studyN/APatients at an elevated risk of melanomaNo comparisonFeasibility

    		Kroemer at al, 2011 [38]Prospective study6 (5)Patients referred to the dermatologistFace to faceaConcordance

    		Manahan et al, 2015 [39]RCT0 (0.0)Patients with a dermatological follow-upFace to faceConcordance

    		Markun et al, 2017 [40]Prospective study1 (0.05)Patients from a melanoma screening campaignFace to faceaConcordance

    		Massone et al, 2007 [41]Prospective study2 (11)Patients with a dermatological follow-upFace to faceConcordance

    		Wu et al, 2015 [42]Prospective studyN/APatients with a dermatological follow-upFace to faceConcordance

    aFor these studies, suspicious lesions were referred for excision and histopathology results were analyzed as a secondary outcome in the study.

    Table 2. Studies based on automated smartphone apps (design, photographs and patients, comparison, and outcomes). N/A: not applicable.
    AuthorsDesignn (%)aPatients, N (characteristics)ComparisonMain outcomes
    Photographs issued from a database



    		Do et al, 2014 [43]Case-control study29 (36)N/AHistopathologyAccuracyb

    		Doukas et al, 2012 [44]Case-control study800 (26.67)N/AClinical evaluationAccuracyb

    		Ferrero et al, 2013 [45]Descriptive study93 (100)N/AHistopathologySensitivity, Specificity

    		Ramlakhan et al, 2011 [46]Case-control study46 (55)N/AUnclearSensitivity, Specificity

    		Wadhawan et al, 2011a [47]Case-control study388 (29.85)N/AHistopathologySensitivity, Specificity

    		Wadhawan et al, 2011b [48]Case-control study110 (31.7)N/AHistopathologySensitivity, Specificity

    		Wolf et al, 2013 [49]Case-control study60 (31.9)N/AHistopathologySensitivity, Specificity
    Photographs taken of patients in real condition



    		Dorairaj et al, 2017 [50]Prospective study9 (28)N/A (referred for an excision)TeledermatologistcSensitivity, Specificity

    		Maier et al, 2015 [51]Prospective study26 (18.1)N/A (with a dermatological follow-up)Face to facecSensitivity, Specificity

    		Ngoo et al, 2017 [52]Prospective study1 (2)30 (with a dermatological follow-up)TeledermatologistSensitivity, Specificity

    		Robson et al, 2012 [53]Prospective study2 (6)31 (referred to the dermatologist)Face to facecSensitivity, Specificity

    		Thissen et al, 2017 [54]Prospective study6 (1.8)256 (referred to the dermatologist)Face to facecSensitivity, Specificity

    aIncluded photographs, proportion with melanoma.

    bAccuracy=(True Negatives+True Positives)/(True Negatives+True Positives+False Negatives+False Positive).

    cFor these studies, suspicious lesions were referred for excision and histopathology results were analyzed as a secondary outcome in the study.


    Overview

    In total, 1450 titles and abstracts were screened for eligibility, utilizing the inclusion and exclusion criteria. A previous review was identified [2] and the related studies from the references list were included in this review. A total of 25 studies [29,31-54,56] were included in the review (Figure 1). Of these, 15 studies had been published as original papers [29,31,32,34-36,38-42,49,51,52,54], 5 were conference papers [43,44,46-48] and 5 were research letters [33,37,45,50,53]. In these, 12 studies had been conducted in European countries, that is, Great Britain [34,53], Austria [38,41], Sweden [29,35], Ireland [50], Germany [51], Switzerland [40], Greece [44], the Netherlands [54], and France [36]; 7 in United States of America [32,42,45-49]; 5 in Australia [31,33,37,39,52]; and one in Singapore [43].

    Store and Forward Teledermatology

    A total of 13 studies assessed store-and-forward teledermatology [29,31-42] (Table 1). There were 12 studies that specified the smartphone model used, that is, 9 tested iPhones [29,33-37,39,40,42] and 3 tested other brands of telephones [32,38,41].

    The study population were patients recruited in primary care in 9 studies, that is, either in the context of a screening campaign [32,36,40] or during targeted screening focusing on patients at an elevated risk of melanoma [31,33,37] or during opportunistic screening conducted in general practice [34,35,38]. For the other 4 studies, the patients had already consulted a dermatologist [29,39,41,42]. The prevalence of melanomas in the related populations varied greatly, ranging from 0% [39] to 17.3% [29].

    Ten studies compared the conclusion of the teledermatologist with the conclusion of a dermatologist who conducted a face-to-face examination (Table 1). There were 2 studies that focused on feasibility without providing any comparison [36,37]. From the 13 studies conducted, 10 studies used a mobile teledermoscope [29,34-42], whereas 3 studies only transferred the pictures taken without the teledermoscope [31-33].

    There were 7 studies that provided information on diagnostic concordance for store-and-forward teledermatology based solely on clinical photographs. The diagnostic concordance between the conclusions of the teledermatologist and the dermatologist (face-to-face) ranged from 62% [32] to 89% [41]. This concordance was analyzed further using the kappa coefficient [29,31,32,38-40,42], which ranged from .20 [40] to .84 [38]. Börve reported 58% concordance between the conclusions of 2 independent teledermatologists [29]. Focusing on whether the patients could take pictures of their lesions themselves, Boyce et al reported 69% concordance between the conclusion of a dermatologist (face-to-face) and the conclusion of a teledermatologist (κ=.23) [31].

    ‎
    Figure 1. Flowchart of studies identified in this systematic review focusing on the use of smartphones in the early detection of melanoma. IMRAD: Introduction, Methods, Results, And Discussion.
    View this figure

    For teledermatology based on pictures taken with a teledermoscope, the diagnostic concordance between the teledermatologist’s conclusions and the conclusion of a dermatologist (face-to-face) ranged from 51% [29] to 97% [42]. The kappa coefficient varied from .29 [40] to .87 [42]. Massone reported 94% concordance between the conclusions of 2 independent teledermatologists who analyzed photographs taken using a teledermoscope [41]. Focusing on whether the patients could take pictures of their lesions themselves, Manahan and Wu reported concordance ranging from 90% [39] to 97% [42] when the patient used a teledermoscope.

    There were 4 studies that reported an acceleration in the management of patients when malignancy was suspected [29,34,36,40]. Börve reported reduced delays in obtaining an appointment (delays shorter than 2 days compared with delays longer than 80 days), a reduced delay for surgical management (36 days vs 85 days), and a lower Breslow index at the time of the diagnosis [29]. Hue reported that patients with a highly suspicious lesion were asked to return within less than 10 days [36]. There were 3 studies [35,36,40] that reported a decrease (40%, 53%, and 74%, respectively) in the proportion of patients referred to a dermatologist, whereas Ford reported a slight increase in referrals (an increase of 2.11 per 1000 patients) [34].

    The proportion of uninterpretable images due to their poor quality was, on average, less than 20% [29,40,42]. Only Massone et al (2007) reported a higher percentage of poor-quality images of 70% [41]. However, only a minority of authors provided information on the modalities for picture taking. A total of 4 studies specified the size of the pictures from 1024×766 to 2592×1224 pixels. Following contact with the authors, the following information on the modalities were collected, that is, the number of pictures could vary from 1 to 12 pictures per lesion [33,39], pictures were taken at a distance of 10 to 30 cm from the skin [29,40], authors reported taking one close-up picture and another of the surrounding area [36,40,41], and specified the lighting conditions (“strong light” [39], “day light” [41], “maximal light” [29]), neutral background [29,40], use of the zoom [41] or macro mode [40,41], and use of the autofocus [29,38,41]). There were 4 authors who reported the time required to take the picture that is, ranging from a few seconds to less than 4 min [29,34-36]. The photographer was a professional in most cases, that is, either a dermatologist [29,38,41], a general practitioner [35], or another professional [32,40]; however, 6 studies reported that the picture was taken by the patient or a family member [31,33,37,39,42]. The notion of encrypting data was not approached systematically. Following individual contact, 8 authors reported encryption of the data, either through the app or through anonymity [29,31,32,34-36,39,40,42]. A total of 8 studies used email for transferring the data [29,31,33-35,37,39,42], and 7 used an encrypted platform [29,32,34-36,40,42].

    Analysis of bias is provided in Table 3. The risk of bias related to patient selection was high, as patients who participated were either volunteers or chosen by doctors in consultation, but no study was based on a random sample. In 2 studies, the same dermatologist participated in both, the store-and-forward teledermatology procedure and the face-to-face clinical evaluation, with insufficient washout, so that the risk of bias related to the test index was high. The final analysis of the studies did not include all the recruited patients so that the risk of bias of flow and timing was high. Applicability was good.

    Automated Smartphone Apps

    A total of 12 studies assessed the performance of automated smartphone apps [43-54] (Table 2), that is, 1 study compared the conclusions of an automated app with the conclusions of a dermatologist who conducted a face-to-face examination [51], 1 study compared the conclusions of an automated app with the conclusions of a teledermatologist [52], 7 studies analyzed the images of an already classified data bank [43-49], and the 3 last ones compared the conclusion of an automated app, both with the pathological report (after excision of the lesion) and with the conclusion of teledermatologists [50,53,54].

    Among the 5 studies based on taking a photograph [51-54], 4 tested iPhones [51-54], 1 tested other brands of telephones [52], and the brand of the telephone was not specified in the last study [50]. The photographer was a dermatologist [53,54] or another professional [50-52].

    Participants who were recruited in studies assessing automated smartphone apps were highly selected. Among the 8 studies based on photographs issued from a database, the proportion of melanoma ranged from 26.7% [44] to 100% [45]. Among the 5 studies based on taking a photograph, 2 studies included patients from a primary-care setting recruited during an opportunistic screening campaign conducted in general practice [53,54]. For the other 3 studies that included patients, the patients had already consulted a dermatologist [50-52]. The prevalence of melanomas in the related populations ranged from 1.8% [52,54] to 28.1% [50].

    The performance of the automated smartphone apps were assessed by referring either to their capacity to classify the lesions at risk [43,45,47,49-54], or to their diagnostic capacity [44,46,48] (Table 2). The references used could be either the histology results in 5 studies [43,45,47-49], the teledermatologist’s conclusion in 2 studies [50,52], or the dermatologist (face-to-face) conclusion in 3 studies [51,53,54]. For 2 studies, the authors referred to histology-based diagnosis without describing how they obtained the histopathological conclusion [44,46].

    A total of 5 studies assessed the apps used in real conditions [50-54]. In 4 studies, the images came from medical [43,45,49] or commercial [47,48] data banks, whereas no details on the photograph data banks were provided for 2 studies [44,46]. The sensitivity ranged from 7% to 87% [48,49], and the specificity ranged from 9% to 100% [50,52]. Only 2 studies described the area under the curve [44,48], providing a better comparison of results. One study reported a kappa coefficient of concordance between the opinion of the app and that of a dermatologist [52].

    None of the studies that assessed automated smartphone apps reported an impact on the patient’s medical-care course.

    Analysis provided by automated smartphone apps were made difficult by ulcerated, blood stained, speckled or tanned areas, the presence of hair, or several lesions on the same photograph. There were 5 studies that had a proportion ranging from 11% to 30% of the lesions that could not be analyzed because of technical problems other than the problems related to the quality of the initial photograph [45,49-51,53]. There were 4 studies based on apps which reported that the time required to analyze the pictures was less than 10 seconds [44,46-48]. The notion of encrypting data was not approached systematically. Following individual contact, 2 authors [52-54] reported encryption of the data, either through the app or through anonymity.

    The analysis of bias is provided in Table 4. For studies based on photographs from databank, references were unknown and the risks of bias for patient selection were high. Applicability related to the patient selection was highly concerning. Automated apps had no information on the diagnosis so that the risk of bias related to the test index was low. In 3 studies, pictures were modified before intervention so that the applicability related to the index test was not good. All the photographs were not analyzed so that the risk of bias related to flow and timing was high.

    Table 3. Studies based on store-and-forward teledermatology procedures. Risk of bias assessment according to quality assessment of diagnostic accuracy 2nd edition (QUADAS-2).
    AuthorsRisk of biasApplicability concerns

    		Patient selectionIndex testReference standardFlow and timingPatient selectionIndex testReference standard
    Store-and-forward teledermatology without teledermoscopy




    		Boyce et al, 2011 [31]HighLowLowHighLowLowLow

    		Lamel et al, 2011 [32]HighLowLowLowLowLowLow

    		Janda et al, 2014 [33]HighLowHighLowLowLowLow
    Store-and-forward teledermatology that included teledermoscopy




    		Ford et al, 2015 [34]LowLowLowHighLowLowLow

    		Börve et al, 2013 [29]HighLowLowLowHighLowLow

    		Börve et al, 2015 [35]HighLowLowHighLowLowLow

    		Hue et al, 2016 [36]HighLowLowHighLowLowLow

    		Janda et al, 2013 [37]HighLowLowHighLowLowLow

    		Kroemer at al, 2011 [38]HighHighLowHighLowLowLow

    		Manahan et al, 2015 [39]HighLowLowHighLowLowLow

    		Markun et al, 2017 [40]HighHighLowHighLowLowLow

    		Massone et al, 2007 [41]HighLowLowLowLowLowLow

    		Wu et al, 2015 [42]HighLowLowHighLowHighLow
    Table 4. Studies based on automated smartphone apps. Risk of bias assessment according to quality assessment of diagnostic accuracy 2nd edition (QUADAS-2).
    AuthorsRisk of biasApplicability concerns

    		Patient selectionIndex testReference standardFlow and timingPatient selectionIndex testReference standard
    Photographs issued from a database




    		Doukas et al, 2012 [44]HighLowUnclearUnclearHighLowUnclear

    		Do et al, 2014 [43]HighLowLowUnclearHighHighLow

    		Ferrero et al, 2013 [45]HighLowLowHighHighLowLow

    		Ramlakhan et al, 2011 [46]HighLowUnclearHighHighLowUnclear

    		Wadhawan et al, 2011a [47]HighLowLowHighHighHighLow

    		Wadhawan et al, 2011b [48]HighLowLowHighHighHighLow

    		Wolf et al, 2013 [49]HighLowLowHighHighHighLow
    Photographs taken of patients in real condition




    		Dorairaj et al, 2017 [50]HighLowLowHighHighLowLow

    		Maier et al, 2015 [51]HighLowLowHighHighLowLow

    		Ngoo et al, 2017 [52]HighLowLowHighLowLowLow

    		Robson et al, 2012 [53]HighLowUnclearHighLowLowLow

    		Thissen et al, 2017 [54]HighLowUnclearLowLowLowLow

    Store-and-forward teledermatology opens several perspectives, that is, it accelerates the care course, and various studies were performed in primary care populations. However, the concordance between the conclusion of a teledermatologist and the conclusion of a dermatologist who conducts a face-to-face examination depended on the study (the kappa coefficient range was .20 to .84, median κ=.60). The use of a dermoscope may improve the concordance (the kappa coefficient range was .29 to .87, median κ=.74). Regarding automated smartphone apps, the major concerns are their low sensitivity, the lack of assessment in clinical practice conditions, and the lack of assessment in primary care populations.

    In a study recently published in Nature, Esteva et al reported that an artificial neuronal network had a better capacity to recognize melanomas than a dermatologist [56]. However, the automated apps available on a smartphone in 2017 do not provide such expertise. Our review shows that the existing automated smartphone apps are unreliable. Some apps have a low diagnostic sensitivity that may induce false negatives and erroneously reassure patients who may then not consult a specialist [50]. Certificates do not guarantee good diagnostic performance [54]. Greater control by administrative authorities is necessary [45,57,58].

    This literature review suggests that teledermatology decreases the delays in the management of melanoma lesions [34-36,40,59], reduces the referrals to a dermatologist by avoiding unnecessary consultations [35,36,40] and limits the number of patients lost from follow-up [36,60]. Moreno-Ramirez et al confirmed this point during an experimental teledermatology program without a smartphone [61]. Other authors have not been as optimistic and suggested that the many false negatives and easy access to a dermatologist’s opinion may increase the number of secondary consultations [34].

    Large variations in the kappa concordance coefficient might be related to the range of melanoma prevalence (depending on the recruited population). On the one hand, this result should lead to multiple studies in the general population to assess the performance of the procedures in a primary care setting. On the other hand, this review emphasizes that the publication of data related to up-to-date technologies is a challenge in teledermatology [20,62]; it is notable that up to 70% of the photographs were of poor quality in a study performed in 2007 [41], whereas all photographs were interpretable in 2016 [54].

    The importance of training the person taking the photograph has been underlined [33,35-37,40], especially when a dermoscope is used [63]. Today, it is surprising to note that no standards for taking photographs exist [62,64]. Our review identified a few characteristics related to the quality of the photographs, such as several views of the same lesion, close-up and distant pictures, use of the autofocus and macro mode, a neutral background, and good lighting. The homogenization of practices based on these quality criteria is required to obtain better study reproducibility.

    This review recalls that to give an opinion solely based on photographs, whatever their quality, is a challenge for dermatologists [29,41,63]. For example, the absence of palpation is one of the limitations of teledermatology. Thissens’s study noted the need to obtain supplementary clinical information [54].

    General practitioners and patients are likely to omit suspect lesions [31,33,39]. This limitation, which has been described in general practice [13,65], exists in teledermatology, that is, general practitioners could miss up to 30% of melanomas [66]. Another difficulty is the omission of specific areas that are either difficult to access, such as hair, wounds, or the ear [51,54], or those considered sensitive (genitalia) [67,68].

    Store-and-forward teledermatology is well-accepted by patients and caregivers [59,69,70]. Patients report that one limitation of the apps is the difficulty of not having any human contact [20]. For both procedures, a limitation is the loss of the face-to-face patient-physician relationship, which may be critical since a melanoma is diagnosed early, at a severe stage, or in the aging population [59,69-71]. Positive perspectives may improve compliance to referrals and possibilities for the patient to participate actively in his or her health [24].

    Confidentiality, security, and traceability of data exchange are the major ethical and legal stakes [20,63,72]. Although the abusive use of clinical photographs has become an increasing preoccupation of health fund organizations [73], only 30% of patients worry about the future of their photographs [74], and our review reported that only slightly more than half of the authors encrypted their data. Although 60% of specialists continue to store photographs of their patients in their personal mobile phones [72,74-77], one perspective could be to develop recourse to encrypt the medical-image libraries [62].

    This review is original because of its specific focus on (1) melanoma early detection (mortality issues are not comparable for other dermatological pathologies) [20,62,78,79], (2) a primary care perspective (the sensitivity and specificity of a test depend on the prevalence of the disease), and (3) the use of a smartphone, that is, a tool implemented worldwide at low cost (not comparable to other expensive videoconference procedures) without limiting the study selection to apps [20,78-80]. However, this study had several weaknesses. First, this review had a large scope because we hypothesized that evidence might be sparse—the heterogeneity in study designs, the populations, the end points, the references used, and the presentations of the results made data comparison difficult. Second, the review was only based on MEDLINE. Third, the selection bias was high, and the prevalence of melanoma, which ranged from 0% to 100%, depended on the population studied. Fourth, the material used differed from one study to the next and from year to year, hence introducing a bias in evaluation.

    Our review confirmed the absence of evidence of the safety and efficacy of smartphone medical apps. In contrast, our review found evidence that store-and-forward teledermatology using smartphones may affect patients’ care courses, delays in obtaining a dermatologist consultation, and patients’ referral to secondary care. Further research is required to determine the quality criteria, as there was major variability among the studies.

    Acknowledgments

    The authors wish to thank Sasha Mann, who helped draft the manuscript. Financial support was provided by a grant from the Ministry of Health. The funding source had no role in the design or conduct of the study; the collection, management, analysis, or interpretation of the data; the preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

    Authors' Contributions

    CR conceived the study, was responsible for its design and supervision, and was responsible for drafting the manuscript. SH participated in the design of the study, managed the selection of the included study, participated in data collection, and helped draft the manuscript. JR participated in the design of the study, managed the selection of the included study, participated in data collection, and helped draft the manuscript. GQ and BD provided administrative and technical support and revised the manuscript critically for important intellectual content. JMN obtained the grant for the corresponding research program, was responsible for the research project, and helped draft the manuscript. All authors read and approved the final manuscript.

    Conflicts of Interest

    None declared.

    1. World Health Organization. 2012. GLOBOCAN 2012: Estimated cancer incidence, mortality, and prevalence worldwide in 2012   URL: http://globocan.iarc.fr/Pages/fact_sheets_population.aspx [accessed 2018-03-07] [WebCite Cache]
    2. National Cancer Institute. Surveillance, Epidemiology, and End Results Program. Cancer Stat Facts: Melanoma of the Skin   URL: https://seer.cancer.gov/statfacts/html/melan.html [accessed 2018-03-07] [WebCite Cache]
    3. Curiel-Lewandrowski C, Kim CC, Swetter SM, Chen SC, Halpern AC, Kirkwood JM, Melanoma Prevention Working Group—Pigmented Skin Lesion Sub-Committee. Survival is not the only valuable end point in melanoma screening. J Invest Dermatol 2012 May;132(5):1332-1337 [FREE Full text] [CrossRef] [Medline]
    4. Geller AC, Elwood M, Swetter SM, Brooks DR, Aitken J, Youl PH, et al. Factors related to the presentation of thin and thick nodular melanoma from a population-based cancer registry in Queensland Australia. Cancer 2009 Mar 15;115(6):1318-1327 [FREE Full text] [CrossRef] [Medline]
    5. Van Durme DJ, Ferrante JM, Pal N, Wathington D, Roetzheim RG, Gonzalez EC. Demographic predictors of melanoma stage at diagnosis. Arch Fam Med 2000 Jul;9(7):606-611. [Medline]
    6. Richard MA, Grob JJ, Avril MF, Delaunay M, Gouvernet J, Wolkenstein P, et al. Delays in diagnosis and melanoma prognosis (I): the role of patients. Int J Cancer 2000;89(3):271-279. [Medline]
    7. Grange F, Barbe C, Aubin F, Lipsker D, Granel-Brocard F, Velten M, et al. Clinical and sociodemographic characteristics associated with thick melanomas: a population-based, case-case study in France. Arch Dermatol 2012;148(12):1370-1376. [Medline]
    8. Davis JL, Buchanan KL, Katz RV, Green BL. Gender differences in cancer screening beliefs, behaviors, and willingness to participate: implications for health promotion. Am J Mens Health 2012;6(3):211-217. [Medline]
    9. Montella M, Crispo A, Grimaldi M, De Marco MR, Ascierto PA, Parasole R, et al. An assessment of factors related to tumor thickness and delay in diagnosis of melanoma in Southern Italy. Prev Med. ? 2002;35(3):271-277. [Medline]
    10. Baumert J, Plewig G, Volkenandt M, Schmid-Wendtner MH. Factors associated with a high tumour thickness in patients with melanoma. Br J Dermatol 2007;156(5):938-944. [Medline]
    11. Mandalà M, Imberti GL, Piazzalunga D, Belfiglio M, Lucisano G, Labianca R, et al. Association of socioeconomic status with Breslow thickness and disease-free and overall survival in stage I-II primary cutaneous melanoma. Mayo Clin Proc 2011;86(2):113-119. [Medline]
    12. Rat C, Quereux G, Monegier du Sorbier M, Gaultier A, Bonnaud-Antignac A, Khammari A, et al. Patients at elevated risk of melanoma: individual predictors of non-compliance to GP referral for a dermatologist consultation. Prev Med 2014;64:48-53. [Medline]
    13. Richard MA, Grob JJ, Avril MF, Delaunay M, Gouvernet J, Wolkenstein P, et al. Delays in diagnosis and melanoma prognosis (II): the role of doctors. Int J Cancer 2000;89(3):280-285. [Medline]
    14. Uhlenhake E, Brodell R, Mostow E. The dermatology work force: A focus on urban versus rural wait times. J Am Acad Dermatol 2009;61(1):17-22. [Medline]
    15. Roetzheim RG, Lee JH, Ferrante JM, Gonzalez EC, Chen R, Fisher KJ, et al. The influence of dermatologist and primary care physician visits on melanoma outcomes among medicare beneficiaries. J Am Board Fam Med 2013;26(6):637-647. [Medline]
    16. Stitzenberg KB, Thomas NE, Dalton K, Brier SE, Ollila DW, Berwick M, et al. Distance to diagnosing provider as a measure of access for patients with melanoma. Arch Dermatol 2007 Aug;143(8):991-998 [FREE Full text] [CrossRef] [Medline]
    17. Tsang MW, Resneck JS. Even patients with changing moles face long dermatology appointment wait-times: a study of simulated patient calls to dermatologists. J Am Acad Dermatol 2006 Jul;55(1):54-58. [CrossRef] [Medline]
    18. WHO Group Consultation on Health Telematics. A health telematics policy in support of WHO's Health-for-all strategy for global health development: Report of the WHO Group Consultation on Health Telematics, 11-16 December, Geneva, 1997   URL: http://apps.who.int/iris/bitstream/10665/63857/1/WHO_DGO_98.1.pdf [accessed 2018-03-07] [WebCite Cache]
    19. World Health Organization. Geneva, Switzerland; 2010. Telemedicine: opportunities and developments in member states: Report on the second Global survey on eHealth   URL: http://apps.who.int/iris/bitstream/handle/10665/44497/9789241564144_eng.pdf?sequence=1&isAllowed=y
    20. Tensen E, van der Heijden JP, Jaspers MW, Witkamp L. Two decades of teledermatology: current status and integration in national healthcare systems. Curr Dermatol Rep 2016;5:96-104 [FREE Full text] [CrossRef] [Medline]
    21. Kozera EK, Yang A, Murrell DF. Patient and practitioner satisfaction with tele-dermatology including Australia's indigenous population: a systematic review of the literature. Int J Womens Dermatol 2016 Sep;2(3):70-73 [FREE Full text] [CrossRef] [Medline]
    22. Resneck J, Kimball AB. The dermatology workforce shortage. J Am Acad Dermatol 2004 Jan;50(1):50-54. [CrossRef] [Medline]
    23. Armstrong AW, Dorer DJ, Lugn NE, Kvedar JC. Economic evaluation of interactive teledermatology compared with conventional care. Telemed J E Health 2007 Apr;13(2):91-99. [CrossRef] [Medline]
    24. Coates SJ, Kvedar J, Granstein RD. Teledermatology: from historical perspective to emerging techniques of the modern era: part I: history, rationale, and current practice. J Am Acad Dermatol 2015 Apr;72(4):563-74; quiz 575. [CrossRef] [Medline]
    25. van der Heijden JP, de Keizer NF, Bos JD, Spuls PI, Witkamp L. Teledermatology applied following patient selection by general practitioners in daily practice improves efficiency and quality of care at lower cost. Br J Dermatol 2011 Nov;165(5):1058-1065. [CrossRef] [Medline]
    26. May C, Giles L, Gupta G. Prospective observational comparative study assessing the role of store and forward teledermatology triage in skin cancer. Clin Exp Dermatol 2008 Nov;33(6):736-739. [CrossRef] [Medline]
    27. Warshaw EM, Hillman YJ, Greer NL, Hagel EM, MacDonald R, Rutks IR, et al. Teledermatology for diagnosis and management of skin conditions: a systematic review. J Am Acad Dermatol 2011 Apr;64(4):759-772. [CrossRef] [Medline]
    28. Senior K. Smart phones: new clinical tools in oncology? Lancet Oncol 2011 May;12(5):429-430. [CrossRef] [Medline]
    29. Börve A, Terstappen K, Sandberg C, Paoli J. Mobile teledermoscopy-there's an app for that!. Dermatol Pract Concept 2013 Apr;3(2):41-48 [FREE Full text] [CrossRef] [Medline]
    30. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015 Jan;4:1 [FREE Full text] [CrossRef] [Medline]
    31. Boyce Z, Gilmore S, Xu C, Soyer HP. The remote assessment of melanocytic skin lesions: a viable alternative to face-to-face consultation. Dermatology 2011;223(3):244-250. [CrossRef] [Medline]
    32. Lamel SA, Haldeman KM, Ely H, Kovarik CL, Pak H, Armstrong AW. Application of mobile teledermatology for skin cancer screening. J Am Acad Dermatol 2012 Oct;67(4):576-581. [CrossRef] [Medline]
    33. Janda M, Loescher LJ, Banan P, Horsham C, Soyer HP. Lesion selection by melanoma high-risk consumers during skin self-examination using mobile teledermoscopy. J Am Med Assoc Dermatol 2014 Jun;150(6):656-658. [CrossRef] [Medline]
    34. Ford JA, Pereira A. Does teledermatology reduces secondary care referrals and is it acceptable to patients and doctors?: A service evaluation. J Eval Clin Pract 2015 Aug;21(4):710-716. [CrossRef] [Medline]
    35. Börve A, Dahlén Gyllencreutz J, Terstappen K, Johansson Backman E, Aldenbratt A, Danielsson M, et al. Smartphone teledermoscopy referrals: a novel process for improved triage of skin cancer patients. Acta Derm Venereol 2015 Feb;95(2):186-190 [FREE Full text] [CrossRef] [Medline]
    36. Hue L, Makhloufi S, Sall N'Diaye P, Blanchet-Bardon C, Sulimovic L, Pomykala F, et al. Real-time mobile teledermoscopy for skin cancer screening targeting an agricultural population: an experiment on 289 patients in France. J Eur Acad Dermatol Venereol 2016 Jan;30(1):20-24. [CrossRef] [Medline]
    37. Janda M, Loescher LJ, Soyer HP. Enhanced skin self-examination: a novel approach to skin cancer monitoring and follow-up. J Am Med Assoc Dermatol 2013 Feb;149(2):231-236. [CrossRef] [Medline]
    38. Kroemer S, Frühauf J, Campbell TM, Massone C, Schwantzer G, Soyer HP, et al. Mobile teledermatology for skin tumour screening: diagnostic accuracy of clinical and dermoscopic image tele-evaluation using cellular phones. Br J Dermatol 2011 May;164(5):973-979. [CrossRef] [Medline]
    39. Manahan MN, Soyer HP, Loescher LJ, Horsham C, Vagenas D, Whiteman DC, et al. A pilot trial of mobile, patient-performed teledermoscopy. Br J Dermatol 2015 Apr;172(4):1072-1080. [CrossRef] [Medline]
    40. Markun S, Scherz N, Rosemann T, Tandjung R, Braun RP. Mobile teledermatology for skin cancer screening: a diagnostic accuracy study. Medicine (Baltimore) 2017 Mar;96(10):e6278 [FREE Full text] [CrossRef] [Medline]
    41. Massone C, Hofmann-Wellenhof R, Ahlgrimm-Siess V, Gabler G, Ebner C, Soyer HP. Melanoma screening with cellular phones. PLoS One 2007 May 30;2(5):e483 [FREE Full text] [CrossRef] [Medline]
    42. Wu X, Oliveria SA, Yagerman S, Chen L, DeFazio J, Braun R, et al. Feasibility and efficacy of patient-initiated mobile teledermoscopy for short-term monitoring of clinically atypical nevi. J Am Med Assoc Dermatol 2015 May;151(5):489-496. [CrossRef] [Medline]
    43. Do TT, Zhou Y, Zheng H, Cheung NM, Koh D. Early melanoma diagnosis with mobile imaging. Conf Proc IEEE Eng Med Biol Soc 2014;2014:6752-6757. [CrossRef] [Medline]
    44. Doukas C, Stagkopoulos P, Kiranoudis CT, Maglogiannis I. Automated skin lesion assessment using mobile technologies and cloud platforms. Conf Proc IEEE Eng Med Biol Soc 2012;2012:2444-2447. [CrossRef] [Medline]
    45. Ferrero NA, Morrell DS, Burkhart CN. Skin scan: a demonstration of the need for FDA regulation of medical apps on iPhone. J Am Acad Dermatol 2013 Mar;68(3):515-516. [CrossRef] [Medline]
    46. Ramlakhan K, Shang Y. A mobile automated skin lesion classification system. Presented at: 23rd IEEE International Conference on tools with artificial intelligence (ICTAI); November 7-9, 2011; Boca Raton, FL, USA. [CrossRef]
    47. Wadhawan T, Situ N, Lancaster K, Yuan X, Zouridakis G. SkinScan©: a portable library for melanoma detection on handheld devices. Proc IEEE Int Symp Biomed Imaging 2011 Mar 30;2011:133-136 [FREE Full text] [CrossRef] [Medline]
    48. Wadhawan T, Situ N, Rui H, Lancaster K, Yuan X, Zouridakis G. Implementation of the 7-point checklist for melanoma detection on smart handheld devices. Conf Proc IEEE Eng Med Biol Soc 2011;2011:3180-3183 [FREE Full text] [CrossRef] [Medline]
    49. Wolf JA, Moreau JF, Akilov O, Patton T, English 3rd JC, Ho J, et al. Diagnostic inaccuracy of smartphone applications for melanoma detection. J Am Med Assoc Dermatol 2013 Apr;149(4):422-426 [FREE Full text] [CrossRef] [Medline]
    50. Dorairaj JJ, Healy GM, McInerney A, Hussey AJ. Validation of a melanoma risk assessment smartphone application. Dermatol Surg 2017 Feb;43(2):299-302. [CrossRef] [Medline]
    51. Maier T, Kulichova D, Schotten K, Astrid R, Ruzicka T, Berking C, et al. Accuracy of a smartphone application using fractal image analysis of pigmented moles compared to clinical diagnosis and histological result. J Eur Acad Dermatol Venereol 2015 Apr;29(4):663-667. [CrossRef] [Medline]
    52. Ngoo A, Finnane A, McMeniman E, Tan JM, Janda M, Soyer HP. Efficacy of smartphone applications in high-risk pigmented lesions. Australas J Dermatol 2017 Feb 27:1-1. [CrossRef] [Medline]
    53. Robson Y, Blackford S, Roberts D. Caution in melanoma risk analysis with smartphone application technology. Br J Dermatol 2012 Sep;167(3):703-704. [CrossRef] [Medline]
    54. Thissen M, Udrea A, Hacking M, von Braunmuehl T, Ruzicka T. mHealth App for risk assessment of pigmented and nonpigmented skin lesions-a study on sensitivity and specificity in detecting malignancy. Telemed J E Health 2017 May 31;23(12):948-954. [CrossRef] [Medline]
    55. Bristol University. QUADAS 2   URL: https://www.bristol.ac.uk/media-library/sites/quadas/migrated/documents/background-doc.pdf [accessed 2018-03-07] [WebCite Cache]
    56. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM, et al. Corrigendum: Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017 Dec 28;546(7660):686. [CrossRef] [Medline]
    57. Kassianos AP, Emery JD, Murchie P, Walter FM. Smartphone applications for melanoma detection by community, patient and generalist clinician users: a review. Br J Dermatol 2015 Jun;172(6):1507-1518. [CrossRef] [Medline]
    58. Boulos MN, Brewer AC, Karimkhani C, Buller DB, Dellavalle RP. Mobile medical and health apps: state of the art, concerns, regulatory control and certification. Online J Public Health Inform 2014;5(3):229 [FREE Full text] [CrossRef] [Medline]
    59. Horsham C, Loescher LJ, Whiteman DC, Soyer HP, Janda M. Consumer acceptance of patient-performed mobile teledermoscopy for the early detection of melanoma. Br J Dermatol 2016 Dec;175(6):1301-1310. [CrossRef] [Medline]
    60. Bezalel S, Fabri P, Park HS. Implementation of store-and-forward teledermatology and its associated effect on patient access in a veterans affairs dermatology clinic. J Am Med Assoc Dermatol 2015 May;151(5):556-557. [CrossRef] [Medline]
    61. Moreno-Ramírez D, Argenziano G. Teledermatology and mobile applications in the management of patients with skin lesions. Acta Derm Venereol 2017 Jul 05:1-1 [FREE Full text] [CrossRef] [Medline]
    62. Ashique KT, Kaliyadan F, Aurangabadkar SJ. Clinical photography in dermatology using smartphones: an overview. Indian Dermatol Online J 2015;6(3):158-163 [FREE Full text] [CrossRef] [Medline]
    63. The Scottish Centre for Telehealth and Telecare, British Association of Dermatologists, British Dermatological Nursing Group, Primary Care Dermatology Society. 2013. Quality standards for teledermatology using Store And Forward Images   URL: https://www.bad.org.uk/shared/get-file.ashx?itemtype=document&id=794
    64. Quigley EA, Tokay BA, Jewell ST, Marchetti MA, Halpern AC. Technology and technique standards for camera-acquired digital dermatologic images: a systematic review. J Am Med Assoc Dermatol 2015 Aug;151(8):883-890. [CrossRef] [Medline]
    65. Rat C, Grimault C, Quereux G, Dagorne M, Gaultier A, Khammari A, et al. Proposal for an annual skin examination by a general practitioner for patients at high risk for melanoma: a French cohort study. Br Med J Open 2015 Jul 29;5(7):e007471 [FREE Full text] [CrossRef] [Medline]
    66. Gendreau JL, Gemelas J, Wang M, Capulong D, Lau C, Bratten DM, et al. Unimaged melanomas in store-and-forward teledermatology. Telemed J E Health 2017 Jun;23(6):517-520. [CrossRef] [Medline]
    67. Lakdawala N, Fontanella D, Grant-Kels JM. Ethical considerations in dermatologic photography. Clin Dermatol 2012;30(5):486-491. [CrossRef] [Medline]
    68. Leger MC, Wu T, Haimovic A, Kaplan R, Sanchez M, Cohen D, et al. Patient perspectives on medical photography in dermatology. Dermatol Surg 2014 Sep;40(9):1028-1037. [CrossRef] [Medline]
    69. Spinks J, Janda M, Soyer HP, Whitty JA. Consumer preferences for teledermoscopy screening to detect melanoma early. J Telemed Telecare 2016 Jan;22(1):39-46. [CrossRef] [Medline]
    70. Mounessa JS, Chapman S, Braunberger T, Qin R, Lipoff JB, Dellavalle RP, et al. A systematic review of satisfaction with teledermatology. J Telemed Telecare 2017 Jan 01:1357633X17696587. [CrossRef] [Medline]
    71. Hall S, Murchie P. Can we use technology to encourage self-monitoring by people treated for melanoma? A qualitative exploration of the perceptions of potential recipients. Support Care Cancer 2014 Jun;22(6):1663-1671. [CrossRef] [Medline]
    72. Kunde L, McMeniman E, Parker M. Clinical photography in dermatology: ethical and medico-legal considerations in the age of digital and smartphone technology. Australas J Dermatol 2013 Aug;54(3):192-197. [CrossRef] [Medline]
    73. Mahar PD, Foley PA, Sheed-Finck A, Baker CS. Legal considerations of consent and privacy in the context of clinical photography in Australian medical practice. Med J Aust 2013 Jan 21;198(1):48-49. [Medline]
    74. Anyanwu CO, Lipoff JB. Smartphones, photography, and security in dermatology. J Am Acad Dermatol 2015 Jan;72(1):193-195. [CrossRef] [Medline]
    75. Chan N, Charette J, Dumestre DO, Fraulin FO. Should 'smart phones' be used for patient photography? Plast Surg (Oakv) 2016;24(1):32-34 [FREE Full text] [Medline]
    76. Burns K, Belton S. Clinicians and their cameras: policy, ethics and practice in an Australian tertiary hospital. Aust Health Rev 2013 Sep;37(4):437-441. [CrossRef] [Medline]
    77. Abbott LM, Magnusson RS, Gibbs E, Smith SD. Smartphone use in dermatology for clinical photography and consultation: current practice and the law. Australas J Dermatol 2017 Feb 28:1-1. [CrossRef] [Medline]
    78. Snoswell C, Finnane A, Janda M, Soyer HP, Whitty JA. Cost-effectiveness of store-and-forward teledermatology: a systematic review. J Am Med Assoc Dermatol 2016 Jun 01;152(6):702-708. [CrossRef] [Medline]
    79. Cook SE, Palmer LC, Shuler FD. Smartphone mobile applications to enhance diagnosis of skin cancer: a guide for the rural practitioner. W V Med J 2015;111(5):22-28. [Medline]
    80. Chao E, Meenan CK, Ferris LK. Smartphone-based applications for skin monitoring and melanoma detection. Dermatol Clin 2017 Oct;35(4):551-557. [CrossRef] [Medline]

    ‎
    QUADAS-2: quality assessment of diagnostic accuracy 2nd edition

    Edited by G Eysenbach; submitted 15.11.17; peer-reviewed by L Ferris, M Janda; comments to author 27.12.17; revised version received 09.01.18; accepted 10.01.18; published 13.04.18

    Copyright

    ©Cédric Rat, Sandrine Hild, Julie Rault Sérandour, Aurélie Gaultier, Gaelle Quereux, Brigitte Dreno, Jean-Michel Nguyen. Originally published in the Journal of Medical Internet Research (http://www.jmir.org), 13.04.2018.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on http://www.jmir.org/, as well as this copyright and license information must be included.