Original Paper
Abstract
Background: Patients with long-term health needs are often expected to actively participate in outpatient care, assuming that they have appropriate health literacy and digital health literacy. However, the association between participation in a digital outpatient service and health literacy remain unclear.
Objective: This study aims to evaluate whether digital outpatient care for 6 months improved health literacy, health-related quality of life (HRQoL), digital/eHealth literacy, and the use of health care services compared with usual care.
Methods: We conducted a multicenter nonrandomized trial with 1 intervention arm and 1 control arm. Patients aged ≥18 years receiving outpatient care in the pain, lung, neurology, or cancer departments at 2 Norwegian university hospitals were allocated in a 1:2 ratio, favoring the intervention arm. The intervention arm received digital outpatient care using tailored patient-reported outcome measures, self-monitoring, and chats for timely contact with the outpatient clinic. Patient responses were assessed by health care workers via a dashboard with a traffic light system to draw attention to the most urgent reports. The control arm received usual care. The data were collected at baseline and after 3 and 6 months. The primary outcome was the change in health literacy according to the Health Literacy Questionnaire domain understanding health information well enough to know what to do from baseline to 6 months. The mean difference in change between the 2 treatment arms was the effect measure. The secondary outcomes were additional domains from the Health Literacy Questionnaire, digital/eHealth literacy, HRQoL, acceptability of the digital intervention, and health service use.
Results: Overall, 162 patients were recruited, 55 (34%) in the control arm and 107 (66%) in the intervention arm, with a 17.3% attrition rate after 6 months. There was no statistically significant difference in the primary outcome, “understanding health information well enough to know what to do,” between the arms at 6 months (mean difference –0.05, 95% CI –0.20 to 0.10; P=.53). After 3 months, the health literacy domains actively managing my own health (–0.15, 95% CI –0.30 to –0.00; P=.048) and understanding health information well enough to know what to do (–0.17, 95% CI –0.34 to –0.00; P=.03), as well as both physical (–3.29, 95% CI –5.62 to –0.96; P=.006) and mental HRQoL (–3.08, 95% CI –5.64 to –0.52; P=.02), improved in the digital outpatient intervention arm compared with the control arm.
Conclusions: This study explored digital outpatient care. Although no statistical differences were observed in patients’ health literacy after 6 months, our data indicate an improvement in health literacy domains and HRQoL at 3 months. The participants reported high satisfaction with the digital outpatient care intervention, and our findings highlight the potential of digital interventions in outpatient care.
Trial Registration: ClinicalTrials.gov NCT05068869; https://clinicaltrials.gov/ct2/show/NCT05068869
International Registered Report Identifier (IRRID): RR2-10.2196/46649
doi:10.2196/60343
Keywords
Introduction
Background
The implementation of digital health services in specialized health care services necessitates an understanding of the purpose and use of digital services by both patients and health care workers [,]. The potential of digital health solutions is recognized by health authorities and stakeholders; however, despite systematic reviews exploring digital health services, the impact of digital solutions on resource use and patient outcomes, such as symptoms, self-management, quality of life, digital health literacy, and satisfaction, remains uncertain [-]. Digital services may improve resource use and provide patients with supplementary and understandable health information; the potential benefit and relevance of digital health solutions increase as medical advancements extend life expectancy and as patients with chronic or long-term conditions in need of frequent and repeated care use a large proportion of the available consultations [,]. Exploring how to exploit this potential is crucial, as the increase in patients, combined with constrained health resources, presents obstacles to the sustainability of current health services []. Furthermore, reductions in hospital beds may increase the need for more outpatient consultations, which is an arena where digital health services have the potential to aid timely and appropriate care.
To fulfill the potential of digital health solutions, attaining a certain level of health literacy is pivotal because health literacy positively influences both self-management and health-related decisions [,-]. Health literacy is often defined, according to Nutbeam [], as “the cognitive and social skills that determine the motivation and ability of individuals to gain access to, understand, and use information in ways which promote and maintain good health.” Consequently, digital health literacy concerns the use of digital solutions for promoting and maintaining good health []. Low health literacy is associated with poor health outcomes and less digital solution use [,,,,], while improved health literacy may enhance self-management and the benefits of digital health solutions []. Thus, in accordance with a conceptual model proposing how health literacy is affecting interaction with services and health care workers and outcomes [], we argue that digital health interventions specifically allowing for more patient-centered and flexible care might support patients’ health literacy and aid in their self-management []. However, there is a lack of research exploring the role of health literacy in digital health interventions, although the capacity to benefit from digital health solutions and improve self-management is likely to be enhanced by improved health literacy [,]. In addition, there exists a similar gap in knowledge on the relation between digital health literacy and digital health interventions [].
Digital health solutions in outpatient care typically enable the subjective reporting of health parameters through patient-reported outcome (PRO) measures. PRO measures may aid in self-management and communication between patients and health care workers []. Digital engagement, self-monitoring, and data sharing offer advantages for patients; however, whether a digital health solution is meaningful depends on its usability, clinical relevance, and convenience, along with whether it has an evidence-based design [,-]. Recent studies indicate that digital solutions in outpatient care can prevent complications, foster patient engagement, and boost confidence and autonomy [-]. However, research on multicomponent digital solutions encompassing PRO measures, asynchronous messaging, remote monitoring, patient notifications, and video consultations is limited. Positive effects have been noted in cancer treatment [], musculoskeletal pain disorders management [], and home monitoring for interstitial lung disease (ILD) []. Mobile apps for patients with epilepsy have shown promising results; however, the value of patients’ collaborating with health care workers remains unclear []. Challenges persist, such as integrating electronic health records and standardizing data for registry use. In summary, studies exploring the implementation and impacts of digital health solutions in outpatient care are warranted.
Objectives
The objective of this study was to evaluate whether digital outpatient care for 6 months resulted in improved health literacy, health-related quality of life (HRQoL), digital/eHealth literacy, and the use of health care services compared with usual care.
Specifically, we hypothesized that digital outpatient care would be associated with a favorable increase in patients’ ability to understand health information well enough to know what to do. By enabling digital access to services, facilitating patient reports and self-monitoring, and promoting interaction with both the digital solution and health care workers, our intervention reinforces a conceptual model for associations between health literacy and health outcomes [,].
Methods
Trial Design
We conducted a prospective, multicenter, nonrandomized controlled trial with 2 arms and a 6-month follow-up, using a longitudinal design with 3 assessment points: baseline, 3 months, and 6 months (from September 2021 to June 2023). The control arm received the usual care, while the intervention arm received the digital outpatient care intervention. The trial was registered in Clinical Trials and the International Registered Report Identifier (DERR1-10.2196/46649). The evaluation was based on elements of the method for assessment of telemedicine [], and our study was reported according to Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) checklist [], while the PRO measures were reported according to the Standard Protocol Items: Recommendations for Interventional Trial (SPIRIT) PRO Extension []. The development of the intervention and study protocol have been reported in detail elsewhere [].
Participants and Setting
Patients were recruited from the department of respiratory diseases, the department of neurology, and the department of pain management and research at the Oslo University Hospital as well as the department of cancer at the University Hospital of North Norway. Eligible patients were required to be aged ≥18 years, based at home, and capable of completing questionnaires in Norwegian. Inclusion criteria were applied to patients who were newly diagnosed or referred or to those with a diagnostic history in outpatient care. Patients with cancer had to receive active treatment, have a life expectancy of at least 6 months, and have an expected need for services beyond 6 months. Patients with ILD were eligible unless they had a very low degree of function with cognitive impairment. Patients with epilepsy were included unless they had complex causes, severe comorbidities, or ongoing assessments or treatments. Patients with long-term pain were eligible if they underwent drug testing and self-administered their medication and were not eligible if they had comorbidities that directly affected drug adjustment, had cognitive impairment, or did not live at home.
Eligible patients were identified through consultations or patient lists provided by health care workers from each department. The patients received project information before written consent was obtained. At the 3- and 6-month follow-ups, the patients received either an SMS text message or an email invitation to complete their follow-up questionnaire. Consent and all questionnaire responses were provided through the encrypted version of the web-based questionnaire service Nettskjema (University of Oslo) []. The data were stored using a service for sensitive data (TSD), complying with the Norwegian Personal Data Act and Health Research Act. This service required a governmental ID portal for log-in, hence ensuring secure data harvesting. In the event of no response after 1 week, a reminder was sent, followed by a phone call 1 week later for pending questionnaires.
Consent and baseline questionnaire completion were prerequisites for a fulfilled allocation for both the control and intervention participants. In addition, this completion of consent and baseline questionnaire was required to access the digital intervention in the intervention arm to reduce contamination in the baseline questionnaire responses.
Participants recruited in the control arm received usual care per routine at the outpatient clinic where they received care. This included the possibility of contacting outpatient departments by telephone during office hours and regular in-person consultations.
Intervention
The details of the development and tailoring of the intervention have been previously presented []. The Dignio Connected Care platform facilitated timely, personalized contact between patients and health care workers []. This platform, including the Dignio Prevent system for health care workers and the MyDignio patient app, is Conformité Européene marked, compliant with privacy regulations, and successfully applied in various global clinical settings. Customizable to individual patient needs, components were added for personalized care, with interface examples provided in .
Health care workers from each department assigned tasks to patients in Dignio Prevent and set thresholds for individual patients using a traffic light model. Notifications alerted health care workers if measures deviated, such as patients reporting an increase in pain and side effects, or if the task went undone. The patients received reminders through the MyDignio app and an SMS text message if the task remained undone. Two-way messaging enabled asynchronous communication, allowing patients to ask questions or provide relevant information to their health care workers; likewise, health care workers could respond in due time based on the urgency of the message and their workload. For instance, treatment plans were shared with the patients through the chat messaging function, allowing them to always have access to their current plan. MyDignio app’s information page, which was updatable to health care workers, allowed for individualized updates and templates for multiple patients if needed.
The intervention’s core focus across all 4 departments was improving outpatient service accessibility. The app allowed patients to engage with PRO measures, self-monitor relevant parameters, and communicate asynchronously with health care workers (
Table 1). Designated staff assessed the responses, allowing for flexible patient care, which is a strategy aligned with a conceptual model linking health literacy to service access, use, interactions between patient and health care worker, and self-management [,].
| Dignio component | Short description | Department using the component in its intervention | |||
| PROa measures | |||||
| Self-reported measures | Standardized and individualized PRO measures with numerical scales, single and multiple-choice answers, and free text |
| |||
| Physiological measures | |||||
| Blood pressure | BYODg |
| |||
| Body temperature | BYOD |
| |||
| Body weight | BYOD |
| |||
| Spirometry values | Bluetooth device |
| |||
| Oxygen saturation | Bluetooth device |
| |||
| Pulse | BYOD |
| |||
| Patient-reported free text | |||||
| Free-text messages | Free-text messages sent from the patient to the service |
| |||
aPRO: patient-reported outcome.
bESAS-r: Edmonton Symptom Assessment System Revised.
cECOG: Eastern Cooperative Oncology Group.
dK-BILD: King’s Brief Interstitial Lung Disease.
ePRO-EPI: patient-reported outcome–epilepsy.
fNRS: Numeric Rating Scale.
gBYOD: bring your own device.
Patients in the intervention arm were briefly introduced to the app, but no extensive training was provided because the MyDignio app was set to be intuitive. Those using spirometers were instructed to ensure the correct technique and synchronization with the MyDignio app. The intervention arm participants could contact health care workers or researchers for technical assistance.
Most health care workers were involved in developing the intervention; therefore, they were familiar with the platform. The training on interpreting patient-reported scores was department specific. The designated health care workers received compensation for 10% to 20% of their time.
Measures
Data Collection
All self-reported data were collected through questionnaires sent to the participants through the Pretty Good Privacy–encrypted version of the University of Oslo web-questionnaire service Nettskjema [], which uses a governmental ID portal for log-in and allows secure data harvesting, previously described in our protocol []. A secure, personal link was sent to the participants’ email or mobile phone, with 1 reminder in the case of unanswered questionnaires after 6 to 7 days, followed by a phone call if they still did not respond. The participants self-reported the demographic data at baseline, while clinical measures were collected from their medical records at baseline and 6-month follow-up. Data on the use of digital service intervention and health services were collected from medical records and the digital platform after 6 months. The primary outcome, domain 9 of the Health Literacy Questionnaire (HLQ), understanding health information well enough to know what to do, and secondary outcomes comprising domains 1, 2, 3, and 6 of HLQ; the eHealth Literacy Questionnaire (eHLQ); and RAND-12 health survey questionnaire were self-reported at baseline and the 3- and 6-month follow-ups. General satisfaction and satisfaction with the digital intervention were self-reported after 3- and 6-month follow-ups.
Demographics
Demographic characteristics included age, sex, education, employment status, marital status, and digital skills. The full range of demographic data and their scores has been reported elsewhere [].
Clinical Measures
Clinical information included the primary diagnosis and its duration, medication, and comorbidities. Medication, contact with health care services, and hospitalization during the 6 months of the trial were recorded at the 6-month follow-up. Lifestyle habits, including smoking, the use of snuff, and alcohol consumption, were self-reported at baseline. History of previous infection with COVID-19 and fear of COVID-19 were self-reported and collected at all 3 time points. The fear of COVID-19 was reported only among those without a previous infection of the virus.
Use of the Digital Service Intervention
Throughout the intervention period, the use of the digital outpatient service was recorded. This included data on the number of PRO measure responses, the number of chat messages sent and received by the patients, and the number of physiological measures uploaded. A total score of intervention use containing all responses from the patient and interactions with health care workers was calculated. The total number was dichotomized into low use and high use based on the expected use per department (Table 1). Low-use patients responded to <30% of the expected PRO measures.
Primary Outcome
The primary outcome was domain 9 of the HLQ, understanding health information well enough to know what to do []. This domain includes 5 items covering health-related competencies essential for using a digital service, such as fundamental reading and comprehension of health information, adherence to health care workers’ instructions, and the capability to accurately complete forms [,]. This domain was self-reported at all 3 time points using a 5-point scoring scale, “cannot do or always difficult,” “usually difficult,” “sometimes difficult,” “usually easy,” and “always easy,” where a higher score indicates a higher level of health literacy. The HLQ is a validated measure translated into Norwegian and adapted to the Norwegian context []. Unfortunately, 1 of the 5 items, item 12, “read and understand written health information,” was not collected because of an administrative error. Thus, the mean score was based on the remaining 4 items. Cronbach α was 0.72 for domain 9.
Secondary Outcomes
Health literacy was assessed using an additional 4 of the 9 domains of the HLQ, including domain 1, feeling understood and supported by health care providers (4 items); domain 2, having sufficient information to manage my own health (4 items); domain 3, actively managing my own health (5 items); and domain 6, ability to actively engage with health care providers (5 items) []. Cronbach α was satisfactory for all domains (domain 1: Cronbach α=0.79; domain 2: Cronbach α=0.84; domain 3: Cronbach α=0.83; and domain 6: Cronbach α=0.90).
Digital/eHealth literacy was measured using all 7 domains of the eHLQ [], including domain 1, using technology to process health information (5 items); domain 2, understanding health concepts and language (5 items); domain 3, ability to actively engage with digital services (5 items); domain 4, feel safe and in control (5 items); domain 5, motivated to engage with digital services (5 items); domain 6, access to digital services that work (6 items); and domain 7, digital services that suit individual needs (4 items). Higher scores indicate higher levels of digital/eHealth literacy. The eHLQ has been validated and translated into Norwegian []. Cronbach α was satisfactory for all domains in this study (domain 1: Cronbach α=0.82; domain 2: Cronbach α=0.75; domain 3: Cronbach α=0.85; domain 4: Cronbach α=0.85; domain 5: Cronbach α=0.79; domain 6: Cronbach α=0.74; and domain 7: Cronbach α=0.78).
HRQoL was assessed using the RAND-12 questionnaire [,], which contains 12 items, informing 1 physical and 1 mental component score, where a higher standardized score indicates better HRQoL []. Cronbach α was 0.86 for the physical score and 0.79 for the mental score in the sample.
General patient satisfaction was assessed among all participants, encompassing the following 3 items: satisfaction with treatment, benefit from treatment, and whether a digital outpatient service would affect safety.
Satisfaction related to the intervention was only assessed in the intervention arm and measured using the Service User Technology Acceptability Questionnaire (SUTAQ), which comprises 22 items with a 6-point Likert scale []. The SUTAQ contains six domains, including domain 1, enhanced care (5 items); domain 2, increased accessibility (4 items); domain 3, privacy and discomfort (4 items); domain 4, care personnel concerns (3 items); domain 5, kit as a substitute (3 items); and domain 6, satisfaction (3 items). For domains 1, 5, and 6, a higher score indicates greater satisfaction, while for domains 3 and 4, the opposite is true, where a higher score represents a concern. Cronbach α was satisfactory (Cronbach α≥0.7) for 4 of the 6 domains in this study, including domain 1 (Cronbach α=0.87), domain 2 (Cronbach α=0.84), domain 3 (Cronbach α=0.73), domain 4 (Cronbach α=0.62), domain 5 (Cronbach α=0.46), and domain 6 (Cronbach α=0.89). The SUTAQ was previously translated and applied in a Norwegian context, where the same domains (ie, domains 4 and 5) showed limited internal consistency [,].
Sample Size
The a priori sample size calculation was based on the HLQ domain 9, “understanding health information well enough to know what to do,” and provided a change from baseline to 6-month follow-up, with an effect size of 0.5-unit difference between the arms on a scale of 1 to 5, with a 20% dropout, a power of 0.90, an SD of 0.6 from the outcome measure, and a 2-sided significance test. Thus, following a 1:2 recruitment ratio, the estimated required sample was 55 participants in the control arm and 110 participants in the intervention arm. The sample size estimate was split into 4 equally sized groups to estimate the number of participants needed to be recruited from each department. Originally, this 2-arm trial had an equal recruitment ratio across 4 departments, including the department of pain management, which planned to recruit patients with both chronic and acute postoperative pain []. However, recruitment of patients with postoperative pain ceased unexpectedly in March 2022, and the remaining recruitment was slow. To ensure the necessary statistical power and allow for more variation in the use of the intervention, our alterations prompted a new power analysis, and the follow-up period was reduced from 12 months to 6 months. Details of the power calculation and rationale for the effect size are summarized in the published protocol [].
Allocation and Blinding
This trial was a prospective, unblinded, nonrandomized controlled study. Allocation was conducted sequentially, meaning that patients were allocated to the control arm first, and when the required number was allocated, the participants were recruited to the intervention arm based on the sample size calculation. Allocation was conducted within each department, and when the required number of control participants were recruited, recruitment for the intervention arm began immediately, even if other departments were still recruiting for the control arm. Blinding was not possible because of the nature of the intervention.
Ethical Considerations
Approval for the project was obtained after review by the institutional data protection officer at the University Hospital of North Norway for the cancer department (2021/4942) and by the data protection officer at the Oslo University Hospital for the remaining departments (21/06826). The project protocol was prereviewed by the regional ethics committee and regarded as outside the mandate according to the Norwegian Health Research Act (regional ethics committee southeast reference number 252051). All participants provided informed and written consent. All data were anonymized before analyses, and only the first author (HH) had access to the anonymizing key to safeguard participant information. Patients did not receive any compensation for their participation.
Statistical Methods
All baseline data were reported as counts and percentages or means and SDs for normally distributed data and medians and ranges for skewed data. To investigate any baseline differences between the arms because of the lack of randomization, all baseline data were compared between the 2 arms using the chi-square test, Student t test (2-tailed), and Mann-Whitney U test, as appropriate. Missing data were not imputed. For all continuous variables, the individual mean change was estimated by subtracting the baseline score from the follow-up score at both 3 months and 6 months. Differences in mean changes between the arms were tested using the Student t test (2-tailed). A post hoc sensitivity analysis was performed using linear regression analysis, adjusting for the observed differences in the baseline characteristics between the arms (history of COVID-19 and domains 1 and 7 in eHLQ). P<.05 was considered statistically significant. Cronbach α was computed to evaluate the internal consistency of the scales based on the assumptions that the responses to individual questions were normally distributed, had equal variance, and equally explained the factor. All analyses were performed using SPSS (version 29; IBM Corp).
Results
Participant Flow
During the recruitment period, 162 patients signed the digital informed consent forms and completed consecutive self-reported baseline questionnaires. Given the 1:2 allocation ratio in favor of the intervention arm, we successfully allocated 55 (34%) patients to the control arm and 107 (66%) patients to the intervention arm. The participant flow and discontinuation are depicted in 
Figure 1. Overall, recruitment and allocation to the control arm started in September 2021, and recruitment to the intervention arm started in January 2022; sufficient participants were recruited by December 2022 (specific dates of the start and end of the recruitment are provided in ). Recruitment to the 2 arms differed slightly among the 4 departments. The data collection ended in June 2023.
In the control arm, 11% (6/55) of the participants did not respond at 3 months, while among those still alive and included (ie, not with retracted consent) at 3 months, 10% (5/52) of the participants did not respond at 6 months. In the intervention arm, 13.1% (14/107) of the participants did not respond at 3 months, and 6.9% (7/101) of the participants did not respond at 6 months.
Baseline Characteristics
The 162 participants included in this study, on average, were aged 52.6 (SD 15.8) years at baseline, with a range of 20 to 83 years (Table 2). There were 90 (56.6%) female participants; most (n=108, 66.7%) participants had a college education or higher, 99 (61.1%) were employed, and 32 (19.8%) were retired. There were no statistically significant differences between the control and intervention arms in terms of demographic or clinical characteristics, except for a history of COVID-19, subsequent fear of COVID-19 (Table 2), and a higher score in the intervention arm in eHLQ domains 1 and 7 (Table 3).
| Characteristics | Intervention arm (n=107), n (%) | Control arm (n=55), n (%) | P valuea | ||||||
| Demographic characteristics | |||||||||
| Age (y; N=162), mean (SD) | 51.8 (16.3) | 54.1 (14.8) | .22 | ||||||
| Age (y; N=162), median (range) | 56.0 (20-83) | 58.0 (22-78) | .29 | ||||||
| Female sex (N=162), n (%) | 60 (56.1) | 30 (54.5) | .99 | ||||||
| Education: college or more (N=162), n (%) | 68 (63.6) | 40 (72.7) | .32 | ||||||
| Employment status (n=161), n (%) | .11 | ||||||||
| Employed | 70 (65.4) | 29 (52.7) | |||||||
| Unemployed | 15 (14) | 15 (27.3) | |||||||
| Retired | 21 (19.6) | 11 (20) | |||||||
| Cohabitation status: cohabiting (N=162), n (%) | 75 (70.1) | 41 (74.5) | .68 | ||||||
| Access to digital devices (N=162), n (%) | |||||||||
| Use of smartphone | 106 (99.1) | 54 (98.2) | .63 | ||||||
| Weekly use of iPad | 50 (46.7) | 31 (56.4) | .32 | ||||||
| Weekly use of PC | 92 (86) | 42 (76.4) | .19 | ||||||
| Use of other health apps: yes | 47 (43.9) | 23 (41.8) | .80 | ||||||
| Clinical characteristics | |||||||||
| Diagnosis, n (%) | .07 | ||||||||
| Chronic pain (n=33) | 21 (19.6) | 12 (21.8) | |||||||
| Epilepsy (n=53) | 40 (37.4) | 13 (23.6) | |||||||
| Interstitial lung disease (n=39) | 26 (24.3) | 13 (23.6) | |||||||
| Cancer (n=34) | 20 (18.7) | 14 (25.5) | |||||||
| Postoperative painb (n=3) | 0 (0) | 3 (5.5) | |||||||
| Diagnosis duration (y; n=156), median (range) | 5.0 (0-40) | 5.0 (0-32) | .43 | ||||||
| History of COVID-19 (n=160), n (%) | 53 (49.5) | 5 (9.1) | <.001c | ||||||
| Fear of COVID-19d (n=102), n (%) | .03 | ||||||||
| Not afraid | 25 (47.2) | 11 (22.4) | |||||||
| A little afraid | 24 (45.3) | 30 (61.2) | |||||||
| Very afraid | 4 (7.5) | 8 (16.3) | |||||||
| Smoking or use of snuff: yes (n=161), n (%) | 17 (15.9) | 11 (20) | .52 | ||||||
| Alcohol consumption: weekly or more (N=162), n (%) | 58 (54.2) | 32 (58.2) | .63 | ||||||
aBetween-arm differences were analyzed using the Student t test (2-tailed) and chi-square test for comparison of proportions as well as the Mann-Whitney U test for comparison of ranks (median).
bRecruitment of patients with postoperative pain was terminated before the required sample was allocated.
cStatistically significant values are italicized.
dOnly those with no history of COVID-19 were asked about their fear.
| Intervention arm (n=107), mean (SD) | Control arm (n=55), mean (SD) | P value | |||||
| Health literacy (N=162; 5 of the 9 domains of the HLQa) | |||||||
| Domain 1: feeling understood and supported by health care providersb | 3.11 (0.52) | 3.10 (0.47) | .91 | ||||
| Domain 2: having sufficient information to manage my own healthb | 2.90 (0.50) | 2.88 (0.46) | .74 | ||||
| Domain 3: actively managing my own healthb | 2.85 (0.49) | 3.01 (0.46) | .05 | ||||
| Domain 6: ability to actively engage with health care providersc | 3.81 (0.72) | 3.83 (0.59) | .85 | ||||
| Domain 9: understanding health information well enough to know what to doc | 3.86 (0.55) | 3.98 (0.43) | .16 | ||||
| Digital/eHealth literacy (N=162; 7 of the 7 domains of the eHLQd)e | |||||||
| Domain 1: using technology to process health information | 2.89 (0.55) | 2.70 (0.58) | .046f | ||||
| Domain 2: understanding of health concepts and language | 2.97 (0.45) | 3.00 (0.43) | .75 | ||||
| Domain 3: ability to actively engage with digital services | 3.17 (0.51) | 3.01 (0.58) | .06 | ||||
| Domain 4: feel safe and in control | 3.15 (0.49) | 3.23 (0.48) | .38 | ||||
| Domain 5: motivated to engage with digital services | 2.92 (0.45) | 2.81 (0.52) | .16 | ||||
| Domain 6: access to digital services that work | 2.73 (0.46) | 2.64 (0.44) | .19 | ||||
| Domain 7: digital services that suit individual needs | 2.83 (0.49) | 2.63 (0.55) | .02 | ||||
| RAND-12g (n=161) | |||||||
| Physical component score | 39.2 (11.2) | 40.2 (11.2) | .39 | ||||
| Mental component score | 42.5 (10.4) | 42.8 (10.4) | .76 | ||||
aHLQ: Health Literacy Questionnaire.
bScored 1 to 4, where higher values indicate higher health literacy.
cScored 1 to 5, where higher values indicate higher health literacy.
deHLQ: eHealth Literacy Questionnaire.
eScored 1 to 4, where higher values indicate higher eHealth literacy.
fStatistically significant values are italicized.
gScored 0 to 100, where a higher standardized score indicates better health.
Use of the Digital Service Intervention
The participants in the intervention arm used an Apple iOS device (52/106, 49.1%) more frequently than an Android operating system (31/106, 29.2%; missing information on system: 23/106, 21.7%). After 3 months, 75% (65/87) of the intervention arm responders self-reported still using the MyDignio app, and after 6 months, 8.6% (91/106) of the participants were categorized as high users. The use of the intervention was reported for the entire 6-month intervention (Table 4). Although patients from the cancer, lung, and neurology departments were expected to use the digital outpatient care throughout the 6 months they were in the trial, the pain department had patients in digital outpatient care for the purpose of both pain medication reduction and pain medication adjustments, typically for a shorter time than 6 months. Thus, patients from the pain department used the app for an average of 11.3 (SD 6.5) weeks, with a median of 9.5 (range 3-27) weeks.
| Interactions | Intervention arm | |||
| Digital interactionb | ||||
| Digital interactions, n | ||||
| Mean (SD) | 42.8 (52.3) | |||
| Median (range) | 11.5 (0-265) | |||
| Dichotomized digital interactions, n (%) | ||||
| Low use | 15 (14.2) | |||
| High use | 91 (85.8) | |||
| Asynchronous chat messages, n | ||||
| Messages sent by patient | ||||
| Mean (SD) | 6.1 (8.8) | |||
| Median (range) | 2.0 (0-57) | |||
| Messages sent by health care worker | ||||
| Mean (SD) | 5.5 (6.6) | |||
| Median (range) | 2.0 (0-27) | |||
| Messages, n | ||||
| Mean (SD) | 11.6 (14.7) | |||
| Median (range) | 5.0 (0-76) | |||
| PROc measures, n | ||||
| PRO measures sent to patient | ||||
| Mean (SD) | 14.2 (18.8) | |||
| Median (range) | 9.0 (0-120) | |||
| PRO measure responses from patient | ||||
| Mean (SD) | 11.8 (17.7) | |||
| Median (range) | 4.0 (0-114) | |||
| Monitoring (only patients with ILDd) | ||||
| Mean (SD) | 79.0 (36.8) | |||
| Median (range) | 76 (0-176) | |||
an=1 missing.
bComputed value of the total number of chat messages, videos, completed patient-reported outcome measures, and monitoring.
cPRO: patient-reported outcome.
dILD: interstitial lung disease.
Changes After the Intervention
After 3 months, we observed a positive change in health literacy, which was measured using the HLQ (mean 0.08, SD 0.44) in the intervention arm, while a negative change (mean change –0.10, SD 0.51; P=.03) was observed in the control arm (Table 5). Similarly, in the HLQ domain 3, the intervention arm had a positive mean change compared with a negative mean change in the control arm. However, this change was not clinically significant (P<.50).
In HRQoL, a favorable mean change was observed in the intervention arm compared with the control arm for the physical component score (mean difference –3.29; P=.006) and mental component score (mean difference –3.08; P=.02).
For the primary outcome, health literacy after 6 months, which was measured using the HLQ domain 9, we did not observe any statistically significant differences in the scores (mean difference –0.05, 95% CI –0.20 to 0.10; P=.53; Table 6). Accounting for the differences in the history of COVID-19 and baseline eHLQ scores in domain 1 and 7 between the arms did not influence the results (mean difference –0.04; 95% CI –0.13 to 0.21, P=.66). However, in the eHLQ, there was a statistically significant difference in domain 4 regarding feeling safe and in control, where the control arm had a positive change, while the intervention arm had a negative mean change.
| Variables, domains, and arms | Baseline, mean (SD) | T1, mean (SD) | Change, mean (SD) | |||||
| Health literacy (5 of the 9 domains of the HLQa) | ||||||||
| Domain 1: feeling understood and supported by health care providers | ||||||||
| Intervention (n=88) | 3.11 (0.52) | 3.14 (0.51) | 0.04 (0.49) | |||||
| Control (n=47) | 3.10 (0.47) | 3.10 (0.47) | –0.02 (0.48) | |||||
| Domain 2: having sufficient information to manage my own health | ||||||||
| Intervention (n=88) | 2.90 (0.50) | 3.01 (0.45) | 0.10 (0.40) | |||||
| Control (n=47) | 2.88 (0.46) | 2.88 (0.54) | 0.02 (0.46) | |||||
| Domain 3: actively managing my own health | ||||||||
| Intervention (n=88) | 2.85 (0.49) | 2.97 (0.48) | 0.13b(0.38)c | |||||
| Control (n=47) | 3.01 (0.46) | 3.03 (0.47) | 0.02 (0.50)c | |||||
| Domain 6: ability to actively engage with health care providers | ||||||||
| Intervention (n=88) | 3.81 (0.72) | 3.86 (0.61) | 0.02 (0.51) | |||||
| Control (n=47) | 3.83 (0.59) | 3.73 (0.67) | –0.12 (0.51) | |||||
| Domain 9: understanding health information well enough to know what to do | ||||||||
| Intervention (n=88) | 3.86 (0.55) | 3.99 (0.46) | 0.08 (0.44)d | |||||
| Control (n=47) | 3.98 (0.43) | 3.90 (0.57) | –0.10 (0.51)d | |||||
| Digital/eHealth literacy (7 of the 7 domains of the eHLQe) | ||||||||
| Domain 1: using technology to process health information | ||||||||
| Intervention (n=87) | 2.89 (0.55) | 2.92 (0.47) | 0.03 (0.44) | |||||
| Control (n=46) | 2.70 (0.58) | 2.78 (0.58) | 0.03 (0.37) | |||||
| Domain 2: understanding of health concepts and language | ||||||||
| Intervention (n=88) | 2.97 (0.45) | 3.04 (0.43) | 0.06 (0.39) | |||||
| Control (n=46) | 3.00 (0.43) | 3.04 (0.43) | 0.00 (0.34) | |||||
| Domain 3: ability to actively engage with digital services | ||||||||
| Intervention (n=88) | 3.17 (0.51) | 3.19 (0.47) | 0.02 (0.37) | |||||
| Control (n=47) | 3.01 (0.58) | 3.03 (0.63) | –0.04 (0.28) | |||||
| Domain 4: feel safe and in control | ||||||||
| Intervention (n=88) | 3.15 (0.49) | 3.03 (0.49) | –0.10 (0.38) | |||||
| Control (n=47) | 3.23 (0.48) | 3.13 (0.52) | –0.12 (0.44) | |||||
| Domain 5: motivated to engage with digital services | ||||||||
| Intervention (n=88) | 2.92 (0.45) | 2.88 (0.51) | –0.04 (0.46) | |||||
| Control (n=47) | 2.81 (0.52) | 2.73 (0.52) | –0.10 (0.46) | |||||
| Domain 6: access to digital services that work | ||||||||
| Intervention (n=88) | 2.73 (0.46) | 2.73 (0.45) | 0.01 (0.42) | |||||
| Control (n=47) | 2.64 (0.44) | 2.61 (0.53) | –0.03 (0.43) | |||||
| Domain 7: digital services that suit individual needs | ||||||||
| Intervention (n=88) | 2.83 (0.49) | 2.88 (0.48) | 0.08 (0.42) | |||||
| Control (n=47) | 2.63 (0.55) | 2.59 (0.54) | –0.07 (0.52) | |||||
| HRQoL | ||||||||
| PCSf | ||||||||
| Intervention (n=90) | 40.2 (11.1) | 43.6 (11.7) | 1.73 (6.41)g | |||||
| Control (n=45) | 39.2 (11.2) | 38.0 (10.7) | –1.57 (6.68)g | |||||
| MCSh | ||||||||
| Intervention (n=90) | 43.0 (10.4) | 44.7 (11.1) | 1.10 (7.43)i | |||||
| Control (n=45) | 42.5 (10.4) | 41.2 (11.7) | –1.98 (6.96)i | |||||
aHLQ: Health Literacy Questionnaire.
bStatistically significant values are italicized.
cThe difference between the arms was in favor of the intervention arm (mean difference –0.15, 95% CI –0.30 to –0.00; P=.048).
dThe difference between the arms was in favor of the intervention arm (mean difference –0.17, 95% CI –0.34 to –0.00; P=.03).
eeHLQ: eHealth Literacy Questionnaire.
fPCS: physical component score.
gPCS difference between the arms was in favor of the intervention arm (mean difference –3.29, 95% CI –5.62 to –0.96; P=.006).
hMCS: mental component score.
iMCS difference between the arms was in favor of the intervention arm (mean difference –3.08, 95% CI –5.64 to –0.52; P=.02).
| Variables, domains, and arms | Baseline, mean (SD) | T2, mean (SD) | Change, mean (SD) | |||||
| Primary outcome: health literacy (domain 9 of the HLQa) | ||||||||
| Domain 9: understanding health information well enough to know what to do | ||||||||
| Intervention (n=91) | 3.86 (0.55) | 3.92 (0.49) | 0.03 (0.44) | |||||
| Control (n=45) | 3.98 (0.43) | 3.96 (0.54) | –0.02 (0.38) | |||||
| Health literacy (4 of the 9 domains of the HLQ) | ||||||||
| Domain 1: feeling understood and supported by health care providers | ||||||||
| Intervention (n=91) | 3.11 (0.52) | 3.10 (0.48) | –0.01 (0.47) | |||||
| Control (n=45) | 3.10 (0.47) | 3.17 (0.49) | 0.04 (0.49) | |||||
| Domain 2: having sufficient information to manage my own health | ||||||||
| Intervention (n=91) | 2.90 (0.50) | 3.01 (0.50) | 0.08 (0.55) | |||||
| Control (n=45) | 2.88 (0.46) | 2.93 (0.47) | 0.05 (0.49) | |||||
| Domain 3: actively managing my own health | ||||||||
| Intervention (n=91) | 2.85 (0.49) | 2.96 (0.43) | 0.11 (0.40) | |||||
| Control (n=45) | 3.01 (0.46) | 3.01 (0.53) | –0.01 (0.51) | |||||
| Domain 6: ability to actively engage with health care providers | ||||||||
| Intervention (n=91) | 3.81 (0.72) | 3.87 (0.57) | 0.02 (0.43) | |||||
| Control (n=45) | 3.83 (0.59) | 3.73 (0.63) | –0.12 (0.52) | |||||
| Digital/eHealth literacy (7 of the 7 domains of the eHLQb) | ||||||||
| Domain 1: using technology to process health information | ||||||||
| Intervention (n=91) | 2.89 (0.55) | 2.92 (0.49) | 0.02 (0.41) | |||||
| Control (n=45) | 2.70 (0.58) | 2.83 (0.63) | 0.16 (0.44) | |||||
| Domain 2: understanding of health concepts and language | ||||||||
| Intervention (n=91) | 2.97 (0.45) | 3.01 (0.40) | 0.03 (0.36) | |||||
| Control (n=45) | 3.00 (0.43) | 3.10 (0.47) | 0.12 (0.43) | |||||
| Domain 3: ability to actively engage with digital services | ||||||||
| Intervention (n=91) | 3.17 (0.51) | 3.14 (0.46) | –0.04 (0.38) | |||||
| Control (n=45) | 3.01 (0.58) | 3.10 (0.63) | 0.09 (0.45) | |||||
| Domain 4: feel safe and in control | ||||||||
| Intervention (n=91) | 3.15 (0.49) | 3.01 (0.42) | –0.15c(0.39)d | |||||
| Control (n=45) | 3.23 (0.48) | 3.25 (0.52) | 0.01 (0.46)d | |||||
| Domain 5: motivated to engage with digital services | ||||||||
| Intervention (n=91) | 2.92 (0.45) | 2.89 (0.51) | –0.03 (0.44) | |||||
| Control (n=45) | 2.81 (0.52) | 2.81 (0.61) | 0.00 (0.47) | |||||
| Domain 6: access to digital services that work | ||||||||
| Intervention (n=91) | 2.73 (0.46) | 2.27 (0.41) | –0.00 (0.47) | |||||
| Control (n=45) | 2.64 (0.44) | 2.70 (0.55) | 0.06 (0.51) | |||||
| Domain 7: digital services that suit individual needs | ||||||||
| Intervention (n=91) | 2.83 (0.49) | 2.88 (0.43) | 0.04 (0.46) | |||||
| Control (n=45) | 2.63 (0.55) | 2.70 (0.65) | 0.09 (0.57) | |||||
| HRQoL | ||||||||
| PCSe | ||||||||
| Intervention (n=90) | 40.8 (11.1) | 43.6 (10.9) | 2.10 (7.85) | |||||
| Control (n=45) | 39.2 (11.2) | 39.2 (11.7) | –0.12 (6.95) | |||||
| MCSf | ||||||||
| Intervention (n=90) | 43.0 (10.4) | 45.2 (10.5) | 1.79 (8.60) | |||||
| Control (n=45) | 42.5 (10.4) | 44.3 (11.6) | 0.82 (7.75) | |||||
aHLQ: Health Literacy Questionnaire.
beHLQ: eHealth Literacy Questionnaire.
cStatistically significant values are italicized.
dThe difference between the arms was in favor of the control arm (mean difference 0.16, 95% CI 0.01-0.31; P=.046).
ePCS: physical component score.
fMCS: mental component score.
Patient Satisfaction at 6-Month Follow-Up
There were no differences in general satisfaction with the outpatient service at 6-month follow-up, with 76% (34/45) of the participants in the control arm and 78% (71/91) in the intervention arm being satisfied (P=.42). There were no differences between the arms in terms of the perceived effect of the treatment on their own health; 18% (8/45) of those in the control arm reported a decline in their health compared with 14% (13/91) in the intervention arm, while the remaining participants reported an improvement or no change (P=.60). The proportion of participants stating that a digital outpatient service would affect their feeling of safety was 49% (27/55) of the control participants and 61.7% (66/107) of the intervention participants (P=.09).
The participants in the intervention arm were generally satisfied or had little concern about the digital intervention (Table 7). Technical difficulties were reported by 18.9% (20/106) of the participants at either the 3- or 6-month follow-up.
| SUTAQa domainsb | Values, mean (SD) |
| Domain 1: enhanced care | 5.10 (1.11) |
| Domain 2: increased accessibility | 4.60 (1.17) |
| Domain 3: privacy and discomfort | 2.99 (0.89) |
| Domain 4: care personnel concerns | 3.27 (0.86)c |
| Domain 5: kit as a substitute | 4.04 (0.94)d |
| Domain 6: satisfaction | 5.35 (1.27) |
aSUTAQ: Service User Technology Acceptability Questionnaire.
bFor domains 1, 2, 5, and 6, a higher score indicates higher satisfaction, while domains 3 and 4 are reversed, where a higher score represents a concern.
cCronbach α=0.62.
dCronbach α=0.46.
Use of Health Care Services
There were no differences between the 2 arms after 6 months in terms of medication changes or the number of times patients had contacted the outpatient clinic. The intervention arm reported significantly fewer hospital admissions than the control arm (P=.03). The intervention participants reported 37 hospital admissions (mean 0.35, SD 1.1; range 0-7) among 13 (12.1%) of 107 participants, while the control arm reported 25 hospital admissions (mean 0.5, SD 1.05; range 0-5) among 13 (26%) of 50 control participants.
Discussion
Principal Findings
This study reports findings from a study of a novel multicomponent digital health intervention comprising PRO measures, self-monitoring of physiological measures, and asynchronous chat messages among a heterogeneous group of patients in outpatient care. We did not observe any statistically significant differences in the HLQ domain 9, which assesses patients’ understanding of health information well enough to know what to do, even though this study was presumably adequately powered. Our results demonstrate a trend that digital outpatient care, at the 3-month follow-up, improved health literacy domains encompassing patients’ ability to actively manage their own health and whether they understood health information well enough to know what to do. In addition, both physical and mental HRQoL improved after 3 months of digital outpatient care compared with those in the control arm; however, this statistically significant change was not considered clinically significant (P<.50). At the 6-month term, these positive findings were not observed; in contrast, the patients in the intervention arm demonstrated a statistically significant decline in digital/eHealth literacy in the eHLQ domain feel safe and in control. Overall, the participants in digital outpatient care had a high satisfaction rate when evaluating the digital outpatient care platform. Previous research was inconclusive regarding the outcomes of digital health interventions [-]. Our study adds to this field by being the first study to explore the associated outcomes of a multicomponent and comprehensive digital outpatient intervention. Despite the noted limitations of our study, some reflections on our findings will be relevant to bring the research field forward.
Several mechanisms may explain our findings. Specifically, we argue that digital outpatient care is positively associated with health literacy and HRQoL because of several mechanisms of the digital care model. First, the use of digital outpatient care allows for the collection of PRO measures on a regular basis, thus increasing patient involvement. Favorable health outcomes related to such involvement have previously been observed [], although evidence remains limited because of the heterogeneity of research studies in this field []. In our study, both the patients and the health care workers had access to a convenient tool for short, asynchronous messages through the digital platform. In line with previous research [,], for many of our patients, these asynchronous messages served as a tool to clarify minor questions, provide little information regarding their clinical status and self-management, and give a response without having to wait in a telephone queue. Finally, the department providing care for patients with ILD offered the patients equipment for home spirometry and pulse oximetry. Such home monitoring gave the patients their first experience of truly being able to self-manage their condition from home while data were sent to health care workers to ensure that proper care was provided.
Answering disease-related PRO measures can, in itself, increase reflection regarding self-management and skills in patients [,]. Favorable outcomes are also seen among those with the opportunity to ask questions regarding their own condition [,]. Altogether, enabling reflection and asking questions in the digital service might be the 2 crucial components supporting the short-term increase in health literacy in our study. Furthermore, limited health literacy has been suggested to predict the increased use of health resources []. Conversely, the observed increase in health literacy in our study might reduce the use of health resources over time, confirming previous research that found an association between supporting health literacy and improved self-management [,]. We found significantly fewer hospital admissions in the intervention arm, although this finding was not observed for the number of contacts with the outpatient clinic. The experience of increased self-management can also reduce the negative impact of living with a long-term or chronic condition, thus explaining the favorable change in HRQoL. However, as improved health literacy at 3 months was not found at 6 months, some uncertainty remains, and more research is needed to explore the association between health literacy and the use of health service resources.
Although the participants used digital tools to varying extents based on clinical indications, these new services provided favorable short-term outcomes. Short-term changes in favor of an intervention are often demonstrated, while long-term effects are more challenging to observe. Although we found an increase in health literacy and HRQoL after 3 months, we did not observe this increase after 6 months. Instead, we found a negative change in the digital/eHealth literacy domain concerning feelings of safety and control. We cannot fully explain these findings based on our data; however, we believe that the 6-month contact point represents a vulnerable time for patients in a digital outpatient care model. Initially, the new services spark curiosity, foster a sense of increased self-management, and help patients feel more capable of doing the right thing. However, after some time, this may change as patients develop their skills, use more digital care, and simultaneously become more aware of the elements they cannot control. Thus, while they might, in fact, have better control, their increased skills make them more cognizant of the uncertain factors, which creates the feeling of less control that we found. Another mechanism supporting this hypothesis is that despite the lower feeling of control, the participants reported high satisfaction with the digital service after 6 months. Previous research has not observed or discussed potential solutions to such issues. Thus, future research should be aware of the potentially increased need for support among patients during participation in a digital outpatient care model. This is not only to reduce dropout but also to reduce any potential feeling of losing control.
Patients with a persistent need for health services to support their self-management challenge traditional outpatient care, and new ways to provide patient care are warranted to uphold sustainable services of high quality []. Most of the participants in our intervention arm used digital outpatient care as intended; that is, they responded to PRO measures, uploaded self-monitoring measures, and asked for clarification when needed. These findings contrast with previous research, which suggests that maintaining participant engagement through a digital intervention can be challenging [], even through a life course with chronic or long-term conditions [,]. Factors that are important for ensuring the use of digital care include a referral from a health care worker and an interest among both the patients and the health care workers in digital services []. In our study, we observed a low dropout rate and high satisfaction, suggesting that the patients were content with the study and the digital outpatient care they received. Whether the use and satisfaction of digital outpatient care would have been the case in a longer term beyond 6 months was beyond the scope of our study and should be investigated in future research. However, given how well the digital outpatient care was received and used by the patients, we suggest that models such as ours should be investigated further to explore their relevance and effects on clinical practice.
Limitations
This study has some limitations, several of which were discussed in the published protocol []. First, there is a risk of selection bias because of the lack of randomization into either a control or intervention arm. However, notably, because the allocation was performed sequentially (first for the control arm and then for the intervention arm), there was no bias directly due to patient preferences. Moreover, the 2 arms were similar at baseline except for the history of COVID-19 and subsequent fear of COVID-19 and eHLQ domains 1, using technology to process health information, and 7, digital services that suit individual needs. The control arm was mainly recruited when very few people had contracted COVID-19, while the intervention arm was recruited when the strategy had shifted because of vaccinations and more people contracting COVID-19. As a result, the timeline of the COVID-19 pandemic and eHLQ may have caused some bias in this study. However, sensitivity analysis adjusted for baseline differences provided similar findings.
Second, we do not know how many patients were assessed for eligibility because these data were not sufficiently systematized during enrollment. We had to alter recruitment during the trial from a 1:1 allocation to a 1:2 allocation because of slow recruitment []. However, the revised power calculation with a sequential 1:2 allocation provided a sufficient sample size as anticipated a priori. This study was not powered to detect clinical differences within each of the 4 patient groups (ie, from the pain, lung, cancer, and epilepsy departments) or in any of the secondary objectives. Because of the many tests performed, false positive findings might have occurred, so the results should be interpreted with caution. Furthermore, we cannot provide evidence of the findings of digital outpatient care on disease-specific clinical parameters. Rather, we suggest that future research investigate whether our observed changes in health literacy affect disease-specific clinical or health outcomes over time, as has been observed in other studies [,].
Conclusions
This study explored digital outpatient care comprising PRO measures; asynchronous messaging; and remote monitoring of clinical indications for patients with chronic pain, ILD, epilepsy, or cancer. No significant differences were observed between the 2 arms in our primary outcome, assessing patients’ health literacy reflected through their understanding of health information after 6 months. Our data indicate an improvement in some health literacy domains and HRQoL at 3 months among the patients using the digital solution. Despite our mixed results, the participants reported high satisfaction with the digital outpatient care intervention; therefore, digital health interventions can be positive for people with chronic conditions and are well received. However, our findings are not entirely consistent, and more research should explore the mixed changes in health or eHealth literacy, the effectiveness of such interventions on various outcomes, and the relationship between participant satisfaction and clinical effects.
Acknowledgments
The authors would like to sincerely acknowledge all the participating patients. They would like to express their utmost appreciation to all the health care workers participating in the development and patient care in this study, particularly Saida Alfrida Martinez Overgaard, Lotte Sandberg Larsen, Monica Mellegård, Anne-Maria Johanna Tanskanen, Erik Taubøll, Merete Lyngstad, Markus Sonnenberg, Kristin Iren Jensen, Magnar Johansen, Audun Stubhaug, Lars Petter Granan, Unni Halvorsen, and Karl Arne Johannessen, as well as to their project team at Dignio Connected Care for facilitating the digital platform. Finally, the authors would like to thank the staff in Dignio for their valuable insights and practical tailoring of their platform and good collaboration, particularly with Anna Hurrød, Meetali Kakad, and Andreas Norling. This work was supported by the Research Council of Norway (grant 316244), internal funding by the Oslo University Hospital and the University Hospital of North Norway to secure staff, and funding from Dignio Connected Care to finance user licenses on the digital platform. The Research Council of Norway and Dignio Connected Care had no role in preparing this manuscript.
Data Availability
The datasets generated or analyzed during this study are available from the corresponding author on reasonable request.
Authors' Contributions
HH drafted and revised the manuscript. AMH, TKK, TML, CE, and EF made a substantial contribution to the development of the intervention. HH conducted all analyses with support from RSF. AMH, RSF, TKK, TML, CE, and EF commented on and revised the versions of the draft, and all authors approved the final manuscript.
Conflicts of Interest
None declared.
Multimedia Appendix 1
Screenshots of MyDignio patient app and Dignio Prevent interface.
DOCX File , 237 KBReferences
- Schreiweis B, Pobiruchin M, Strotbaum V, Suleder J, Wiesner M, Bergh B. Barriers and facilitators to the implementation of eHealth services: systematic literature analysis. J Med Internet Res. Nov 22, 2019;21(11):e14197. [CrossRef] [Medline]
- Cheng C, Beauchamp A, Elsworth GR, Osborne RH. Applying the electronic health literacy lens: systematic review of electronic health interventions targeted at socially disadvantaged groups. J Med Internet Res. Aug 13, 2020;22(8):e18476. [FREE Full text] [CrossRef] [Medline]
- Stoumpos AI, Kitsios F, Talias MA. Digital transformation in healthcare: technology acceptance and its applications. Int J Environ Res Public Health. Feb 15, 2023;20(4):33. [FREE Full text] [CrossRef] [Medline]
- Gandrup J, Ali SM, McBeth J, van der Veer SN, Dixon WG. Remote symptom monitoring integrated into electronic health records: a systematic review. J Am Med Inform Assoc. Nov 01, 2020;27(11):1752-1763. [FREE Full text] [CrossRef] [Medline]
- Garg S, Williams NL, Ip A, Dicker AP. Clinical integration of digital solutions in health care: an overview of the current landscape of digital technologies in cancer care. JCO Clin Cancer Inform. Dec 2018;2:1-9. [CrossRef] [Medline]
- Richesson RL, Marsolo KS, Douthit BJ, Staman K, Ho PM, Dailey D, et al. Enhancing the use of EHR systems for pragmatic embedded research: lessons from the NIH Health Care Systems Research Collaboratory. J Am Med Inform Assoc. Nov 25, 2021;28(12):2626-2640. [FREE Full text] [CrossRef] [Medline]
- Barbati C, Maranesi E, Giammarchi C, Lenge M, Bonciani M, Barbi E, et al. Effectiveness of eHealth literacy interventions: a systematic review and meta-analysis of experimental studies. BMC Public Health. Jan 23, 2025;25(1):288. [FREE Full text] [CrossRef] [Medline]
- Martins AI, Ribeiro Ó, Santinha G, Silva T, Rocha NP, Silva AG. Effectiveness of integrated digital solutions to empower older adults in aspects related to their health: systematic review and meta-analysis. J Med Internet Res. Jan 09, 2025;27:e54466. [FREE Full text] [CrossRef] [Medline]
- Verweel L, Newman A, Michaelchuk W, Packham T, Goldstein R, Brooks D. The effect of digital interventions on related health literacy and skills for individuals living with chronic diseases: a systematic review and meta-analysis. Int J Med Inform. Sep 2023;177:105114. [CrossRef] [Medline]
- Blumenthal D, Chernof B, Fulmer T, Lumpkin J, Selberg J. Caring for high-need, high-cost patients - an urgent priority. N Engl J Med. Sep 08, 2016;375(10):909-911. [CrossRef] [Medline]
- GBD 2019 Demographics Collaborators. Global age-sex-specific fertility, mortality, healthy life expectancy (HALE), and population estimates in 204 countries and territories, 1950-2019: a comprehensive demographic analysis for the Global Burden of Disease Study 2019. Lancet. Oct 17, 2020;396(10258):1160-1203. [FREE Full text] [CrossRef] [Medline]
- Nutbeam D. Health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. Health Promot Int. 2000;15(3):259-267. [CrossRef]
- Larsen MH, Mengshoel AM, Andersen MH, Borge CR, Ahlsen B, Dahl KG, et al. "A bit of everything": health literacy interventions in chronic conditions - a systematic review. Patient Educ Couns. Oct 2022;105(10):2999-3016. [FREE Full text] [CrossRef] [Medline]
- van der Gaag M, Heijmans M, Spoiala C, Rademakers J. The importance of health literacy for self-management: a scoping review of reviews. Chronic Illn. Jun 2022;18(2):234-254. [CrossRef] [Medline]
- Paasche-Orlow MK, Wolf MS. The causal pathways linking health literacy to health outcomes. Am J Health Behav. Jul 01, 2007;31(1):19-26. [CrossRef]
- Nutbeam D. Health promotion glossary. Health Promot Int. Jan 01, 1998;13(4):349-364. [CrossRef]
- Baccolini V, Rosso A, Di Paolo C, Isonne C, Salerno C, Migliara G, et al. What is the prevalence of low health literacy in European Union member states? A systematic review and meta-analysis. J Gen Intern Med. Mar 2021;36(3):753-761. [CrossRef] [Medline]
- Berkman ND, Sheridan SL, Donahue KE, Halpern DJ, Crotty K. Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. Jul 19, 2011;155(2):97-107. [CrossRef] [Medline]
- Holmen H, Holm AM, Kilvær TK, Ljoså TM, Granan LP, Ekholdt C, et al. Digital outpatient services for adults: development of an intervention and protocol for a multicenter non-randomized controlled trial. JMIR Res Protoc. Jul 10, 2023;12:e46649. [FREE Full text] [CrossRef] [Medline]
- Greenhalgh J, Gooding K, Gibbons E, Dalkin S, Wright J, Valderas J, et al. How do patient reported outcome measures (PROMs) support clinician-patient communication and patient care? A realist synthesis. J Patient Rep Outcomes. Dec 2018;2:42. [FREE Full text] [CrossRef] [Medline]
- Boyce MB, Browne JP. Does providing feedback on patient-reported outcomes to healthcare professionals result in better outcomes for patients? A systematic review. Qual Life Res. Nov 2013;22(9):2265-2278. [CrossRef] [Medline]
- Nelson EC, Eftimovska E, Lind C, Hager A, Wasson JH, Lindblad S. Patient reported outcome measures in practice. BMJ. Feb 10, 2015;350:g7818. [CrossRef] [Medline]
- Althobiani MA, Evans RA, Alqahtani JS, Aldhahir AM, Russell A, Hurst JR, et al. Home monitoring of physiology and symptoms to detect interstitial lung disease exacerbations and progression: a systematic review. ERJ Open Res. Oct 2021;7(4):25. [FREE Full text] [CrossRef] [Medline]
- Escriva Boulley G, Leroy T, Bernetière C, Paquienseguy F, Desfriches-Doria O, Préau M. Digital health interventions to help living with cancer: a systematic review of participants' engagement and psychosocial effects. Psychooncology. Dec 2018;27(12):2677-2686. [CrossRef] [Medline]
- Hewitt S, Sephton R, Yeowell G. The effectiveness of digital health interventions in the management of musculoskeletal conditions: systematic literature review. J Med Internet Res. Jun 05, 2020;22(6):e15617. [FREE Full text] [CrossRef] [Medline]
- Shegog R, Braverman L, Hixson JD. Digital and technological opportunities in epilepsy: toward a digital ecosystem for enhanced epilepsy management. Epilepsy Behav. Jan 2020;102:106663. [CrossRef] [Medline]
- Kidholm K, Ekeland AG, Jensen LK, Rasmussen J, Pedersen CD, Bowes A, et al. A model for assessment of telemedicine applications: mast. Int J Technol Assess Health Care. Jan 2012;28(1):44-51. [FREE Full text] [CrossRef] [Medline]
- von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. Apr 2008;61(4):344-349. [CrossRef] [Medline]
- Calvert M, Kyte D, Mercieca-Bebber R, Slade A, Chan A, King MT, the SPIRIT-PRO Group, et al. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the SPIRIT-PRO extension. JAMA. Feb 06, 2018;319(5):483-494. [FREE Full text] [CrossRef] [Medline]
- Nettskjema. University of Oslo. URL: https://www.uio.no/english/services/it/adm-services/nettskjema/ [accessed 2024-04-29]
- Connected care. Dignio. URL: https://dignio.com/ [accessed 2024-04-29]
- Watanabe SM, Nekolaichuk C, Beaumont C, Johnson L, Myers J, Strasser F. A multicenter study comparing two numerical versions of the Edmonton Symptom Assessment System in palliative care patients. J Pain Symptom Manage. Feb 2011;41(2):456-468. [FREE Full text] [CrossRef] [Medline]
- Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, McFadden ET, et al. Toxicity and response criteria of the Eastern Cooperative Oncology group. Am J Clin Oncol. Dec 1982;5(6):649-655. [Medline]
- Patel AS, Siegert RJ, Brignall K, Gordon P, Steer S, Desai SR, et al. The development and validation of the King's Brief Interstitial Lung Disease (K-BILD) health status questionnaire. Thorax. Sep 2012;67(9):804-810. [CrossRef] [Medline]
- National network for evidence-based epilpsy care in Norway. EpilepsyNet. URL: https://www.epilepsinett.org/digitale-verkt%C3%B8y [accessed 2024-04-29]
- Osborne RH, Batterham RW, Elsworth GR, Hawkins M, Buchbinder R. The grounded psychometric development and initial validation of the Health Literacy Questionnaire (HLQ). BMC Public Health. Jul 16, 2013;13:658. [FREE Full text] [CrossRef] [Medline]
- Wahl AK, Hermansen Å, Osborne RH, Larsen MH. A validation study of the Norwegian version of the health literacy questionnaire: a robust nine-dimension factor model. Scand J Public Health. Jun 2021;49(4):471-478. [FREE Full text] [CrossRef] [Medline]
- Kayser L, Karnoe A, Furstrand D, Batterham R, Christensen KB, Elsworth G, et al. A multidimensional tool based on the eHealth literacy framework: development and initial validity testing of the eHealth literacy questionnaire (eHLQ). J Med Internet Res. Feb 12, 2018;20(2):e36. [FREE Full text] [CrossRef] [Medline]
- Hermansen Å, Andersen MH, Borge CR, Dahl KG, Larsen MH, Lønning K, et al. Preliminary validity testing of the eHealth Literacy Questionnaire (eHLQ): a Confirmatory Factor Analysis (CFA) in Norwegian hospitalized patients. BMC Psychol. Nov 23, 2023;11(1):409. [FREE Full text] [CrossRef] [Medline]
- Hays RD, Prince-Embury S, Chen H. RAND-36 health status inventory. Psychological Corporation. 1998. URL: https://labs.dgsom.ucla.edu/hays/files/view/docs/surveys/title_page-table_of_contents_introduction.pdf [accessed 2024-04-29]
- Farivar SS, Cunningham WE, Hays RD. Correlated physical and mental health summary scores for the SF-36 and SF-12 Health Survey, V.I. Health Qual Life Outcomes. Sep 07, 2007;5:54. [FREE Full text] [CrossRef] [Medline]
- Loge JH, Kaasa S. Short form 36 (SF-36) health survey: normative data from the general Norwegian population. Scand J Soc Med. Dec 1998;26(4):250-258. [Medline]
- Hirani SP, Rixon L, Beynon M, Cartwright M, Cleanthous S, Selva A, et al. Quantifying beliefs regarding telehealth: development of the whole systems demonstrator service user technology acceptability questionnaire. J Telemed Telecare. May 2017;23(4):460-469. [CrossRef] [Medline]
- Torbjørnsen A, Småstuen MC, Jenum AK, Årsand E, Ribu L. The service user technology acceptability questionnaire: psychometric evaluation of the Norwegian version. JMIR Hum Factors. Dec 21, 2018;5(4):e10255. [FREE Full text] [CrossRef] [Medline]
- Torbjørnsen A, Småstuen MC, Jenum AK, Årsand E, Ribu L. Acceptability of an mHealth app intervention for persons with type 2 diabetes and its associations with initial self-management: randomized controlled trial. JMIR Mhealth Uhealth. May 21, 2018;6(5):e125. [FREE Full text] [CrossRef] [Medline]
- Laurberg T, Schougaard LM, Hjollund NH, Lomborg KE, Hansen TK, Jensen AL. Randomized controlled study to evaluate the impact of flexible patient-controlled visits in people with type 1 diabetes: the DiabetesFlex Trial. Diabet Med. Jan 13, 2022:e14791. [CrossRef] [Medline]
- Greenhalgh T, Rosen R, Shaw SE, Byng R, Faulkner S, Finlay T, et al. Planning and evaluating remote consultation services: a new conceptual framework incorporating complexity and practical ethics. Front Digit Health. 2021;3:726095. [FREE Full text] [CrossRef] [Medline]
- Verhoeven F, Tanja-Dijkstra K, Nijland N, Eysenbach G, van Gemert-Pijnen L. Asynchronous and synchronous teleconsultation for diabetes care: a systematic literature review. J Diabetes Sci Technol. May 01, 2010;4(3):666-684. [FREE Full text] [CrossRef] [Medline]
- Campbell R, Ju A, King MT, Rutherford C. Perceived benefits and limitations of using patient-reported outcome measures in clinical practice with individual patients: a systematic review of qualitative studies. Qual Life Res. Jun 2022;31(6):1597-1620. [CrossRef] [Medline]
- Rasmussen SP, Schougaard LM, Hjøllund NH, Christiansen DH. Patient-reported outcome measures as determinants for the utilization of health care among outpatients with epilepsy: a prognostic cohort study. J Patient Rep Outcomes. Oct 20, 2023;7(1):103. [FREE Full text] [CrossRef] [Medline]
- O'Connor S, Hanlon P, O'Donnell CA, Garcia S, Glanville J, Mair FS. Understanding factors affecting patient and public engagement and recruitment to digital health interventions: a systematic review of qualitative studies. BMC Med Inform Decis Mak. Sep 15, 2016;16(1):120. [FREE Full text] [CrossRef] [Medline]
- Allegrante JP, Wells MT, Peterson JC. Interventions to support behavioral self-management of chronic diseases. Annu Rev Public Health. Apr 01, 2019;40:127-146. [FREE Full text] [CrossRef] [Medline]
- Barlow J, Wright C, Sheasby J, Turner A, Hainsworth J. Self-management approaches for people with chronic conditions: a review. Patient Educ Couns. 2002;48(2):177-187. [CrossRef] [Medline]
- Virtanen L, Kaihlanen AM, Kainiemi E, Saukkonen P, Heponiemi T. Patterns of acceptance and use of digital health services among the persistent frequent attenders of outpatient care: a qualitatively driven multimethod analysis. Digit Health. 2023;9:20552076231178422. [FREE Full text] [CrossRef] [Medline]
- Borge CR, Larsen MH, Osborne RH, Aas E, Kolle IT, Reinertsen R, et al. Impacts of a health literacy-informed intervention in people with chronic obstructive pulmonary disease (COPD) on hospitalization, health literacy, self-management, quality of life, and health costs - a randomized controlled trial. Patient Educ Couns. Jun 2024;123:108220. [CrossRef] [Medline]
Abbreviations
| eHLQ: eHealth Literacy Questionnaire |
| HLQ: Health Literacy Questionnaire |
| HRQoL: health-related quality of life |
| ILD: interstitial lung disease |
| PRO: patient-reported outcome |
| SPIRIT: Standard Protocol Items: Recommendations for Interventional Trial |
| STROBE: Strengthening the Reporting of Observational Studies in Epidemiology |
| SUTAQ: Service User Technology Acceptability Questionnaire |
Edited by A Schwartz; submitted 08.05.24; peer-reviewed by A Gangadhara Rao, Y Yuan; comments to author 02.10.24; revised version received 24.11.24; accepted 28.02.25; published 28.04.25.
Copyright©Heidi Holmen, Are Martin Holm, Ragnhild Sørum Falk, Thomas Karsten Kilvær, Tone Marte Ljosaa, Christopher Ekholdt, Erik Fosse. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 28.04.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research (ISSN 1438-8871), is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.