Using the Technology Acceptance Model to Explore User Experience, Intent to Use, and Use Behavior of a Patient Portal Among Older Adults With Multiple Chronic Conditions: Descriptive Qualitative Study

Introduction

Background

Patient portals, also referred to as tethered personal health records, are secure websites for personal health information and patient resources directly linked to a provider’s electronic health record [1,2]. Patient portals offer modern digital tools for older adults to engage in health management and with their health care system [3]. In the United States, patient portals vary greatly by provider and health care system, but often provide access to personal health information, email messaging with providers, appointment schedulers, and prescription management [1]. Patient portals are designed to help patients better manage their health with the intention to improve health outcomes, health care communication, and reduce costs [4,5]. With access to lab results and health indicators such as weight, blood pressure, and cholesterol, patients can promote early intervention when they encounter a deviation or problem or monitor improvements if following a new medication, exercise, or diet regimen [6]. Patients can also access information from various providers and necessary medical histories during emergencies to improve care transition coordination [7]. Patient portals can be convenient for medication refills, scheduling appointments, and allowing patients to communicate asynchronously with providers [6,8]. Owing to consumer demand and US government incentives for health information technology expansion, the adoption and use of patient portals is on the rise [9].

Patient portals are a promising but understudied clinical tool particularly in aging populations [10]. As older adults are more likely to have multiple chronic conditions (MCC) and higher health care utilization [11,12], they are likely to benefit from patient portal use to manage their conditions and health care services [13,14]. Although older adults are the fastest growing users of the internet [15,16], a lag in patient portal adoption remains, particularly among the oldest, less affluent, and lower educated older adults [17,18]. Although older adults show interest in patient portals [19-21], adoption barriers and low utilization have been identified [22]. Older adults pinpoint technology discomfort, privacy and security concerns, and lack of relative advantages as primary reasons for not using patient portals [22,23].

Understanding older adult opinions about portal user interface (UI) and user experience (UX) may lead to improvements in the accessibility, acceptability, and adoption of patient portals among older adults with MCC. UI typically focuses on the visual look of the design, including elements related to color, font, and images. UX targets the overall experience related to usability, usefulness, function, credibility, and satisfaction with the technology [24]. There are UI design recommendations for older users [25]; however, little is known about portal use and UX among older adults with MCC [26]. For example, portal email communication, lab results access, and electronic refill capabilities are important features for portal users [18,27], yet, it is unknown if these tools are commonly used or valued specifically by older adults with MCC. As older adults with MCC have much to gain from using portals, research is needed to better understand use behavior, perceived benefits, and strategies for increasing portal use among this specific population [26]. Therefore, the purpose of this study was to qualitatively explore perspectives from older adults with MCC regarding Kaiser Permanente Colorado’s (KPCO) established patient portal, My Health Manager. Framed by the Technology Acceptance Model (TAM), we qualitatively described the UI/UX, intent to use, and use behaviors among older Kaiser patients with MCC. Our specific research questions included the following: (1) How do participants use the portal?; (2) Why do participants use (or not use) the portal as they describe?; (3) How is the portal useful and usable?; and (4) How do these opinions and experiences influence participants’ intent to use the portal?

Technology and Acceptance Model

The TAM is an information technology framework for understanding users’ adoption and use of emerging technologies particularly in the workplace environment and has been tested in older populations [28,29]. The theory posits that a person’s intent to use (acceptance of technology) and usage behavior (actual use) of a technology is predicated by the person’s perceptions of the specific technology’s usefulness (benefit from using the technology) and ease of use. Simply, users are more likely to adopt a new technology with high-quality UX design (ie, usable, useful, desirable, and credible). The TAM also suggests that perceptions of usefulness and ease of use are mediated by external variables including individual differences, system characteristics, social influences, and facilitating conditions.

Kaiser Permanente Patient Portal: My Health Manager

My Health Manager (Table 1) provides personal health information related to patient diagnosis, prescriptions, laboratory results, and vaccination records. To improve provider-patient communication, My Health Manager offers features for patients to email providers and schedule appointments. Health management features that are designed to foster healthy eating and exercise habits incorporate personalized assessments and health self-management tools.

Methods

This is an exploratory, descriptive qualitative study based on data collected from a series of focus groups [30,31]. This method is primarily used to better understand the needs and desired outcomes from a particular group of people [32]. As such, this study employed an exploratory, descriptive approach to describe UI and UX preferences, intent to use, and use behaviors of KPCO’s My Health Manager of older patients with MCC . All procedures were approved by the KPCO Institutional Review Board.

Sample and Recruitment

We identified KPCO patients meeting the following inclusion criteria: aged ≥65 years, KPCO member for ≥1 year, presence of MCC (Charlson Comorbidity Index >2), and connected to 1 of 3 clinics in the Denver metro area with large geriatric patient populations. Non-English–speaking patients, individuals residing in skilled nursing facilities, and patients with a diagnosis of dementia were excluded. We then randomly selected potential participants stratified by age group (65 to 75 years; 76 to 85 years; and 86+ years) and portal user status (nonusers and users) to ensure participation from older participants and maximize the variability of experience with the portal. Nonusers were patients not registered for the portal, and users were those registered for the portal and logged into the portal within the last 6 months. Recruitment letters were mailed to 225 potential participants summarizing the study and providing an opt-out phone number to call if disinterested. A total of 210 potential participants (n=90 users and n=120 nonusers), who did not initially opt-out, were contacted via phone and invited to participate in focus groups. Recruitment resulted in an 18% acceptance rate (n=19 users and n=18 nonusers). Of the 37 patients that were scheduled to attend focus groups, 24 patients (n=15 users; n=9 nonusers) participated. We contacted the 13 participants who did not show up for their scheduled focus group to reschedule: 2 patients were unable to reschedule and 11 were lost to follow-up.

Focus Groups and Question Guide

We conducted 6 focus groups (3 nonuser groups and 3 user groups) lasting approximately 90 min at the KPCO facility most convenient for the participant. Around 3 to 6 patients participated in each group. Focus groups were semistructured in format, allowing for probing and extended discussion on topics of interest to the participants. Before group discussion, participants were asked to complete a demographics and technology utilization survey. The survey collected information regarding income, education, cell phone, email, internet, digital communication, and social media use. Other demographic variables including race/ethnicity, age, and days since last portal log-in were captured from KPCO’s electronic medical record during sampling procedures. Participants were then asked questions related to My Health Manager. In addition to a traditional question answer session, patients were asked to think aloud [33] as a group, whereas the interviewer navigated a mock My Health Manager portal (Textboxes 1 and 2). This method has previously been used to assess health literacy and numeracy of patient portals among patients over 65 years [34]. Focus groups were audio-recorded for accuracy in data.

Data Analysis

A theoretically driven approach [35] based on the TAM was used for analysis to capture participants’ opinions and experiences. The analysis was completed by 2 female doctoral-level researchers: JDP, a social work assistant professor with prior qualitative experience, and KG, a research assistant new to qualitative approaches. Audio files were first professionally transcribed verbatim. The unit of analysis, defined as a completion of 1 thought, was determined by the analytic team. Units of analysis ranged from a brief 3-word sentence to a paragraph of 6 sentences and were entered into Microsoft Excel to ensure that the units were consistent across coders. For initial coding, coders used a TAM-based theoretically driven code book developed a priori to code the units. The codebook included a list of TAM codes (eg, TAM constructs related to user intent, usage behavior, usefulness, and ease of use), code meaning, and criteria for using each code to capture participants’ perceptions of their intent to use and UX of My Health Manager. To prevent coders from forcing the units to fit with the TAM framework, coders used a no code option for responses that did not meet the code meaning and criteria for the a priori codes.

However, to ensure that participants’ thoughts were fully captured, coders also used a combination of open and in vivo coding (the use of participants’ own words as a code) to add inductive codes to the code book as needed. Inter-rater reliability was calculated for all transcripts (K=0.98), reflecting adequate consistency in coding across coders. Patterned coding was then employed on initial codes to identify (1) patterns in responses between users and nonusers and (b) patterns in responses between ease of use, perceived usefulness, intent to use, and UX. These patterns were then used to form themes related to each research question and develop overall findings. The analytics team met regularly through the analytic process to discuss codes and correct any disagreements in coding and thematic findings.

Results

Participants

Participants (N=24) were of a mean age of 78 years and were primarily white women (Table 2). Patient portal users had logged in to My Health Manager on an average 17.1 (SD 28.3) days before recruitment. All but one participant used a cell phone regularly, primarily a mobile phone. The majority of participants, regardless of user status, used email and looked up information on the Web. Approximately half of the participants used social media, played video games, and used video chat, whereas instant messaging was less popular.

Technology Acceptance Model Description for My Health Manager

On the basis of the TAM, Figure 1 illustrates the findings from the focus groups regarding UI and UX, intent to use, and use behavior for My Health Manager.

Use Behavior

Portal users described their use of various My Health Manager features (listed in Table 1). The email Message Center was the most popular feature used by My Health Manager users. Of the participants, 1 stated, “Yeah, I email my doctors a lot!” whereas another stated, “And I like text chatting with the sending an e-mail to my doctors [feature]; just to ask a question”. The Pharmacy Center was also commonly used to refill medications. For example, a participant said, “I use the pharmacy part every time. I hardly ever call in the pharmacy any more”. Viewing lab results in the My Medical Record page was frequently used. As 1 participant noted, “I especially like looking up the results of my test and finding out what those tests are for and if there [is] anything I need to be concerned about.”

Participants did not commonly use other My Medical Record features including viewing diagnosis list, care plans, or immunizations records. In terms of using the Appointment Center to schedule visits, 1 participant explained, “I really like using the website. I have made appointments and been shocked when I got them the next day on the computer. So sometimes I try to check there first, and then I’ll call if I feel like I need to come in and I can’t”. However, some participants unsuccessfully tried to use the Appointment Center and the majority of participants called the Kaiser phone line to schedule appointments. Only 1 participant stated they used the health guides and health management tools: “I think that’s a pretty neat feature. And then I’ve looked up things for my friends when they have questions.” None of the participants had used the newly added My Health Manager features including the e-visit or provider chat.

‎View this figure

Perceived Ease of Use

Although participants stated that the website was “pretty easy to use,” both portal user and nonuser participants (as shown during the focus group) were relatively negative about the UI and UX of My Health Manager (Textbox 3). Nonusers quickly identified UI design problems related to font size and colors while viewing the mock portal. In addition to some design issues, portal users noted challenges with using the portal or follow-up from using the portal system specifically related to registering with the system, logging in, and scheduling appointments. Several challenges were related to the back end of the system. For example, it was easy for patients to order their prescriptions on the Web, but in some cases, there were problems when participants went to the clinic to pick up the prescription. Participants who tried to use the appointment center said that they could not figure out how to schedule a visit on the portal. Those that were able to schedule an appointment on the portal experienced back-end problems at the clinic when they arrived to check-in for the visit. Despite some challenges in using the portal, many users thought My Health Manager was easy to use, and in some cases, My Health Manager was easier to use than more traditional services.

Perceived ease of use was impacted by participants’ level of computer anxiety and computer self-efficacy.

Computer Anxiety

Both portal users and nonusers expressed issues of computer anxiety, defined as apprehension or even fear of using the technology, which led to specific difficulties in using My Health Manager. As 1 user stated, “Well, I have just a couple of general thoughts about the Kaiser website. One is that what we all hate is instant upgrades, right? You hear the word ‘upgrade’ and you shudder.” Users worried that with every update, at some point they would be unable to use My Health Manager and that each new upgraded version created new use challenges. For example, during an upgrade, the log in page was modified, making it difficult for users to find the username and password text box that had previously been easy to use. The participant explained, “So recently—I don’t know how recently—but Kaiser upgraded, and all of a sudden, I couldn’t figure out how to log in. Now how stupid is that? So eventually, you know, I scanned and the ‘oh, there’s my name in the little arrow,’ but I thought, ‘couldn't they have warned us?’ That was my first thought.”

Therefore, after an upgrade, users were apprehensive about using the new version and typically found the new version to be more difficult to use than the previous version.

Computer Self-Efficacy

Users and nonusers both indicated that they were pretty confident in their ability to “figure things out” on the portal and felt good when they learned how to use a new My Health Manager feature or technology in general. However, participants discussed that although confident in their ability to use the portal, the process of learning how to use the website was challenging. A user stated:

Me and computers have problems anyway. It's like, ugh. Probably just inexperience of using the computer. I mean, I use the things that they have here, but it's not anything like going onto the website and that kind of stuff. But it was probably just not knowing where I was going or what I was doing.

Responses suggested that older participants believed they could use My Health Manager but acknowledged there would be a steep learning curve. Although learning to use the portal presented difficulties, some participants explained they simply did not have the ability to use specific features. One nonuser, who was interested in learning more about the provider email function, stated they could not use the new provider chat function because they could not type or respond quickly enough to instant messages with a provider. They explained, “You have to type in your chat...It’s fast...So email would be better.”

Perceived Usefulness

Participants described benefits and drawbacks regarding My Health Manager (Textbox 4).

Benefits

Portal users expressed a clear benefit in using My Health Manager, in general and for very specific features. Generally speaking, users expressed that My Health Manager was useful in communicating with their provider, accessing health information, saving time and money, and addressing health concerns without a clinic visit.

Drawbacks

On the contrary, nonusers stated they preferred to accomplish health-related tasks using their current process and indicated the portal would be more helpful for particular people: those living at a distance and those with serious illness. Nonusers preferred to use the telephone or clinic pharmacy for prescriptions, seeing their provider in person when asking nonemergent questions and calling to make clinical appointments. However, it is important to note that participants described alternative benefits for their current methods. Although portal users thought using My Health Manager for prescription refills was useful in getting medications, nonusers liked going to the pharmacy in person because it got them out of the house and kept them active. Nonusers acknowledged that My Health Manager would be useful for patients far away from their providers and sicker patients. Nonusers understood why people would want to use the system. It simply did not seem useful to them personally.

Intent to Use

Participants’ intention to continue using or start using My Health Manager was influenced by their perceived ease of use and perceived usefulness.

Previous Negative User Experience

Once a participant had a negative UI with My Health Manager or a specific feature, they had little interest in trying again. For example, a nonuser who tried to register for My Health Manager was so frustrated with their registration experience that he/she did not want to try again, and gave the following explanation:

I tried to get on [My Health Manager] several years ago and that was when they were sending the password by mail. I lost the password, and I forgot that you had to have (a password). I forgot all that. I again tried to get on it and didn’t have a password so I thought, “Well, I’ll just start over again.” It wouldn’t let me, so I said, “Well, the heck with it then.

Also, as stated above, users expressed many challenges in using the Appointment Center; therefore, participants showed little intent to use the Appointment Center until the glitches were fixed.

Lack of Interest in New Functions

Nonusers and users alike were not interested in using the recently added features of e-visits and chat functions. Few participants saw the value in using these features, articulating comments such as, “But I wouldn’t use it because I don’t see any need to, personally. I’m not saying other people wouldn’t.” Nonusers wanted to continue seeing their providers in person or talking on the phone, and users wanted to continue using the portal as is. A participant explained about possibly using the e-visit feature:

Usually, if I want to see the doctor, I want to see the doctor. And I know what it is why I’m going, and what it is I want to talk to them about. If it’s just real simple, I can just e-mail him or call him. If I want to see my doctor, I undoubtedly have to make an appointment to see him. And if I want to see my doctor, I want to go see my doctor.

Lack of Awareness of Functions Available

Intent to use was also influenced by participants’ awareness of My Health Manager features and access to help using the website. Most participants did not know about the new features, and nonusers did not know about the basic features available via My Health Manager in general. As 1 nonuser participant stated, “You can see how I read on the computer, because I’d never seen that—make a—schedule an appointment. That wouldn’t occur to me”; participants did not know what features were available or how to use them.

Discussion

Principal Findings

This study supports the growing literature suggesting many older patients, including those with MCC, are interested in using and are already using patient portals to help manage their health [22]. This is also the first study to use the TAM to qualitatively explore the connections between perceived usefulness, ease of use, and intent to use for a patient portal among older patients with MCC.

The TAM framework and supporting evidence [17,18] indicates several external variables influencing perceived ease of use with patient portals. Our study participants identified only 2 external variables: computer self-efficacy and anxiety. Specific patient portal user trainings offered in-person and/or on the Web may help older adults learn how to use the portal and when to use specific features [36]. Caregivers and family members are also helpful in reducing technology-specific anxiety [22,37], but more research is needed to inform portal design for shared access with caregivers [38].

Email, pharmacy, and medical lab result sections were popular and perceived as both useful and easy to use. This use behavior is consistent with other patient portal research in older populations [4,8]. These features are simple and quick while improving perceived patient-provider communication, satisfaction with access to health information, and fast medication management. Nonusers interested in the portal may be directed to these most popular, usable features. Research shows that once older adults are engaged in a technology, they tend to be high utilizers [23]. Therefore, promoting adoption of popular, easy-to-use features may foster patient satisfaction and further use of additional portal features. For example, promoting the email feature initially to encourage a patient to then try the portal pharmacy system.

Other features, particularly the Appointment Center, are difficult to use and do not offer perceived benefit to patients in this study. It is easier for patients to simply call to schedule appointments. There are also UI design issues related to small fonts and poor coloring, and negative UX influenced participants’ intent to use the portal. These results align with the TAM and previous work suggesting that technology acceptance is determined by the perceived value and degree of burden. Older adults are unlikely to adopt burdensome technologies. Therefore, health systems should obtain ongoing UI and UX feedback from older adults with MCC when developing new tools and updates. The Department of Veteran Affairs implemented an ongoing feedback strategy that fostered adoption of their patient portal [39].

In terms of perceived usefulness, participants in this study suggested that patients far away from their providers would particularly benefit from the patient portal. However, older adults in rural areas are less likely to use patient portals [40], and internet use is lower among people in rural settings, especially among people with MCC [41]. Low internet and health technology use in rural communities is often attributed to limited access and awareness [42]. Recent improvements to broadband access [43] in rural communities may lead to increased portal adoption. However, more research is needed to determine best strategies for promoting portal engagement among older adults with MCC living in rural settings.

Participants indicated the portal would be helpful for sicker patients. Although we did not follow up to acquire a better definition of sicker, older patients with serious illness, owing to complex care needs, may benefit from portal use. There is some evidence substantiating increased portal adoption among older adults with worse health status [44]. A few studies indicate that people with cancer have positive perceptions of patient portals [45,46]. Although older adults with serious illness may be a target population for portal adoption, little is known about patient portal utility for patients with advanced or serious illness.

Preference for current methods is also a drawback to perceived usefulness and barrier to patients’ intent to use. Participants value going to the clinic or pharmacy for physical activity and social engagement. With these values in mind, portal designers should consider adding functions that encourage older patients to get out of the house and connect personally with their providers. As portals advance, it is important to also respect the patients’ need for a face-to-face connection with their providers. However, providers may consider using some face-to-face and phone-based time to encourage portal use [21]. Face-to-face and phone-based encouragement (eg, “Did you know you can schedule your next appointment in the convenience of your home online? Just go to My Health Manager”) from providers and staff may increase patient awareness of beneficial features. In this study, health management tools and newer features were not used primarily owing to a lack of awareness.

In this sample, patient portals are not preferred by everyone, and other older adults with MCC may feel similarly. Usage varies greatly: some patients will never use the portal, other current users will continue to use only a few features, whereas another group will use every available option. Explicit nonusers appear to prefer human and face-to-face contact, which has previously been reported from a diverse sample of Kaiser patients [47]. Regardless of preference, technology-based health care interactions are increasing, and portal use may be expected. Addressing UI and UX challenges and promoting perceived benefits (improving commination, saving time, and access to personal information) may improve the intent to use patient portals among older adults with MCC.

Limitations

Although our study is an in-depth analysis of perceptions of older adults with MCC of a specific patient portal, the use behaviors and experience may differ across portal systems. While employing a descriptive qualitative approach for understanding portal UI and UX, intent to use, and use behavior of My Health Manager, we used only focus groups for data collection. This work would have benefited from inclusion of other data sources such as observation. We were unable to recruit as many portal nonusers resulting in limitations of our nonuser feedback. Although we recruited participants representing a wide age range, we did not maintain equal participation from each age group. The sample was also relatively well educated, middle income, and technology users, lacking specific input from underprivileged populations with less access to technology resources.

Conclusions

Older adults are interested in using patient portals and are already taking advantage of the features available to them. We have the opportunity to better engage older adults to use portals but need to pay close attention to key considerations promoting usefulness and ease of use. We recommend implementing portal user trainings, family and caregiver support, ongoing user feedback, and provider encouragement to improve intent to use and adoption among older adults with MCC.