A Web-Based Application for Personalized Ecological Momentary Assessment in Psychiatric Care: User-Centered Development of the PETRA Application

Background Smartphone self-monitoring of mood, symptoms, and contextual factors through ecological momentary assessment (EMA) provides insights into the daily lives of people undergoing psychiatric treatment. Therefore, EMA has the potential to improve their care. To integrate EMA into treatment, a clinical tool that helps clients and clinicians create personalized EMA diaries and interpret the gathered data is needed. Objective This study aimed to develop a web-based application for personalized EMA in specialized psychiatric care in close collaboration with all stakeholders (ie, clients, clinicians, researchers, and software developers). Methods The participants were 52 clients with mood, anxiety, and psychotic disorders and 45 clinicians (psychiatrists, psychologists, and psychiatric nurses). We engaged them in interviews, focus groups, and usability sessions to determine the requirements for an EMA web application and repeatedly obtained feedback on iteratively improved high-fidelity EMA web application prototypes. We used human-centered design principles to determine important requirements for the web application and designed high-fidelity prototypes that were continuously re-evaluated and adapted. Results The iterative development process resulted in Personalized Treatment by Real-time Assessment (PETRA), which is a scientifically grounded web application for the integration of personalized EMA in Dutch clinical care. PETRA includes a decision aid to support clients and clinicians with constructing personalized EMA diaries, an EMA diary item repository, an SMS text message–based diary delivery system, and a feedback module for visualizing the gathered EMA data. PETRA is integrated into electronic health record systems to ensure ease of use and sustainable integration in clinical care and adheres to privacy regulations. Conclusions PETRA was built to fulfill the needs of clients and clinicians for a user-friendly and personalized EMA tool embedded in routine psychiatric care. PETRA is unique in this codevelopment process, its extensive but user-friendly personalization options, its integration into electronic health record systems, its transdiagnostic focus, and its strong scientific foundation in the design of EMA diaries and feedback. The clinical effectiveness of integrating personalized diaries via PETRA into care requires further research. As such, PETRA paves the way for a systematic investigation of the utility of personalized EMA for routine mental health care.


Brief introduction to the Supplementary Materials
In the Supplementary Materials, we will fully share all material that we have developed for PETRA, in the hope this aids other researchers and software platforms in their efforts to bring EMA to clinical practice. S1 and S2 provide more background for the results of Phase 1-2. S3-S9 provide more details on the various components of the decision aid, EMA schedules, and feedback module of PETRA.
Like explained in the main manuscript, there are four possible EMA schedules in PETRA: 1. Semi-random schedule (5,7 or 10 assessments per day). In this schedule, participants are prompted in semi-random assessment blocks spread out over the day. 2. Fixed schedule (3,5,7 or 10 assessments per day). In this schedule, participants are prompted at fixed moments throughout the day. 3. Once-per-day evening schedule. In this schedule, participants are prompted only once each day, in the evening. 4. Once-per-day morning schedule. In this schedule, participants are prompted only once each day, in the morning.
The choices for each schedule depend on the personal 1) goal of the EMA, 2) symptom profile, and 3) perceived burden of each client. In the next pages, we outline the preferred EMA schedules for each goal and symptom profile. We also describe the relevant diary constructs for each goal and symptom profile, and provide the background for the burden indicator. We further share how specific needs of clients and clinicians were translated into specific features of the decision aid and feedback module. Finally, we provide our full EMA item sets for each diary schedule. 6

Phase 3: development of PETRA prototypes
Finally, we included the interview guides for the usability sessions with clients and clinicians. These sessions were more structured according to goals and associated questions, which are outlined below.
Interview guide for usability sessions with clients and clinicians. ID34: Maybe you should be able to remove certain items because that is not relevant to that person. I can imagine that. That you make a selection, select items, like 'that should be in there, that shouldn't be in there'. ID11: But also that, at a certain point… There's that item, "I feel anxious", I think. Well, I've you've been treated for that, and the anxiety is gone, then you should be able to say 'I'll remove an item from my diary'. That saves another three questions.
ID41: That you can add, every week, if you see people recover from the depression, that you can add some elements every week. So you say: 'can you go try swimming again this week, once or twice this week, and that you then add that element to the assessment. And then after a week, you can say 'right, the swimming went well, let's continue that. What else can you do this week?' And then add another element to the assessment.

Resourceefficiency
ID1: Well, I think that if you could, together with your clinician, select some items to fill out, that you can choose for the most important ones, select them from a list. What kind of topics are there to select from? Sleep, activities, mood, and that you choose for yourself, which items you find important and you want to measure. ID24: It would be helpful if we would have a list to select from, a bit like a menu. That would really lower the threshold to use it, you only have to click. […] And that you have a sort of construction where you are led through a menu of questions. It should look attractive.
ID14: Because, well, I'm constantly thinking about that psychiatrist that has to work through so many things before you arrive for the session. I just don't see that happening. So, it should be really easy to use and straightforward, so that you can easily see, hey this jumps out, this is something to talk about in the session.
ID32: It should really be ready to use. I shouldn't have to do a lot with it, this sounds really lazy. But all information should be in there, all items should be in there, there should be a fully functional app, and I should be instructed in how it works.
8 According to best practices on EMA ID2: There is also a danger in there, because, for example, it has been shown that exercise, especially outside, is really important for depressed people. And if I don't find that important at all, then I won't add that [to the assessment], whereas it is proven, over time, that if I have walked in the morning, I feel less depressed in the afternoon. And those are really important things that shouldn't be missed. ID25: Yes but it really is very strongly a case of garbage in, garbage out, so if you put rubbish in… ID24: You get rubbish out. ID25: You get rubbish out, and then you either see nothing, or you see things that are not right. So you have to carefully define what you are putting in before you put someone to work with it. And potentially it might not have any effect or even adverse effects. But I don't think anybody knows that.

Note. Brief explainer
A semi-random EMA schedule consists of several assessment blocks, within which a beep (i.e. a text message with the request to fill in the EMA assessment) will occur at a semi-random moment. A semi-random schedule is helpful when the assessments need to occur at varying time-points from day to day, but also need to be relatively evenly spread out throughout the day. For example, a semi-random schedule with 10 assessments per day will have 10 assessments blocks spread-out over 15 hours, resulting in 10 assessment blocks of 90 minutes each. Within each block, an assessment will occur at a random moment, that is within the block 09.00h-10.30h, at 10.05h. The moment that each assessment occurs thus varies per assessment block and per day. To make sure that assessments do not occur too closely together, we specified a minimal time in between assessments of 30 minutes. This means that assessments occur at least 30 minutes apart, and at most 180 minutes (in case the first assessment occurs in the beginning of the first block, and the second assessment occurs in the end of the second block).
A fixed EMA schedule may be preferred when burden for clients need to minimized. In this schedule, beeps will occur at prespecified times that are consistent across days. For example, for a fixed schedule with 10 beeps, beeps will occur exactly 90 minutes apart, that is each day at 9:00, then at 10:30, etc.

S5. Matching symptom profiles with diary constructs
Overview of symptom profiles and related diary constructs and items.

S6. Burden indicator
The PETRA decision aid contains a 'burden bar', which dynamically visualizes the number of items that are selected for a personalized EMA diary. This provides an index of the expected burden for the client, and by extension the clinician, who has to interpret multiple items in the PETRA feedbackmodule. PETRA's decision aid offers extensive possibilities for personalization; each schedule offers 18-128 items. We therefore want to aid clients and clinicians in finding a balance between gaining additional information and reducing the burden. The burden indicator is thus primarily intended to remind clients and clinician to remain conscious of participant burden and to actively discuss this when compiling the EMA diary.
Currently, research on the effect of the number of items on compliance and experienced burden offers no clear-cut guidelines [3]. Therefore, the PETRA burden indicator is based on our experience with designing EMA studies and will be updated based on user experiences and emerging research. It is likely that the number of items and associated burden differs between and even within individuals, across populations, treatment stages, EMA schedules, and in clinical versus research settings. A recent meta-analysis reported that the number of items did not significantly impact compliance and retention [4]. An empirical study in students reported that 60 EMA items per diary was perceived as more burdensome than 30 EMA items [5]. A qualitative study that compiled researcher experiences with designing EMA studies reported that on average, EMA diaries consist of 30 items [6]. Finally, an overview paper recommended a maximum duration of 2-3 minutes spent per diary [7]. In line with these studies, it is our experience with EMA studies that an average of 25 EMA items is usually perceived as doable by most clients. The additional categories were based on this number. Repeat function: sets the diary specifications (e.g., selected items, assessment frequency, duration) exactly as a previous diary, which clients and clinicians can then adapt Clinicians want to save the main conclusions on the diary results in a report, to be able to review the progress in future sessions Report function: saves current specification of the graph, as well as typed in notes by client or clinician, to the electronic health record S8. Design of the feedback module During the focus groups and user sessions, clients and clinicians highlighted several important goals for EMA feedback. These goals were grouped in three main themes, and resulted in different feedback types that were integrated in the feedback module of PETRA.

Burden indicator for semi-random and fixed schedules
As outlined in the table below, clients and clinicians mostly agreed on relevant feedback types. An interesting challenge that arose was not necessarily the difference between clients and clinicians, but within-group variability. For example, clinicians with little EMA experience expressed the need for more straightforward and standardized feedback, whereas those with more EMA experience preferred more extensive customization of the EMA diaries and EMA feedback. A similar discrepancy emerged from the sessions with clients as well. Together with the user experience designer, we therefore decided it to be helpful if the feedback module first provides an overview picture of a given topic (e.g., mood, activities). Clients and clinicians can then zoom into moments of interest to get more details. This also makes sure that clients and clinicians are not overwhelmed with all information at once, but rather have the option to go into more detail if desired. Therefore, all the graphs are interactive and customizable. Line graph depicting fluctuations in symptoms/strengths, with a context bar that offers contextual information for specific moments Word cloud that summarizes the text and provides context to thoughts and triggers

Main theme
All EMA items are originally in Dutch, but were translated to English for the purposes of this paper.

Response scale
Most items were answered on a 0-100 visual analogue scale (VAS). Other response options are outlined.

Item ordering
The EMA diary items are ordered differently in the decision aid versus in the actual diary that the participants complete on their smartphones. In the decision aid, all items are grouped in constructs (as presented in the EMA item sets below). Some items (e.g., "I feel stressed") are relevant to multiple constructs and are therefore offered multiple times in the decision aid.
After all diary items are compiled in the decision aid, PETRA will remove duplicate items and reorder the items to form the diary for the participant. In this diary, all EMA items are ordered as follows: 1. In-the-moment mood items (e.g., "Right now, I feel cheerful") 2. In-the-moment other items (e.g., "Right now, my thoughts are racing") 3. In-the-moment contextual items (e.g., "Right now, what am I doing?") 4. Retrospective items (e.g., "Since the previous measurement, I have had fun") 5. Personally formulated items 6. Qualitative item ("If you would like to note down something else about the period since the previous measurement, you can do that here."

Overlap in schedules
The semi-random and fixed EMA schedules are very similar. In the fixed schedule, a few EMA items were excluded or adapted from 'in-the-moment' to retrospective items, to be able to capture the momentary experience.
The once-per-day schedules often are an insightful addition to the semi-random and fixed EMA schedules. Therefore, the PETRA decision aid has integrated some of the items of the once-per-day-schedules in the relevant (sub)constructs of the more frequent EMA schedules. These items are marked by * in the once-per-day schedules.

Match with (sub)goals or symptom profiles
Green highlighted items are automatically selected if it is matched to a (sub)goal or symptom profile. For example, selecting the profile "Anxiety" automatically selects the subconstruct "Anxiety", which selects three out of five EMA items of this subconstruct. These highlighted items were chosen because they were considered relevant for a large group of patients.

References
For each item, we have included a (non-exhaustive) reference list of published EMA studies that have used these EMA items in diverse populations. In the fixed and semirandom schedules, 74% of the items originated from previous EMA research, and 26% were newly developed based on suggestions by clients and clinicians. For the once-perday evening schedule, 92% items stemmed from previous EMA research, and 8% were newly developed.  [11,12,19] Hoe heftig was deze gebeurtenis?
How intense was this event?
How intense was this event?
How intense was this event?
Think of the most unpleasant event or activity since the previous measurement. How unpleasant was this event?
How intense was this event?
How intense was this event?