Preparing Facilitators From Community-Based Organizations for Evidence-Based Intervention Training in Second Life

Introduction

Background

Multi-user virtual environments (MUVEs), such as Second Life (SL), are a promising alternative to face-to-face training for health care program delivery [1]. While rigorously tested evidence-based interventions (EBIs) have the potential to influence health outcomes, challenges related to training and capacity building access (eg, funding, high travel costs, limited to no technical assistance offered, and limited guidance on appropriate adaptations) [2,3] can impede implementation fidelity and the effectiveness of the EBI. Due to these challenges, more accessible training modalities on how to deliver the intervention accurately are needed to better disseminate EBIs and to ensure their effectiveness. Virtual environments such as SL have not been extensively used for EBI training of individuals who work in community-based organizations and deliver training to adolescents (ie, facilitators). This is important as facilitators deliver the curriculum to the adolescents and facilitate group discussions and other activities that are part of the curriculum. As a cost-effective and highly interactive MUVE, SL has potential to serve as a delivery platform for comprehensive EBI facilitator training.

A MUVE is a computer-based, simulated virtual environment allowing users to inhabit an online virtual world and interact with others via self-representations known as avatars in synchronous sessions [1]. The advantage of MUVEs like SL is that they increase access to capacity building opportunities [2] and are a promising alternative to face-to-face training due to substantial features mirroring human interactions. These features include advanced and realistic voice chat, speech gestures, and the ability to manipulate voices in a multidimensional space just as one would experience in real life [4]. This study describes the process of preparing individuals from community-based organizations to participate in a SL ¡Cuídate! (Take care of yourself) Training of Facilitators [5].

Second Life Training

SL has been previously used for small-scale health-related programs focusing on education and awareness, support, training, marketing, and promotion of health services [1,6-11]. For example, SL was used to implement a 1-hour seminar to enhance clinician knowledge of insulin therapy for type 2 diabetics for primary care physicians (n=14) [9]. Results of the training indicated a positive impact on self-efficacy to perform insulin therapy, gains in clinical competence around the use of insulin therapy, and positive feedback on the sense of presence attributed to the use of avatars. There was high variability in the SL user learning curve depending on past experience with video games, but an average of 78 minutes was spent with each participant to gain proficiency in SL. Similarly, Mitchell et al pilot-tested motivational interviewing training in SL with primary care physicians (n=13) focusing on colorectal cancer screening [10]. A self-directed approach to learning SL skills was used to reduce the amount of time necessary to prepare participants in SL. The actual amount of time participants spent on preparation was not explicitly reported. Finally, Tschannen et al used SL for diabetes self-management training for nurses. Participants reported that they were highly satisfied with the virtual simulation experience. Further, participants’ ability to apply knowledge gained was comparable to those who completed a face-to-face training alternative [11]. While these studies demonstrate the utility of SL for training, none of these studies reported quantitative data on SL proficiency outcomes, or self-efficacy of SL skills. Further, training participants were limited to highly trained clinicians.

Technical Challenges

Virtual training presents a number of technical challenges, including computer system capacity and security issues [10]. For example, training participation requires minimum system specifications, installation of a separate computer program, and the development of specific computer skills to facilitate the interaction required in SL to engage fully in the training. Specifically, SL requires a high level of processing power, graphical memory, and a need for a large bandwidth with high-speed Internet access [12]. Individuals may encounter network policies that do not allow access to public virtual worlds like SL or allow downloads of software or a continuous Internet connection [9]. Despite the technological complexities, SL has proven to be an accessible medium for effective training and education.

Purpose

The purpose of this study was to examine the feasibility of using SL to train facilitators to use an EBI, ¡Cuídate! (Take Care of Yourself), one of the few sexual risk reduction interventions developed and tested with Latino youth [13]. The participants received training in SL to prepare them to deliver the ¡Cuídate! curriculum to Latino youths in their community. This paper examines the management of technical issues and participant self-evaluation of SL skills.

Methods

Overview

This is a descriptive study. The study protocol was reviewed by the institutional review board at the University of Michigan and was deemed exempt and not regulated.

A virtual training center (Figure 1) was developed in SL to deliver the ¡Cuídate! training. This training center was built on university-owned space in SL (Wolverine Region). The training center consisted of a conference room with a table and chairs and a training room with movable chairs, which were scripted to provide avatars with gestures such as hand-raising. The training was set up with Web prims to display interactive posters and Google Docs.

Preparation to use SL consisted of four phases: (1) recruitment and computer capacity screening, (2) enrollment, (3) orientation to the SL program, and (4) technical support throughout the synchronous training sessions.

‎View this figure

Phase One: Recruitment and Computer Capacity Screening

Community-based organizations throughout the United States were sent information about the SL Cuídate Training of Facilitators opportunity via social media (eg, Facebook, Twitter), distribution lists, and newsletters. To ensure agency staff could feasibly participate in the SL training, an online portal and a screening process were established. Community-based organizations submitted contact information and answered screening questions to determine if their computer system capacity met SL’s system requirements. The minimum system requirements for SL include at least a Windows XP or Mac OS X 10.6 operating system, a graphics card model NVIDIA GeForce 6600 or better, an ATI Radeon 9500 or better, or an Intel 945 chipset for three-dimensional functionality, and a high-speed Internet connection [12]. To identify possible SL installation barriers, access to the Internet (ie, wired or wireless connection) and availability of information technology (IT) personnel at their facility to assist in mitigating technical difficulties was assessed.

If registrants had questionable capacity (eg, older operating systems or a low-power graphics card), research staff guided each registrant through an online scanner available from System Requirements Lab, Inc. [14]. If minimum requirements were met, instructions for downloading and installing the program were provided. In rare cases, a member of the study team walked through the installation steps over the phone with them or their IT personnel.

Phase Two: Enrollment

Once an agency met the minimum computer system capacity, they were presented with a Memorandum of Understanding (MOU) that included information on the purpose of the study, responsibilities of participants, and expectations and accountability of the research team. Participants received an enrollment phone call to review the MOU and were provided with an outline of the project activities. Once the MOU was signed, a ¡Cuídate! Implementation Kit [15] and a universal serial bus (USB) microphone headset were mailed to participants. The USB microphone headsets were sent to standardize equipment for easier troubleshooting of SL voice chat difficulties during training. For their participation, community-based organizations were provided a financial incentive of US $500.00 to deliver ¡Cuídate! in their community post-participation in the Training of Facilitators.

Communication occurred with 120 different community agencies and entities throughout the United States via recruitment phone calls. Of those, personnel from 59 agencies underwent the screening process previously described. Of the agencies that met computer system capacity, 55% (24/44) committed to the training and completed the tasks required to participate. Details of the recruitment and screening process are reported in Figure 2.

‎View this figure

Phase Three: Orientation to Second Life

Participants were oriented to SL in three steps: (1) self-directed online orientation manual, (2) a one-on-one SL review session with research staff, and (3) a group skill review on SL training day one. In the first step, we developed an online orientation manual that provided instructions on how to install SL, create an avatar, perform basic navigation functions, teleport to different parts of the environment, manipulate camera angles, and use text and voice communication features. The online manual consisted of text and screenshots of steps and guided video demonstrations embedded within the manual.

In the next step, one-on-one meetings in SL were scheduled to review and practice basic skills and to introduce participants to more complex skills. These additional skills focused on interactions with three-dimensional objects in the program including seats, working on collaborative Google documents [16], changing their display, and participating in small group voice chats. The SL ¡Cuídate! training was developed to limit participant burden with learning SL to ensure the focus of the training was on learning the ¡Cuídate! Facilitator’s Curriculum.

The third step was incorporated into the first day of the SL facilitator training. This orientation session served as an introduction to the ¡Cuídate! training and provided an opportunity to practice features of the SL environment relevant to training. Best practices in SL were shared to ensure a smooth training experience. The practices included silencing microphones when they were not speaking to minimize distracting noises, ensuring their avatars remained active by participating in discussions, raising their avatar’s hand when they had a question or comment, and how best to view the interactive documents and posters during the training.

Phase Four: Technical Support During Training

In the training center, a designated staff member with high SL proficiency was available to provide technical support. Participants were asked to sign in 15 minutes prior to scheduled SL ¡Cuídate! training and to send a message using text chat to the technical staff if they were having trouble during the session. The staff member would either work in the background with the participant or have them teleport (a quick navigation method in SL) to another location to troubleshoot issues, thus minimizing distractions for other participants.

Measures

Overview

To evaluate the process of preparing individuals to participate in SL training, several different descriptive methods were used. Data were gathered from the participant screening survey, technical support sessions, and a post-orientation survey. The surveys were administered using Qualtrics. Data were then categorized in three different areas for analysis: computer system capacity and experience, SL self-efficacy, and participant preparation time.

Computer System Capacity and Experience

A measure of overall computer system capacity consisted of two components: technical specifications and participant experience with online environments. During enrollment, participants were asked about technical specifications related to their operating system and Internet connectivity. Participants were also asked about previous online learning experiences (Yes/No) and if they had used SL prior to this training (Yes/No). They were also asked to report the frequency of engagement in online and gaming software on a 3-point scale (1=rarely, 2=often, 3=very often).

Second Life Self-Efficacy

Self-efficacy to use SL was assessed with nine questions adapted from a scale developed by Gist, Schwoerer, and Rosen, which was developed to measure computer and software self-efficacy [17]. The first three questions (eg, User Capability to Navigate SL) measured the participant’s ability to navigate SL after using either the written instructions or the video demonstrations (both part of the orientation manual). The next six questions (eg, Self-Efficacy of SL Skills) in the scale measured the self-efficacy of participants to (1) download SL software, (2) create an avatar, (3) walk with their avatar, (4) teleport to different locations, (5) change the view in SL, and (6) interact with items in SL. The questions were on an 11-point scale, from 0 (completely unable to do the task) to 10 (totally capable of doing the task). Reliability coefficient alpha=.83 for the User Capability to Navigate SL (3 questions) and .87 for the Self-Efficacy of SL Skills (6 questions).

Participant Preparation Time

Two questions assessed participants’ estimates of the time spent reviewing the SL orientation manual. Additionally, the research team recorded actual time spent supporting participants in the use of SL or addressing computer issues. Technical difficulties experienced by participants, the associated cause, and the mitigation strategy implemented were also recorded throughout the training and preparation.

Results

Overview

A total of 35 participants from 24 agencies participated in 1 of 5 cohorts of SL ¡Cuídate! training. The majority of participants were female (69%, 24/35) and ranged from 20-59 years of age. Most participants had received at least a Bachelor’s degree (80%, 28/35). Slightly more than half of the participating community-based organizations had fewer than 50 employees (58.3%, 14/24), while 10 community-based organizations reported having more than 50 employees (41.7%, 10/24).

Computer System Capacity and Experience

Most participants had Windows Vista or better (74%, 26/35), while only a few reported using Windows XP (14%, 5/35), or Mac OSX 10+ (9%, 3/35). Regarding Internet connections, nearly equal numbers of individuals (40%, 14/35) reported having a wired connection or only a wireless connection (37%, 13/35), while a small percentage of participants (23%, 8/35) had access to both wired and wireless connections.

A majority of participants (77%, 27/35) reported having online learning experience as well as experience with computer and video games, but very few participants (9%, 3/35) had ever used SL. Of those reporting having played computer or video games in the past 12 months (51%, 18/35), 2 participants (11.1%) played very often, 3 (16.7%) often, and 13 (72%) had rarely played.

Second Life Self-Efficacy

Most individuals rated their confidence levels high (no mean scores were below 7) on the SL self-efficacy scale. Participants reported high capability to navigate SL after reviewing written instructions and additional video clips (mean 7.97, SD 2.19). Participants had high capability scores for skills such as downloading the SL program and creating an avatar (mean 8.29, SD 2.19, and mean 8.03, SD 2.19 respectively), but slightly lower scores for changing the view on their computer screen and interacting with items in SL (mean 7.49, SD 2.89, and mean 7.49, SD 2.92 respectively). See Table 1.

Participant Preparation Time

The majority of participants (91%, 32/35) reviewed both the written instructions as well as the video demonstrations in the SL orientation manual. The amount of time participants spent using the orientation manual ranged from less than 1 hour (43%, 15/35), between 1 and 2 hours (17%, 6/35) or more than 2 hours (40%, 14/35). It is unknown how many more hours were spent using the orientation manual as the scale ended at 2 plus hours.

The one-on-one session in SL required an average of 28.8 (SD 11.8) minutes per participant with a range of 15-60 minutes. Troubleshooting technical issues required an average of 28.4 (SD 16.4) minutes per participant for 74% (26/35) of the participants. The shortest troubleshooting time (2 minutes) was used to correct simple settings issues, and the longest (75 minutes) was used to assist participants with installation difficulties.

Some participants had issues with SL installation due to either agency policies (n=3) or inadequate skill (n=3). Some agencies had policies prohibiting access to public virtual worlds like SL. The most prominent technical difficulty after installation was headset malfunction and incompatibility with users’ computers. The majority of these issues were corrected by adjusting settings on the operating system or on the SL program itself. For others, there were SL malfunctions such as lags in performance, periodic interruptions to audio or an ejection from the program. Internet connectivity was thought to be the root cause of these performance issues, particularly those with wireless Internet connections. The overwhelming majority of participants who continued to receive troubleshooting during the training reported wireless Internet connections. Technical difficulties and support are highlighted in Table 2.

Discussion

Principal Findings

The purpose of this study was to describe the process and related challenges of preparing individuals to participate in an EBI facilitator training program adapted for SL. The response to initial recruitment for participants was positive despite training being offered in a very non-traditional format (68/120 agencies indicated interest). Throughout the screening process, a variety of agencies were able to meet technical requirements and complete the training, half of which were small community-based agencies (<50 employees). Reasons that agencies did not participate, despite initial interest, were primarily related to inadequate computer system capacity and/or the inability to implement the training within 6 months post training. The variation in computer system capacity and Internet connections of the agencies that participated allowed for the opportunity to test the feasibility of using SL in areas that do not have available technical support staff or high-speed Internet capabilities.

In terms of SL skills orientation, the majority of participants reported high confidence levels for basic skills in SL. This supports the feasibility of using a self-directed orientation manual since participants were able to reasonably orient themselves to the SL environment without additional support. Further, the use of a self-directed orientation manual resulted in limited staff time (less than one hour) for SL orientation and limited time necessary for additional troubleshooting. The time spent on orientation is less than previous reported results of Wiecha et al who spent 78 minutes coaching each participant [9]. Findings from this study provide the overall process that can be implemented to facilitate other similar trainings in SL.

Lessons Learned

In this study, participants did identify technical challenges that impacted facilitator training. Major difficulties encountered included hardware and software compatibility issues and Internet issues (bandwidth) and not with individual ability to use the SL software (ratings for self-efficacy were high and very few troubleshooting encounters were reported requiring assistance with SL program skills). Although system requirements were identified pre-training, further testing is needed to determine specifically what system requirements should be mandated in order to prevent technical hurdles. For some, the use of a laptop interfered with a seamless experience because some laptops are not designed with the capacity (ie, adequate graphic cards, reliance on wireless) to run even less burdensome three-dimensional environments such as SL. Of these connection-related difficulties, audio input and output malfunctions were most prevalent even when using a standard audio headset and after pre-training audio testing.

One unanticipated cause of these issues was the load on an individual’s network and computer system when multiple users were simultaneously logged on to the virtual environment. Smaller training cohorts (fewer than 7 participants) had fewer connection-related issues in the live training. Therefore, providing installation instructions upon initial registration may allow registrants to test their bandwidth by having them enter more populated areas of SL to see if their computers can handle a large number of users. This early bandwidth testing should reduce the number of technical difficulties that may arise in the training program from a lack of bandwidth capacity. One additional approach that may be warranted is to limit the amount of participants per training session to minimize the technical challenges that may occur.

Conclusions

Technical support was a vital component of this study. Delivering training in SL requires technical expertise to remotely troubleshoot with participants. Advanced knowledge of both computer hardware and software components is essential for the technical staff who will provide the support before and during training. By having support staff available throughout the training, major interruptions were avoided and participant burden was minimized allowing for the successful retention of all 35 participants across 3 days of training.

In summary, preparing participants to use SL for the ¡Cuídate! Training of Facilitators was feasible, and participants were confident in their ability to use the program for training after completing orientation. In addition, we developed processes to decrease technical difficulties prior to and during training. The potential advantages of conducting SL trainers include the ability to access EBI training without the high cost of travel. This is especially relevant for agencies and communities with limited resources. Decreasing access barriers through the use of technology can facilitate the dissemination and use of evidence-based interventions, resulting in improved health outcomes.