User-Centered Development and Testing of the Online Patient-Reported Outcomes, Burdens, and Experiences (PROBE) Survey and the myPROBE App and Integration With the Canadian Bleeding Disorder Registry: Mixed Methods Study

Background The Patient-Reported Outcomes, Burdens, and Experiences (PROBE) questionnaire is a tool for assessing the quality of life and disease burden in people living with hemophilia. Objective The objectives of our study were (1) to assess the needs of relevant stakeholders involved in the use of PROBE, (2) to develop the software infrastructure needed to meet these needs, and (3) to test the usability of the final product. Methods We conducted a series of semistructured interviews of relevant stakeholders, including PROBE investigators, people with hemophilia, and representatives of the sponsor. Based on these, we developed an online survey and a mobile app for iOS and Android. A user group evaluated the final product using the System Usability Scale (SUS) and an open feedback framework. Results The online survey was updated, and the myPROBE app for mobile devices and a new application programming interface were developed. The app was tested and modified according to user feedback over multiple cycles. The final version of the app was released in July 2019. Seventeen users aged 23 to 67 years evaluated the final version of the app using the SUS. The median (first, third quartile) SUS score for the app was 85 (68, 88) out of 100. The newly introduced functionalities were as follows: (1) capability to longitudinally track repeated fillings of the questionnaire at different time points by the same participant (as opposed to anonymous completion); (2) linking of the questionnaire with hemophilia registries, starting with the Canadian Bleeding Disorders Registry as a proof of concept; (3) removing or adding questions as needed; and (4) sending notifications to the users (eg, reminders). A new secure database was built for securely storing personal information separately from the questionnaire data. The PROBE online survey is currently available in 96 countries and 34 languages. Conclusions The online survey was updated successfully, and the myPROBE app was developed, with a SUS score of 85 (out of 100). The app has been released in 81 countries and 34 languages. This will facilitate data collection for research and advocacy purposes, and the use of this tool in everyday clinical practice.


Detailed description of the components PROBE-R website
This is a new website that will be created to allow the longitudinal repetition of the PROBE questionnaire. For this purpose, PROBE-identifiers, the integration with bleeding disorders registries (starting with the Canadian Bleeding Disorders Registry, CBDR) and the collection of data in the frame of clinical studies, with the possibility to customize the questionnaire. Initially, the user interface will support English and French.

Front-end functionalities
The users will be able to choose from a list of languages that are already available.

Back-end capacities User Settings
Save country of origin

Save preferred language
Store information that is not likely to change over time (as specified in the main document), and re-propose them for submitting/editing in subsequent questionnaires.

Questionnaire Management
Support for additional questions (to be added on top of the core PROBE questions)

Patient Notification
Send email reminders for the longitudinal repetition of the questionnaire. Reminders will be based on the time elapsed from the last questionnaire. Also send reminders based on polling CBDR for occurrences of specific events (e.g. bleeding, invasive procedures). The latter option only applies to CBDR users, in the future might be

PROBE website
This is the already existing website. Users who choose to complete the questionnaire anonymously will be directed here from the PROBE-R website.

Application programming interface (API) Service
An API will be created, in order to allow: The mobile APP to exchange data with various systems (PROBE databases -DBs -, Registries -starting with CBDR -, Study DB in the future). Create all the methods necessary to accomplish the task The PROBE-R website to communicate with various registries, PROBE DB, and Probe Templates DB.
The API server will manage the following functionalities: -Authenticate User using email and password

Survey Manager
The survey manager is a service that exists to facilitate modifying questions in the existing PROBE questionnaire and to generate questions to be added to the PROBE-R templates (e.g. if 8 this is needed for a new research project). In the first release, the survey manager will be available only on the back-end.

Template Builder
The Template Builder is a service that will be created to group sections and questions (generated through survey manager) to create PROBE-R templates(questionnaires).
It will allow: -Creating new templates(questionnaires) using different sections/questions from PROBE and, optionally, additional questions (e.g. when needed for other projects).
-Retrieving questions and pre-requisites from other DBs.
-Generating new logical requisites for disparate survey questions (if needed).

PROBE DB (anonymous data)
This is the existing Data Base (DB). It contains the questionnaire answers and an ID which is unique for each questionnaire. This DB will remain as it is.

PROBE Extended DB (personal data)
PROBE Extended will hold the patient identifiers. This DB will be created to allow the longitudinal repetition of the questionnaire. For patients consenting to that, it will allow

PROBE template DB (templates/questionnaire data)
This DB will be created to store the different versions of the PROBE questionnaire and any other section or question created for future ancillary research projects.