Interactive Visual Displays for Interpreting the Results of Clinical Trials: Formative Evaluation With Case Vignettes

Background At the point of care, evidence from randomized controlled trials (RCTs) is underutilized in helping clinicians meet their information needs. Objective To design interactive visual displays to help clinicians interpret and compare the results of relevant RCTs for the management of a specific patient, and to conduct a formative evaluation with physicians comparing interactive visual versus narrative displays. Methods We followed a user-centered and iterative design process succeeded by development of information display prototypes as a Web-based application. We then used a within-subjects design with 20 participants (8 attendings and 12 residents) to evaluate the usability and problem-solving impact of the information displays. We compared subjects’ perceptions of the interactive visual displays versus narrative abstracts. Results The resulting interactive visual displays present RCT results side-by-side according to the Population, Intervention, Comparison, and Outcome (PICO) framework. Study participants completed 19 usability tasks in 3 to 11 seconds with a success rate of 78% to 100%. Participants favored the interactive visual displays over narrative abstracts according to perceived efficiency, effectiveness, effort, user experience and preference (all P values <.001). Conclusions When interpreting and applying RCT findings to case vignettes, physicians preferred interactive graphical and PICO-framework-based information displays that enable direct comparison of the results from multiple RCTs compared to the traditional narrative and study-centered format. Future studies should investigate the use of interactive visual displays to support clinical decision making in care settings and their effect on clinician and patient outcomes.

. Open-ended comments and category Comments Category The legends and keys for the bar graphs are a little hard to see, it seems they require dragover with the cursor, it would be nice if they were clearly displayed so I knew what the bars meant intuitively.

Request for clearer display
Provide serial numbers in the first column. (I) will be happy to be involved in further refining this tool later as well.
Request for more information Request for more features It was very effective as a clinical tool. I cannot think of any improvements.

Great tool
It was great, I can imagine using this tool all the time in my practice!

Great tool
It is awesome! Maybe a way to display more RCTs at once with the comparison tools. 1) Great tool; 2) Allow more than 4 studies for comparison Include more than 4 studies at a time Allow more than 4 studies for comparison I would have liked to see error bars on some of the graphs. In these vignettes, the 10 studies were provided, but how will you propose coming up with those 10 studies in real-life clinical practice?
Request for more information I think that it would be nice to make graphical comparisons of more than 4 studies at once if need be Allow more than 4 studies for comparison I am very familiar with abstract format, so I think that is why I liked PubMed. I question whether the right trials were included in interactive visual display. I have control over this is PubMed search. A quick link to the right clinical guidelines may be more helpful than the actual RCTs because I know the process by which that data is vetted for accuracy and I know I would practicing in accordance with current standards.

Prefer narrative abstracts in
PubMed great tool, less incentive to read methods and figure out the validity of a study as the results are prominently displayed but could easily do. if simple validity criteria are available that would be really something. 1) Great tool; 2) Request for more information Continue to maintain easy use of the tool.
Great tool Allow the option to select more than 4 trials which will give a better view overall. Put select all option in the right-hand corner to quickly deselect all selected options 1) Allow more than 4 studies for comparison; 2) Request for more features

Acute Coronary Syndrome
A 56-year-old Chinese patient, Mr. Lee presents to the Emergency room with symptoms of acute coronary syndrome. He is seen by a resident who notices an elevated lipid profile. He says that he is on some lipid lowering therapy but more details are not available. The resident wants to find out the best lipid lowering therapy that he could start Mr. Lee on, in this situation.
Which lipid lowering drug should the resident choose?
Clinical Trial Studies PMID: 26486166 and 25879728.

Rheumatoid Arthritis
Mary

Diabetes Mellitus
Agnes is a 51-year-old female with hypertension who received a diagnosis of type 2 diabetes a decade ago. She has been worried about her diabetes since then because she has not been able to gain complete control over it. Her HbA1c was 7.0% for 1 year but gradually increased to 9.0%. For the past 2 years, she has been taking metformin 2000 mg daily. She is maintaining her weight at 165 pounds (75 kg), but she is not able to lose weight. Agnes goes to the gym and walks on a treadmill three times a week. She tells you that she has made as many lifestyle changes as she can. Agnes' hypertension is well controlled with an ACE inhibitor; she also takes a statin. Her most recent tests showed LDL cholesterol of 85 mg / dL and HDL of 62 mg / dL.
Agnes hates needles and won't use insulin. Her sister, who also has diabetes, was receiving glipizide but had episodes of hypoglycemia while taking that drug, including one episode that resulted in an auto accident. Agnes is also worried about weight gain associated with that drug. Her sister recently switched from glipizide to saxagliptin and has had no further episodes of hypoglycemia. Agnes has also heard about a new type of drug that works by eliminating excess glucose through the urine. She wants to know about the safety of the newer drugs. You explain to her that the drugs she is asking about are in different classes -dipeptidyl peptidase 4 (DPP-4) inhibitors ("gliptins") and sodium glucose cotransporter 2 (SGLT2) inhibitors ("gliflozins").