Slit Lamp Report Generation and Question Answering: Development and Validation of a Multimodal Transformer Model with Large Language Model Integration

Ziwei Zhao   ^{1 *} , MD ;   Weiyi Zhang   ^{1 *} , MS ;   Xiaolan Chen   ¹ , MD ;   Fan Song   ¹ , MD ;   James Gunasegaram   ² , MD ;   Wenyong Huang   ³ , MD, PhD ;   Danli Shi   ^{1, 4 *} , MD, PhD ;   Mingguang He   ^{1, 4, 5 *} , MD, PhD ;   Na Liu   ^{6 *} , MD

¹ School of Optometry, The Hong Kong Polytechnic University, Hong Kong, China

² Monash University, Victoria, Australia

³ Zhongshan Ophthalmic Center, Sun Yat-sen University, Guangzhou, China

⁴ Research Centre for SHARP Vision, The Hong Kong Polytechnic University, Hong Kong, China

⁵ Centre for Eye and Vision Research (CEVR), Hong Kong, China

⁶ Guangzhou Cadre and Talent Health Management Center, Guangzhou, China

*these authors contributed equally