This paper is in the following e-collection/theme issue:

Medicine 2.0: Social Media, Open, Participatory, Collaborative Medicine (1641) Crowdsourcing and Mechanical Turks (98)

Published on in Vol 15, No 4 (2013): April

Web 2.0-Based Crowdsourcing for High-Quality Gold Standard Development in Clinical Natural Language Processing

Web 2.0-Based Crowdsourcing for High-Quality Gold Standard Development in Clinical Natural Language Processing

Web 2.0-Based Crowdsourcing for High-Quality Gold Standard Development in Clinical Natural Language Processing

Authors of this article:

Haijun Zhai1 ;   Todd Lingren1 ;   Louise Deleger1 ;   Qi Li1 ;   Megan Kaiser1 ;   Laura Stoutenborough1 ;   Imre Solti1
Article Authors Cited by (47) Tweetations (3) Metrics

Journals

  1. Şahin G, Adalı E. Annotation of semantic roles for the Turkish Proposition Bank. Language Resources and Evaluation 2018;52(3):673 View
  2. Abrams M, Milisavljević M, Šoškić A. Childhood abuse: Differential gender effects on mental health and sexuality. Sexologies 2019;28(4):e89 View
  3. Thawrani V, Londhe N, Singh R. Crowdsourcing of Medical Data. IETE Technical Review 2014;31(3):249 View
  4. Kuang J, Argo L, Stoddard G, Bray B, Zeng-Treitler Q. Assessing Pictograph Recognition: A Comparison of Crowdsourcing and Traditional Survey Approaches. Journal of Medical Internet Research 2015;17(12):e281 View
  5. Wang S, Dang D. Incentive mechanism for the listing item task in crowdsourcing. Information Sciences 2020;512:80 View
  6. Li T, Bravo À, Furlong L, Good B, Su A. A crowdsourcing workflow for extracting chemical-induced disease relations from free text. Database 2016;2016:baw051 View
  7. Zhang A, Chen J, Chai W, Xu J, Hong L, CHI E. Evaluation and Refinement of Clustered Search Results with the Crowd. ACM Transactions on Interactive Intelligent Systems 2018;8(2):1 View
  8. Abrams M, Milisavljević M, Šoškić A. Maltraitance infantile : effets différentiels liés au sexe sur la santé mentale et la sexualité. Sexologies 2019;28(4):191 View
  9. Saunders D, Bex P, Woods R. Crowdsourcing a Normative Natural Language Dataset: A Comparison of Amazon Mechanical Turk and In-Lab Data Collection. Journal of Medical Internet Research 2013;15(5):e100 View
  10. Lee Y, Arida J, Donovan H. The application of crowdsourcing approaches to cancer research: a systematic review. Cancer Medicine 2017;6(11):2595 View
  11. Khare R, Good B, Leaman R, Su A, Lu Z. Crowdsourcing in biomedicine: challenges and opportunities. Briefings in Bioinformatics 2016;17(1):23 View
  12. Lalor J, Woolf B, Yu H. Improving Electronic Health Record Note Comprehension With NoteAid: Randomized Trial of Electronic Health Record Note Comprehension Interventions With Crowdsourced Workers. Journal of Medical Internet Research 2019;21(1):e10793 View
  13. Guo L, Mays K, Lai S, Jalal M, Ishwar P, Betke M. Accurate, Fast, But Not Always Cheap: Evaluating “Crowdcoding” as an Alternative Approach to Analyze Social Media Data. Journalism & Mass Communication Quarterly 2020;97(3):811 View
  14. Assis Neto F, Santos C. Understanding crowdsourcing projects: A systematic review of tendencies, workflow, and quality management. Information Processing & Management 2018;54(4):490 View
  15. Bouadjenek M, Zobel J, Verspoor K. Automated assessment of biological database assertions using the scientific literature. BMC Bioinformatics 2019;20(1) View
  16. Burger J, Doughty E, Khare R, Wei C, Mishra R, Aberdeen J, Tresner-Kirsch D, Wellner B, Kann M, Lu Z, Hirschman L. Hybrid curation of gene–mutation relations combining automated extraction and crowdsourcing. Database 2014;2014 View
  17. Talikka M, Bukharov N, Hayes W, Hofmann-Apitius M, Alexopoulos L, Peitsch M, Hoeng J. Novel approaches to develop community-built biological network models for potential drug discovery. Expert Opinion on Drug Discovery 2017:1 View
  18. Cui L, Carter R, Zhang G. Evaluation of a Novel Conjunctive Exploratory Navigation Interface for Consumer Health Information: A Crowdsourced Comparative Study. Journal of Medical Internet Research 2014;16(2):e45 View
  19. D’Orazio V, Kenwick M, Lane M, Palmer G, Reitter D, Ebrahimi M. Crowdsourcing the Measurement of Interstate Conflict. PLOS ONE 2016;11(6):e0156527 View
  20. Khare R, Burger J, Aberdeen J, Tresner-Kirsch D, Corrales T, Hirchman L, Lu Z. Scaling drug indication curation through crowdsourcing. Database 2015;2015 View
  21. Wazny K. Applications of crowdsourcing in health: an overview. Journal of Global Health 2018;8(1) View
  22. Dasgupta N, Freifeld C, Brownstein J, Menone C, Surratt H, Poppish L, Green J, Lavonas E, Dart R. Crowdsourcing Black Market Prices For Prescription Opioids. Journal of Medical Internet Research 2013;15(8):e178 View
  23. Cocos A, Qian T, Callison-Burch C, Masino A. Crowd control: Effectively utilizing unscreened crowd workers for biomedical data annotation. Journal of Biomedical Informatics 2017;69:86 View
  24. Lalor J, Wu H, Chen L, Mazor K, Yu H. ComprehENotes, an Instrument to Assess Patient Reading Comprehension of Electronic Health Record Notes: Development and Validation. Journal of Medical Internet Research 2018;20(4):e139 View
  25. Vélez P, Rey Piedrahita A. Control de calidad en sistemas crowdsourcing: un mapeo sistemático. Scientia et technica 2017;22(1):73 View
  26. Hirschman L, Fort K, Boué S, Kyrpides N, Islamaj Doğan R, Cohen K. Crowdsourcing and curation: perspectives from biology and natural language processing. Database 2016;2016:baw115 View
  27. Hochheiser H, Ning Y, Hernandez A, Horn J, Jacobson R, Boyce R. Using Nonexperts for Annotating Pharmacokinetic Drug-Drug Interaction Mentions in Product Labeling: A Feasibility Study. JMIR Research Protocols 2016;5(2):e40 View
  28. Klein A. Crowdsourcing voice editing and quality assessment of data collected from the largest mobile phone-based research study of Parkinson disease. Research Ideas and Outcomes 2016;2:e8848 View
  29. Créquit P, Mansouri G, Benchoufi M, Vivot A, Ravaud P. Mapping of Crowdsourcing in Health: Systematic Review. Journal of Medical Internet Research 2018;20(5):e187 View
  30. Lossio-Ventura J, Hogan W, Modave F, Guo Y, He Z, Yang X, Zhang H, Bian J. OC-2-KB: integrating crowdsourcing into an obesity and cancer knowledge base curation system. BMC Medical Informatics and Decision Making 2018;18(S2) View
  31. Heilbrun M, Chapman B, Narasimhan E, Patel N, Mowery D. Feasibility of Natural Language Processing–Assisted Auditing of Critical Findings in Chest Radiology. Journal of the American College of Radiology 2019;16(9):1299 View
  32. Foster M, Pandey A, Kreimeyer K, Botsis T. Generation of an annotated reference standard for vaccine adverse event reports. Vaccine 2018;36(29):4325 View
  33. de Amorim M, Saleme E, Assis Neto F, Santos C, Ghinea G. Crowdsourcing authoring of sensory effects on videos. Multimedia Tools and Applications 2019;78(14):19201 View
  34. Dumitrache A, Inel O, Timmermans B, Ortiz C, Sips R, Aroyo L, Welty C, Sabou M, Aroyo L, Bontcheva K, Bozzon A. Empirical methodology for crowdsourcing ground truth. Semantic Web 2021;12(3):403 View
  35. Kononova O, He T, Huo H, Trewartha A, Olivetti E, Ceder G. Opportunities and challenges of text mining in materials research. iScience 2021;24(3):102155 View
  36. Akanbi T, Zhang J. Design information extraction from construction specifications to support cost estimation. Automation in Construction 2021;131:103835 View
  37. Dumitrache A, Aroyo L, Welty C. Crowdsourcing Ground Truth for Medical Relation Extraction. ACM Transactions on Interactive Intelligent Systems 2018;8(2):1 View
  38. Ozcan S, Boye D, Arsenyan J, Trott P. A Scientometric Exploration of Crowdsourcing: Research Clusters and Applications. IEEE Transactions on Engineering Management 2022;69(6):3023 View
  39. Shingjergji K, Celebi R, Scholtes J, Dumontier M. Relation extraction from DailyMed structured product labels by optimally combining crowd, experts and machines. Journal of Biomedical Informatics 2021;122:103902 View
  40. Yu S, Chen T, Han L, Demartini G, Sadiq S. DataOps-4G: On Supporting Generalists in Data Quality Discovery. IEEE Transactions on Knowledge and Data Engineering 2022:1 View
  41. Williams J, Aleroud A, Zimmerman D. Detecting science-based health disinformation: a stylometric machine learning approach. Journal of Computational Social Science 2023;6(2):817 View

Books/Policy Documents

  1. Dumitrache A. The Semantic Web. Latest Advances and New Domains. View
  2. Şahin G. Computational Linguistics and Intelligent Text Processing. View
  3. Ebener S. Künstliche Intelligenz in Wirtschaft & Gesellschaft. View