Published on in Vol 15, No 5 (2013): May

Crowdsourcing a Normative Natural Language Dataset: A Comparison of Amazon Mechanical Turk and In-Lab Data Collection

Crowdsourcing a Normative Natural Language Dataset: A Comparison of Amazon Mechanical Turk and In-Lab Data Collection

Crowdsourcing a Normative Natural Language Dataset: A Comparison of Amazon Mechanical Turk and In-Lab Data Collection

Authors of this article:

Daniel R Saunders1 ;   Peter J Bex2 ;   Russell L Woods1

Journals

  1. Dixon G. Making Vaccine Messaging Stick: Perceived Causal Instability as a Barrier to Effective Vaccine Messaging. Journal of Health Communication 2017;22(8):631 View
  2. Lu W, Guttentag A, Elbel B, Kiszko K, Abrams C, Kirchner T. Crowdsourcing for Food Purchase Receipt Annotation via Amazon Mechanical Turk: A Feasibility Study. Journal of Medical Internet Research 2019;21(4):e12047 View
  3. Thawrani V, Londhe N, Singh R. Crowdsourcing of Medical Data. IETE Technical Review 2014;31(3):249 View
  4. Sadler P, Sonnert G, Coyle H, Miller K. Identifying Promising Items: The Use of Crowdsourcing in the Development of Assessment Instruments. Educational Assessment 2016;21(3):196 View
  5. Costela F, Saunders D, Kajtezovic S, Rose D, Woods R. Measuring the Difficulty Watching Video With Hemianopia and an Initial Test of a Rehabilitation Approach. Translational Vision Science & Technology 2018;7(4):13 View
  6. Costela F, Woods R. When Watching Video, Many Saccades Are Curved and Deviate From a Velocity Profile Model. Frontiers in Neuroscience 2019;12 View
  7. Lewinson R, Katz J. Influencing Pain Inferences Using Random Numerical Anchoring: Randomized Controlled Trial. JMIR Human Factors 2020;7(1):e17533 View
  8. Dixon G. Negative Affect as a Mechanism of Exemplification Effects. Communication Research 2016;43(6):761 View
  9. Sauter M, Draschkow D, Mack W. Building, Hosting and Recruiting: A Brief Introduction to Running Behavioral Experiments Online. Brain Sciences 2020;10(4):251 View
  10. Dixon G, McComas K, Besley J, Steinhardt J. Transparency in the food aisle: the influence of procedural justice on views about labeling GM foods. Journal of Risk Research 2016;19(9):1158 View
  11. Costela F, Woods R. The Impact of Field of View on Understanding of a Movie Is Reduced by Magnifying Around the Center of Interest. Translational Vision Science & Technology 2020;9(8):6 View
  12. Martins N, Matthews N, Ratan R. Playing by the Rules: Parental Mediation of Video Game Play. Journal of Family Issues 2017;38(9):1215 View
  13. Costela F, Sheldon S, Walker B, Woods R. People with Hemianopia Report Difficulty with TV, Computer, Cinema Use, and Photography. Optometry and Vision Science 2018;95(5):428 View
  14. Turner-McGrievy G, Helander E, Kaipainen K, Perez-Macias J, Korhonen I. The use of crowdsourcing for dietary self-monitoring: crowdsourced ratings of food pictures are comparable to ratings by trained observers. Journal of the American Medical Informatics Association 2015;22(e1):e112 View
  15. Kreitewolf J, Wöstmann M, Tune S, Plöchl M, Obleser J. Working-memory disruption by task-irrelevant talkers depends on degree of talker familiarity. Attention, Perception, & Psychophysics 2019;81(4):1108 View
  16. Woods K, Siegel M, Traer J, McDermott J. Headphone screening to facilitate web-based auditory experiments. Attention, Perception, & Psychophysics 2017;79(7):2064 View
  17. Hosseini M, Shahri A, Phalp K, Taylor J, Ali R. Crowdsourcing: A taxonomy and systematic mapping study. Computer Science Review 2015;17:43 View
  18. Sato P, Hötzel M, Von Keyserlingk M. American Citizens’ Views of an Ideal Pig Farm. Animals 2017;7(8):64 View
  19. Saunders D, Bex P, Rose D, Woods R, Pelli D. Measuring Information Acquisition from Sensory Input Using Automated Scoring of Natural-Language Descriptions. PLoS ONE 2014;9(4):e93251 View
  20. Costela F, Saunders D, Rose D, Katjezovic S, Reeves S, Woods R. People With Central Vision Loss Have Difficulty Watching Videos. Investigative Opthalmology & Visual Science 2019;60(1):358 View
  21. Lansford K, Borrie S, Bystricky L. Use of Crowdsourcing to Assess the Ecological Validity of Perceptual-Training Paradigms in Dysarthria. American Journal of Speech-Language Pathology 2016;25(2):233 View
  22. Kuang J, Argo L, Stoddard G, Bray B, Zeng-Treitler Q. Assessing Pictograph Recognition: A Comparison of Crowdsourcing and Traditional Survey Approaches. Journal of Medical Internet Research 2015;17(12):e281 View
  23. Ogletree A, Katz B. How Do Older Adults Recruited Using MTurk Differ From Those in a National Probability Sample?. The International Journal of Aging and Human Development 2021;93(2):700 View
  24. Cardoso C, Hötzel M, Weary D, Robbins J, von Keyserlingk M. Imagining the ideal dairy farm. Journal of Dairy Science 2016;99(2):1663 View
  25. Cui L, Carter R, Zhang G. Evaluation of a Novel Conjunctive Exploratory Navigation Interface for Consumer Health Information: A Crowdsourced Comparative Study. Journal of Medical Internet Research 2014;16(2):e45 View
  26. Clarke C, Weberling McKeever B, Holton A, Dixon G. The Influence of Weight-of-Evidence Messages on (Vaccine) Attitudes: A Sequential Mediation Model. Journal of Health Communication 2015;20(11):1302 View
  27. Bateman D, Brady E, Wilkerson D, Yi E, Karanam Y, Callahan C. Comparing Crowdsourcing and Friendsourcing: A Social Media-Based Feasibility Study to Support Alzheimer Disease Caregivers. JMIR Research Protocols 2017;6(4):e56 View
  28. Mares M, Stephenson L, Martins N, Nathanson A. A house divided: Parental disparity and conflict over media rules predict children's outcomes. Computers in Human Behavior 2018;81:177 View
  29. Costela F, Reeves S, Woods R. An implementation of Bubble Magnification did not improve the video comprehension of individuals with central vision loss. Ophthalmic and Physiological Optics 2021;41(4):842 View
  30. Huynh H, Dicke-Bohmann A, Nelson C. What do humble doctors do? A conceptualization of clinician humility from the patient’s perspective. Patient Education and Counseling 2021;104(12):3066 View
  31. Costela F, Reeves S, Woods R. The Effect of Zoom Magnification and Large Display on Video Comprehension in Individuals With Central Vision Loss. Translational Vision Science & Technology 2021;10(8):30 View
  32. Segen V, Avraamides M, Slattery T, Colombo G, Wiener J, Castro-Alonso J. Comparable performance on a spatial memory task in data collected in the lab and online. PLOS ONE 2021;16(11):e0259367 View
  33. Del Popolo Cristaldi F, Granziol U, Bariletti I, Mento G. Doing Experimental Psychological Research from Remote: How Alerting Differently Impacts Online vs. Lab Setting. Brain Sciences 2022;12(8):1061 View
  34. Morales A, Costela F, Woods R. Saccade Landing Point Prediction Based on Fine-Grained Learning Method. IEEE Access 2021;9:52474 View
  35. Knowlton K, von Keyserlingk M. To Treat or Not to Treat: Public Attitudes on the Therapeutic Use of Antibiotics in the Dairy Industry—A Qualitative Study. Animals 2023;13(18):2913 View
  36. Kuo C, Koralesky K, von Keyserlingk M, Weary D. Gene editing in animals: What does the public want to know and what information do stakeholder organizations provide?. Public Understanding of Science 2024;33(6):725 View