Department(s)

Social Science

Document Type

Article

Publication Date

2019

Keywords

Amazon’s Mechanical Turk; MTurk; Online Data Collection; Psychometrics; Score Reliability

Abstract

Previous research has supported the use of Amazon’s Mechanical Turk (MTurk) for online data collection in individual differences research. Although MTurk Masters have reached an elite status because of strong approval ratings on previous tasks (and therefore gain higher payment for their work) no research has empirically examined whether researchers actually obtain higher quality data when they require that their MTurk Workers have Master status. In two different online survey studies (one using a personality test and one using a cognitive abilities test), the psychometric reliability of MTurk data was compared between a sample that required a Master qualification type and a sample that placed no status-level qualification requirement. In both studies, the Master samples failed to outperform the standard samples.

Publication Title

Journal of Individual Differences

DOI

https://doi.org/10.1027/1614-0001/a000300

Download

Share

COinS