I mentioned this elsewhere but wanted to post here too. Might want to reconsider using old 3 CRT q's for online studies. Much has changed in past few years.

Many current MTurkers have answers memorized/bookmarked. Bots may be programmed w/ answers too. https://www.reddit.com/r/TurkerNation/comments/9nx8wc/lily_pads_and_bats_balls_what_survey_answers_have/
Lily pads and bats & balls - what survey answers have you...

**Edited to add:** There is an interesting paper out that says that CRT exposure doesn't make a difference to the data collected. [Read it...

https://www.reddit.com/r/TurkerNation/comments/9nx8wc/lily_pads_and_bats_balls_what_survey_answers_have/
As noted in the post, it's likely that many MTurkers have pre-written answers to "priming" questions and simply copy/paste. Helps explain why many MTurkers finish HITs w/ writing prompts in blazing fast times. From the linked post (from an MTurk subreddit) in my previous tweet:
Here are examples of some of the browser scripts they use to highlight words from commonly asked (i.e., overused) scales, psychometric tasks, and attn chks so they can paste in pre-written answers or pull up bookmarked answers.

https://greasyfork.org/en/scripts/8093-mturk-survey-highlight-words https://chrome.google.com/webstore/detail/multihighlighter/ocifbglmlbpgpbflnkfpclkmckoollbn?hl=en
MultiHighlighter

MultiHighlighter allows multi-word searching/highlighting on almost any web page. Each term is highlighted with a unique color.

https://greasyfork.org/en/scripts/8093-mturk-survey-highlight-words
Lastly, many issues w/ online data quality could easily be solved if more/all researchers would compensate workers fairly (i.e., treat them like actual human beings) & not be lazy or sloppy w/ research design.

Shit pay + shit design = shit data https://blog.turkerview.com/2019-sentiment-analysis-mturk-wages-measuring-fairness/
Sentiment Analysis: MTurk Wages & Measuring Fairness

We took a look at wage trends for in-task payments on MTurk over the last two years and have developed updated guidelines to share with researchers based on that data.

https://blog.turkerview.com/2019-sentiment-analysis-mturk-wages-measuring-fairness/
Forgot to add that another glaring issue with the CRT (in general) is we still don't know for sure what it actually/specifically measures. https://twitter.com/wimdeneys/status/1273321415111053313
You can follow @MetacogniShane.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled:
Those who think consciousness raising & movement building are disconnected, or even a distraction from electoral politics have not paid attention to the history of social change.In this thread, we
Read more
The 2020s will be known as the Remote Work decadeA few predictions of what is likely to emerge[ a thread ] Third Space: Office and Working from Home will
Read more
1/29: Have you ever had a concept explained to you that helps frame complex issues you’ve been wrestling with and opens your eyes to new possibilities? A concept that I
Read more
By continuing to use the site, you are consenting to the use of cookies as explained in our Cookie Policy to improve your experience.