In my recent @Prolific online study (a response time task), I included an attention check halfway through the experiment. In this check, the instruction screen said to ignore the "correct" answer, and to just press the "B" key throughout the block (1/n)

I've just run the analysis, and 135 / 300 people (45%) failed this attention check. To me, this suggests people aren't reading the instructions. (2/n)

Do you think it's unfair to reject these participants? If I don't, how can I be sure my data are valid?

If you think I shouldn't reject, why not? The task is a speed--accuracy tradeoff experiment where I need to be sure participants are paying attention to the instructions.

Further complication: I've pre-registered this experiment and one of the exclusion criteria were that I would remove such participants. But I can't afford to replace 40%, so it's either reject them or reject my pre-reg plan.