The first approach ‘hacks’ the number of participants in studies, the second approach ‘hacks’ the number of variables in the analysis. Second, researchers may group participants post hoc along potential but unplanned independent grouping variables.

First, researchers may violate the data collection stopping rules of null hypothesis significance testing by repeatedly checking for statistical significance with various numbers of participants.

I illustrate several forms of two special cases of data dredging. In order to build better intuition to avoid, detect and criticize some typical problems, here I systematically illustrate the large impact of some easy to implement and so, perhaps frequent data dredging techniques on boosting false positive findings. Hidden data dredging (also called p-hacking) is a major contributor to this crisis because it substantially increases Type I error resulting in a much larger proportion of false positive findings than the usually expected 5%. There is increasing concern about the replicability of studies in psychology and cognitive neuroscience.

Department of Psychology, University of Cambridge, Cambridge, UK.

How to avoid data dredging