The Transparent Psi Project (TPP): A Consensus-Based Replication of Bem 2011 Experiment 1

• Published in: The Journal of parapsychology 2022-10, Vol.86 (2), p.284-286
• Main Authors: Kekecs, Zoltan, Aczel, Balazs, Palfi, Bence, Szaszi, Barnabas, Szecsi, Peter, Zrubka, Mark, Kovacs, Marton, Bakos, Bence E, Giran, Kyra, Cousineau, Denis, Parent, Gabrielle, Tressoldi, Patrizio, Grassi, Massimo, Ballati, Arianna, Arnold, Dana, Evans, Thomas Rhys, Yam-ada, Yuki, Miller, Jeremy K, Liu, Huanxu, Yonemitsu, Fumiya, Dubrov, Dmitrii, Röer, Jan, Becker, Marvin, Schnepper, Roxane, Wiechert, Sera, Do, Linda, Raichert, Teddy, te Marvelde, Sophie, Poldver, Nele, Kreegipuu, Kairi, Oliveira, Raquel, Arriaga, Patrícia
• Format: Article
• Language: English
• Subjects: ESP; Extrasensory perception; Hypotheses; Pilot projects
• ISSN: 0022-3387

Introduction: Growing evidence for a systematic positive bias in the published research reports in various scientific fields had led to a 'crisis of confidence'. Studies testing controversial hypotheses, such as studies of extrasensory perception (ESP), suffer even more of the burden of the confidence crisis because the lack of trust promotes risk-aversion and the maintenance of the status quo. We need to develop objective indicators of trustworthy, reliable research studies independent of the hypothesis that is tested. Methods: This project aims to develop methodological tools that facilitate highly credible and rigorous research. Furthermore, we aim to conduct a multi-site, fully transparent replication of Bern's (2011) Experiment 1 using these credibility-enhancing methodologies. We have conducted a multi-site, fully transparent replication of Bem's (2011) Experiment 1. We have developed a study protocol for this replication effort via a Consensus Design Process. During this process more than twenty experts on the field (both proponents and opponents of the original ESP hypothesis) contributed to finalizing the protocol. The protocol includes a comprehensive toolkit of safeguards against researcher biases and mistakes that are often thought of as the primary cause for the abundance of non-replicable findings in psychology and biomedicine. The safeguards include radical transparency about the whole research pipeline via Born Open Data, Direct Data Deposition, Real-time Research Reports, automation, trusted third party oversight, tamper evident seals on data and software, documented training, and lab logs. Results: We have conducted a pilot study involving two research sites and one hundred eighty-four participants. This pilot study demonstrated the feasibility of our approach, and the adequacy of the consensus-derived study protocol. During this pilot investigation we observed 49.49% successful guesses (99.5% CI = 47%, 51.9%; posterior mode = 50.6%, posterior 90% HDI = 49.4%, 51.7%). Following the acceptance of our research protocol as a Stage 1 registered report in Royal Society Open Science, we started the main study. We have collected data from 2,097 participants at nine research sites worldwide. The data were more consistent with the model assuming that humans' guesses about the future, randomly determined, position of a target do not have a higher than chance success rate, rather than the model assuming that they do. In the main study so far, we obs