Improved Protocol of Scientific Replications
By Ruth Daniel, Violet Henriques, and whatever friends swing by our house
In the early 2010s, the Replication Crisis shocked the world of psychology. Every year since, formerly “classic” studies have been replicated and found to be false. There are many factors that led to the Replication Crisis, but the most impactful are 1. Almost no one does replications and 2. when they do, scientific journals won’t publish them. But there’s an easy solution: we have decided to replicate and publish as many studies as we can (some in psychology, some in other fields) over the summer.
1. What is the existing target protocol you are hoping to improve or enhance?
The existing protocol we aim to improve is the practice of replication in scientific research. Currently, there is no standard for doing replications because 1. Most academic institutions require publishing research in scientific journals in order to work there 2. Scientific journals don’t publish replication studies, and 3. Scientists want jobs.
Up until now the main replication efforts in psychology have been big expensive teams of scientists replicating a lot of studies. We think that this protocol can be improved by the existence of many small teams (even those of just two people) running replications and publishing their results independently.
2. What is the core idea or insight about potential improvement you want to pursue?
Replications are crucial to the scientific process. There are two problems with unreplicated studies:
- Methodological problem - science is (in theory) built on the idea that anyone can look at your work, replicate it, and say “this guy was right!” Or “this guy was wrong!” If scientists aren’t doing that, science isn’t what scientists say that it is.
- Practical problem - in one study, two thirds of psychology research didn’t replicate. If this is true for all unreplicated science across all fields then there are a lot of falsehoods in science that we don’t have a protocol to notice or correct for.
3. What is your discovery methodology for investigating the current state of the target protocol?
Most of the discovery of the target protocol has already been done by bloggers, and some academic scientists. Also, we both have experience in the protocol of academic research because we are college students. A syllabus for a class on the replication crisis can be found here.
4. In what form will you prototype your improvement idea?
We will live together and replicate as many studies as we can over the summer and document the process on a blog and on YouTube. We will start by doing seven studies in the first week, though this is an ambitious rate and it’s unlikely we will keep it up for the whole summer. More detailed plan can be found here.
5. How will you field-test your improvement idea?
First, we find a spot to live. Next, we move in. Third, we start doing studies and we document the whole thing. This will involve a combination of blog and vlog posts, followed by publications of our studies.
6. Who will be able to judge the quality of your output? Ideally name a few suitable judges.
Anyone! We believe that scientific research should be accessible and we want our work to be solid enough and easy enough to understand that anyone can see our work and judge it accordingly.
7. How will you publish and evangelize your improvement idea?
As mentioned above, we will post all of our research on a blog and on YouTube. In addition, we plan on getting communities involved through in person events at local libraries, local colleges, and ideally local K-12 schools.
8. What is the success vision for your idea?
Lots of people, regardless of academic status, start doing replications and engaging with science as fellow scientists instead of as lay observers.