Today, November 25, 2021, we organized the yearly workflow hackathon of the CORE lab (you can read last year’s version here)! To get all the lab members on one page and to reduce error as much as possible, we have a lab philosophy that is accompanied by various documents to facilitate our workflow. But research standards evolve and we also often notice that the way we conduct our research is not as optimal as we would like it to be. As new tools seem to be created daily, we can also not keep up with new developments without hurting our research. And it is important that those who do research daily provide input in the procedures they work with. That’s why we decided on a meeting once a year, where all lab members can have their say in how we are going to work that academic year. The new product is available here. Generally, the revision was much smaller than in previous years.
Here’s what we did:
We followed last year’s procedure to change the workflow:
- All lab members (Alessandro, Olivier, Ronan, Ade, Migs) named three things they liked (and definitely wanted to keep), three things they did not like (and wanted to get rid of), and three things they’d like to add. All of this was sent in a DM to Hans so as not to influence each other.
- Hans then put all the comments in one Google Doc on which the lab members got to vote (Hans vetoed two things).
- This year, we placed our emphasis a bit more on our research templates and reviewed them in more detail to see if they needed adjustments. We then discussed matters that people disagreed about or that were unclear and then revised the documents where necessary. Below you can see what we were discussing/working on.
Here are some things people really liked (non-exhaustive list):
- Our research templates in general (our team members thought they were more extensive than the standard templates provided by the OSF and by other teams)
- Our journal club and our 2-5pm Wednesday writing block
- Standup (fundamental according to most!)
- Independent code review
Here are some of our major changes (again, non-exhaustive list):
- We made the goals of our lab broader. Migs Silan joined us and he will focus on relationship quality, not just on co-regulation.
- We added a research template for meta-analyses (based on Alessandro’s work)
- We added some relevant training materials, such as:
- A MOOC on Reproducible research: methodological principles for transparent science
- A Video for Signing up for CREP
- A Video for Creating an OSF page
- A Video for Preparing and submitting a CREP project
- A Video for Completing a CREP project
- Hans will not do reviews for for-profit journals anymore (except when reciprocating) and follows Felix Schönbrodt’s reasoning for that.
Some things we realized we need to do better (again, non-exhaustive list):
- We did not update our Research Milestones Sheet as well as we could.
- We have not signed collaboration agreements. Alessandro and Ade will create them for their projects.
- We have not used CRediT for our projects. We will apply them.
- We have not done independent code reviews for all of our projects.
We decided that an independent reviewer from the CORE lab will do some checks on people’s projects to ensure some of these (and other) tasks are completed.
- We also did not really release many products or update our presentations on GitHub frequently enough. We will try to implement a workflow that improves this.
We also did some other things:
- We created a joint Zotero library
- We did a self-audit on our contributions to replications/big team science. We had committed to doing 10-25% of our projects to big team science/replications. We are at 25% now.
- We revised our lab canon (thanks Lorne Campbell and Denny Borsboom for your suggestions!)
In the coming weeks, we will dedicate ourselves to finalizing our Zotero libraries. Overall, it seemed like the lab philosophy has stabilized to some extent and we will focus mostly on consolidating existing practices, particularly for those in the last year of their Ph.D.!
(PS, thanks, SIPS, for giving us a commendation for this work!)