So here is my #OHBM2020Posters 'thread' for #OHBM2020

eCOBIDAS: a webapp checklist to improve neuroimaging methods & results reporting.

The app: https://ohbm.github.io/cobidas/#/ 
Poster: https://osf.io/wt8fj/ 
Chat: https://mattermost.brainhack.org/brainhack/channels/cobidas_checklist
Video chat: https://meet.jit.si/cobidas_checklist

#WIP
COBIDAS checklist

Using a checklist to help with more thorough and transparent methods and result reporting in neuroimaging. Hosted on the Open Science Framework

https://ohbm.github.io/cobidas/#/
OK so here is the rest of the thread but before we dive in here are the places where a lot of this information is centralized.

https://osf.io/anvqy/ 
https://github.com/Remi-Gau/COBIDAS_chckls

OK now let’s do this...
Remi-Gau/COBIDAS_chckls

Working repository to turn the COBIDAS guidelines to report methods and results in neuroimaging into a user friendly checklist - Remi-Gau/COBIDAS_chckls

https://osf.io/anvqy/
Have you ever been in one of the following situations?

1) Writing your first neuroimaging paper but unsure what level of detail to put in your methods section.

2) Reviewing a paper and being unsure what information is missing in that paper.
3) Wanting to replicate some study, reading the paper and finding there is not enough information in the methods section to do that or even to actually make sense of the results.
4) Comparing your results to those of a paper, but not finding enough information about how the authors got there. So you can’t assess where the differences between your numbers and theirs are coming from.
5) Having to look up into your MRI dataset acquisition parameters to write up part of your methods.
And then having to repeat yourself by copy-pasting parts of your methods section when filling in a journal checklist or uploading your results on @VaultNeuro.
6) Doing a meta-analysis and having to design a system to list the methods section of each study

7) Wanting to create pre-registration but not knowing what to put in it.

#ReinventTheWheel
Well we hope that this app can help with a lot of those in the long term.
The fact that we have a chronic problem in methods reporting quality is not new. In a 2012 paper Joshua Carp found that the methods of many papers would not allow independent investigators to fully reproduce their results. https://www.ncbi.nlm.nih.gov/pubmed/22796459 
The Secret Lives of Experiments: Methods Reporting in the fMRI Literature - PubMed

Replication of research findings is critical to the progress of scientific understanding. Accordingly, most scientific journals require authors to report experimental procedures in sufficient detail...

https://www.ncbi.nlm.nih.gov/pubmed/22796459
Hoping to help with this problem, the COBIDAS committee of @OHBM was created. It extended previous guidelines to create a report listing what should actually go into a methods section.

COBIDAS: https://www.humanbrainmapping.org/i4a/pages/index.cfm?pageid=3728

The report: https://www.biorxiv.org/content/10.1101/054262v
These guidelines are a great resource and they come with a very useful checklist in the form of a table at the end. The only drawback: it is a 30 PAGES LONG checklist...
And many items only concern some very specific use case ("why do I have to flip through the 2 pages for fMRI if I do tractography?")
So... Some of us decided to make this more user friendly. This was started at the @OhbmOpen hackathon in Rome in 2019. We got together and started breaking down those 30 pages into a spreadsheet.
@cassgvp had already started a lot of the work so that definitely helped. https://osf.io/ery2q/ 
COBIDAS reporting breakdown

Hosted on the Open Science Framework

https://osf.io/ery2q/
So now those 30 pages have been turned into a "much more manageable" checklist spreadsheet with... 462 items (and growing).

If you want to get dizzy you can have a look at it here:
https://docs.google.com/spreadsheets/d/1dCXP0MTK3DjY09ZFd7FXgv0Ngx16_YJwVBiXOeQbTho/edit?usp=sharing
How is that any better? Well I am glad you asked!

This format now becomes machine-readable. We can feed that into some app to render a checklist. And that brings us to the other aspect of that project. How to visualize this checklist?
We have been using some of the work done by the @ReproNim team to standardize how questionnaires are “represented” across studies in a project called Reproschema. https://github.com/ReproNim/reproschema
ReproNim/reproschema

standardizing form schema for interoperability. Contribute to ReproNim/reproschema development by creating an account on GitHub.

https://github.com/ReproNim/reproschema
This project aims to create a standardized library of questionnaires using linked-data and has a user interface to display those questionnaires and collect data.
And that’s pretty much all we were looking for to get that checklist running.

https://ohbm.github.io/cobidas/#/ 
At the moment the app is just a proof of concept based on the 90 metadata items one can fill in when uploading their neuroimaging results to @VaultNeuro. But we have been working on extending that to 200 items from our mother of all spreadsheets very soon.
You can click your way through the different sections of the checklist. There is a bare minimum of input validation. The app tries to minimize which items you are “exposed” to. You can export the results in a very simple text file that encapsulates “all” of your method section.
The next steps (beyond expanding the number of items)?
1) Using that simple machine-readable text file to automatically write large chunks of the method section--like fMRIprep does for you, depending on the options you chose to run it.

https://fmriprep.org/en/stable/citing.html
2) Minimize as much as possible the time it takes to fill in the checklist by making sure that users don’t have to scroll through irrelevant items and by allowing the app to read from a @BIDSstandard dataset to fill in automatically many aspects of the list.
3) The same could be done for some points of the “analysis section” of the checklist by reading info from a NIDM-results zip file. https://github.com/incf-nidash/nidmresults-spm
incf-nidash/nidmresults-spm

Export SPM results in a lightweight format. Contribute to incf-nidash/nidmresults-spm development by creating an account on GitHub.

https://github.com/incf-nidash/nidmresults-spm
4) Expand to the checklist to cover EEG and MEG work

@aina_puce @CyrilRPernet have been leading a similar effort to create guidelines for electrophysiology studies.

https://osf.io/a8dhx/ 
https://cobidasmeeg.wordpress.com/ 
Best Practices in Data Analysis and Sharing in Neuroimaging using MEEG

Non-invasive neuroimaging methods, including magnetoencephalography and electroencephalography (MEEG), have been critical in advancing the understanding of brain function in healthy people and in...

https://osf.io/a8dhx/
And @fedeadolfi @martinagvilas already turned a lot of those into another “tiny” spreadsheet.

https://docs.google.com/spreadsheets/d/1OhkmbtgIWdFxSVjpu6A8PWoAuqev0jY-98GFQlwBCy0/edit?usp=sharing

So there is a good chance that the app could cover EEG and MEG studies too.
COBIDAS_MEEG

clean Apdx,Table,A1,A2,A3,A4,Ref1(formula),D,Ref2(formula),Reference,ActivityName,aspect_1,aspect_2,aspect_3,aspect_4,item_name,question,details,format,choice_levels MEEG.,01,01,01,00,00,MEEG.01.01...

https://docs.google.com/spreadsheets/d/1OhkmbtgIWdFxSVjpu6A8PWoAuqev0jY-98GFQlwBCy0/edit?usp=sharing
Important thing to mention: all of this is a community based effort where most of the progress was done during hackathons @OhbmOpen 2019, @improvingpsych
2019 and @openmrbenelux 2020.

https://github.com/ohbm/hackathon2019/issues/49
https://osf.io/har94/ 
https://github.com/OpenMRBenelux/openmrb2020-hackathon/issues/12
A checklist for improving neuroimaging methods and results reporting · Issue #49 · ohbm/hackathon...

Improving the COBIDAS checklist for better neuroimaging methods and results reporting Remi Gau (ORCID) Project Description In 2012, in his review of the methods and results reporting of more than 2...

https://github.com/ohbm/hackathon2019/issues/49
So a special thanks to all the people who have so much time looking at spreadsheets that they ended up crying tears of blood.

@cassgvp @IRuotsa @fedeadolfi @akeshavan_ @Tim_van_Mourik @davidwmoreau @ZSjoerds @angietepp @martinagvilas @k_wiebels @johalgermissen @DorienHuijser
@WeedaWouter @etklapwijk @RuudHortensius @JamesEBartlett

Contributors list:
https://github.com/Remi-Gau/COBIDAS_chckls/blob/master/contributions.md#whos-who-on-the-cobidas-checklist-team
A special thanks to @ten_photos for connecting me with the amazing work done by Sanu and @satra_ on the Reproschema @ReproNim
And in case you want more methods checklists, you might want to check:

1) The transparency checklist @BalazsAczel
http://www.shinyapps.org/apps/TransparencyChecklist/

2) The CLAIM checklist for AI in medical imaging @neuroccino
https://claim.shinyapps.io/CLAIM/ 
3) The CREDnf checklist for cognitive-beharioural neurofeedback studies @rt_thibault
https://crednf.shinyapps.io/CREDnf/ 
You can follow @RemiGau.
Tip: mention @twtextapp on a Twitter thread with the keyword “unroll” to get a link to it.

Latest Threads Unrolled:
Those who think consciousness raising & movement building are disconnected, or even a distraction from electoral politics have not paid attention to the history of social change.In this thread, we
Read more
The 2020s will be known as the Remote Work decadeA few predictions of what is likely to emerge[ a thread ] Third Space: Office and Working from Home will
Read more
1/29: Have you ever had a concept explained to you that helps frame complex issues you’ve been wrestling with and opens your eyes to new possibilities? A concept that I
Read more
By continuing to use the site, you are consenting to the use of cookies as explained in our Cookie Policy to improve your experience.