Skip to content

Contributing

Nolan Nichols edited this page Nov 18, 2016 · 5 revisions

Overview

You can contribute your open access dataset to the MetaSearch App by submitting a pull request with a csv file that follows a conventions that enables us to process your file and add it to the repository where it will show up in search results and visualizations.

We have a few required pieces of project and demographic information that allows us to merge with other datasets, but additional information is more than welcome. Generally each csv file has a corresponding T1 weighted MRI scan in NIfTI for each session/visit - this is optional but will allow you to link out to BrainBox to view and annotate the imaging data.

The general layout is that the crawler directory includes a list of organization/repository/owner directory that each have project directories with phenotype and mapping files.

  • The repository directory represents a data distribuion hub, such as a bucket on AWS, the NITRC-IR, NeuroVault, etc.
  • The project directory represents a specific distribution of data that contains files that all follow a common schema.
  • The phenotype files in a project all have the same column names and value representations expressed in csv format.
  • The mapping files in a project provide a description of the phenotype files for harmonizing with common data elements and value sets using json format.
- metasearch
| - crawler
  | - fcp-indi
    | - adhd200
      | - Brown_TestRelease_phenotypic.csv
      | - KKI_phenotypic.csv
      | - ...
      | - mapping.json
    | - project ... n
  | - organization ... n

Steps to Contribute

Setup

  1. Fork the MetaSearch Repo
  1. Create a repository directory (unless there is already one present)
  • Copy the example repository to ./metasearch/crawler/<repository_dir> using lowercase letters with words separated with a dash (e.g., ./metasearch/crawler/fcp-indi/)
  1. Create project directory(ies)
  • Rename the example project ./metasearch/crawler/<repository_dir>/<project_dir> using lowercase letters with words separated with an underscore (e.g., ./metasearch/crawler/fcp-indi/abide_initiative/)

Adding Raw Phenotypic Data

  1. Add your raw csv files to ./metasearch/crawler/<repository_dir>/<project_dir>
  • You can add multiple files (e.g., from different study sites), but each file should have the same columns.
  • If your csv files are separated by demographics, neuropsych tests, etc., first concatenate them into a single file with all the columns.

Adding a Mapping File

  1. Edit the mapping file that was copied over to ./metasearch/crawler/<repository_dir>/<project_dir>/mapping.json

Generating the Common (Mapped) Phenotypic File

  1. Submit a Pull Request on Github and we will pull in your changes and update things on our end!