add fMRIPRep example

README.md

@@ -7,7 +7,8 @@ If you see something that is missing or should be improved, please contribute!
 
 | name                         | description                                   |
 | ----                         | -----------                                   |
+| [fMRIPrep](fmriprep/)        |                                               |
 | [matlab](matlab/)            |                                               |
-| [python](python/)            |                                               |
 | [ncal](ncal/)                |                                               |
+| [python](python/)            |                                               |
 

fmriprep/README.md

+# fMRIPrep
+
+This template serves only as an example. It will need to be adapted in order to
+run for your project. Most importantly the
+
+1) `bids_dir`
+2) `output_dir`
+3) [command-line arguments](https://fmriprep.readthedocs.io/en/stable/usage.html#command-line-arguments)
+   to control which workflows are run
+
+## Prerequisites
+* a FreeSurfer license file located at `~/freesurfer_license.txt` (you can
+  copy the system one using `cp /opt/freesurfer/6.0/license.txt
+  ~/freesurfer_license.txt`. Hopefully this step can be removed soon.
+* a Singularity image of fMRIPrep. You can build it by running:
+  ```
+  singularity build ~/fmriprep-<version>.simg docker://poldracklab/fmriprep:<version>
+  ```
+  Be sure to substitute `<version>` for the version of fMRIPrep you want to build.
+* a BIDS dataset
+
+## Things to Note
+* **CPU:**: 1 CPU was chosen as (with the option used in this example) there was
+  no appreciable speed difference between 1 CPU and multi-CPU jobs.
+* **RAM:** 8GB is [recommended by upstream](https://brainscan.uwo.ca/research/cores/computational_core/img/Nov26-Fmriprep.pdf)
+* **DISK:** This script sets fMRIPrep's `--work-dir` to `/tmp` (which will use
+  the local cache disk on the compute nodes). The keeps the ~90GB of temp files
+  that fMRIPrep generates local to each node, and avoids running over storage
+  quota limits.
+
+## Running
+Adjust the variables in the script, most especially the `BIDS_DIR`,
+`OUTPUT_DIR`, and `FMRIPREP`. Also adjust the subjects for-loop to meet your
+needs.
+
+When you've made your changes, run it like so:
+```
+./gen_submit.sh | condor_submit
+```

fmriprep/gen_submit.sh

+#!/bin/bash
+# v1.0
+
+CPUS='1'
+RAM='8G'
+DISK='90G'
+LOGS_DIR="~/htcondor-templates/fmriprep/logs"
+
+FMRIPREP='~/fmriprep-20.0.3.simg'
+BIDS_DIR='/data/project/llamas'
+OUTPUT_DIR='/data/project/llamas/Derivatives'
+FREESURFER_LIC='~/freesurfer_license.txt'
+
+# create the logs dir if it doesn't exist
+[ ! -d "${LOGS_DIR}" ] && mkdir -p "${LOGS_DIR}"
+
+# print the .submit header
+printf "# The environment
+universe       = vanilla
+getenv         = True
+request_cpus   = ${CPUS}
+request_memory = ${RAM}
+request_disk   = ${DISK}
+
+# Execution
+initial_dir    = \$ENV(HOME)/htcondor-templates/fmriprep
+executable     = /usr/bin/singularity
+\n"
+
+# loop over all subjects
+for sub in sub-{01..36}; do
+    printf "arguments = run --cleanenv -B ${BIDS_DIR} ${FMRIPREP} \
+                        --n_cpus ${CPUS} \
+                        --skull-strip-fixed-seed \
+                        --fs-license-file ${FREESURFER_LIC} \
+                        --work-dir /tmp \
+                        ${BIDS_DIR} ${OUTPUT_DIR} participant \
+                        --participant-label ${sub}\n"
+    printf "log       = ${LOGS_DIR}/\$(Cluster).\$(Process).sub-${sub}.log\n"
+    printf "output    = ${LOGS_DIR}/\$(Cluster).\$(Process).sub-${sub}.out\n"
+    printf "error     = ${LOGS_DIR}/\$(Cluster).\$(Process).sub-${sub}.err\n"
+    printf "Queue\n\n"
+done