fmreli – A MATLAB toolbox for fMRI reliability analyses

The toolbox source code is hosted on GitHub and can be retrieved from this repository:
https://github.com/nkroemer/reliability

When using the toolbox, please cite as follows: Fröhner, J.H., Teckentrup, V.,  Smolka, M.N. & Kroemer, N.B. (2017). Addressing the reliability fallacy: Similar group effects may arise from unreliable individual effects.
bioRxiv. doi: https://doi.org/10.1101/215053.

fmreli toolbox beta v1.0

What is the fmreli toolbox?

The toolbox was designed to simplify the assessment of reliability and similarity of fMRI data across different sessions and contrasts. It incorporates common measures of global and local reliability. Moreover, it implements the assessment of reliability for cross-sectional designs with only a single run by randomly splitting the trials in half. fmreli offers a graphical user interface (GUI) and incorporates functions provided by SPM and and the Nifti and ANALYZE toolbox. A preprint of the paper detailing the use of the toolbox is available at bioRxiv.org.

Dependencies

To use the toolbox, you need to have MATLAB (v2015 or later) and SPM12 installed (see instructions).

Installing the toolbox

  1. Make sure that you have SPM12 installed.
  2. Download the latest version of the toolbox on github and save it into your intended directory.
  3. In the MATLAB 'Home' tab, click on the 'Set path' button and select 'Add with subfolders…'. Now, select the fmreli toolbox folder, save, and close. Windows users might have to open MATLAB as an administrator (by right-clicking on the MATLAB icon) to make permanent changes. Alternatively, you may use the command line to add the path:
pathtool
addpath(genpath('fmreli toolbox folder'))
savepath  fmreli toolbox folder/pathdef.m

By default, the toolbox comes with the CONN atlas (i.e., Harvard-Oxford brain atlas + AAL cerebellum atlas). Other customized atlases can be used, if you provide an 'atlas.nii' and a corresponding 'labels.mat' file containing the atlas labels.

That's it, you are good to go.

Basic setup

  1. After installing, you can call the toolbox by entering fmreli in the command line.

  2. The toolbox was developed to work primarily with first-level statistics in SPM12. The required folder structure is (at the moment) as follows:

    1. Project folder containing one subfolder for every participant
    2. Within every participant folder, there should be one folder for every session. The session should be given in the folder name and contain the first-level SPM stats including all outputs. For example: C:\user\projects\fmreli\subj_123\new_paradigm_1

  3. Next, you can specify the design:

    1. Click on the 'Design' dialog.
    2. Enter the number of subjects.
    3. Click on the 'load subject list' dialog and load a *.mat file containing the subject IDs as a (N-by-1) vector.
    4. Specify the number of sessions.
    5. Specify the directory where your first-level outputs are stored.
    6. Pick a prefix to be added to all output files and choose a directory where output files from the toolbox will be saved.
    7. Save your study design and close the dialog box. A *.mat file containing the information on your study design will be saved to your output folder.

  4. Now that you have specified the design, you can open it as a template and define the contrasts of interest for the following analysis.

    1. Click on the 'Contrast(s) of interest' dialog.
    2. Define the contrast(s) by the name they were given in the first-level SPM contrast manager. In case you renamed the output files, you can adjust the prefix accordingly in the dialog on the left.
    3. Click on the 'Save contrast definition' dialog. This saves a *.mat file to your fmreli output folder containing your contrast information.

If no first-level statistics with design matrices are available (e.g., resting state or DWI)

  1. fmreli beta 1.0 offers the possibility to use existing 4D files as input as well. Since this function was not at the heart of the development, there is only a limited work flow available at the moment.
    1. Please save your 4D files for the analysis in the defined results directory.
    2. If you have several conditions, please name the 4D files accordingly. For example, when there is data for two sessions in a placebo and drug condition: 4D_int_1.nii, 4D_int_2.nii, 4D_pla_1.nii, 4D_pla_2.nii
    3. If you have only one condition for several sessions, just denote the sessions by 4D_1.nii, 4D_2.nii ...
    4. Please also provide an exemplary 3D nifti called 'template_3D.nii' to facilitate creating result images with the image dimensions of your study. This can be simply one image of the 4D.nii (this will be fixed in a future version)
    5. In the GUI, please select design, enter the number of sessions, enter the number of subjects, and select the "yes" checkbox for only 4D data as input. As result directory, you will have to enter the directory where the corresponding 4D niftis are stored.
    6. Please note that if you do not provide the first-level statistics, the features split-half and overlap cannot be used, at least in the current version.

Split-half reliability: Assessing reliability within a single contrast and session

If you have only data from one fMRI run/session and are interested in the reliability of one contrast/condition, the toolbox offers the option to split the data.

  1. Click on the 'Split-Half' dialog.
  2. Load your study design.
  3. Indicate the number of the condition you want to split (according to the design information in the SPM.mat).
  4. Define a name for the split-half output file.
  5. Always check the 'split data' box throughout the use in different modules in analyzing split-half data.
  6. Click 'RUN' to start.

Assessment of global brain reliability

Whole-brain/ROI similarity of brain response patterns

The similarity module computes the global similarity of activation maps between participants across runs or sessions.

  1. Define the study design and contrasts of interest.
  2. OPTIONAL Select 'use ROI' and define the ROI name and directory if you want to restrict the analysis to a subset of voxels.
  3. Click 'RUN'.

The output from this module includes heatmaps, density plots, and histograms depicting the similarity between participants across runs or sessions. Moreover, a *.mat file will be saved containing the correlation coefficients and corresponding p-values.

Whole-brain/ROI overlap

In this module, the Dice and Jaccard coefficients of overlap of activated voxels can be computed.

  1. Define the study design and contrasts of interest.
  2. Set a p-value used as threshold for significant activation of a voxel. By default, it is applied to the first-level statistics, but you can choose to apply the threshold to the group-level statistics as well.
  3. OPTIONAL Select 'use ROI' and define the ROI name and directory if you want to restrict the analysis to a subset of voxels.
  4. Click 'RUN'.

In the output folder, you will find a file containing the coefficients for every subject.

Voxel-wise/ROI reliability

This module is used to calculate the correlation coefficients (intra-class coefficient, Pearson & Spearman correlation coefficients) on the voxel level within subjects and between contrasts.

  1. Define the study design and contrasts of interest.
  2. Check the boxes to select coefficients you want to analyze.
  3. OPTIONAL Select 'use ROI' and define the ROI name and directory if you want to restrict the analysis to a subset of voxels.
  4. Click 'RUN'.

This module will create several 3D-nifti output files named after the coefficient you had selected.

Summary functions

Atlas-based reliability

With this module, you can create an output aggregated for ROIs provided by the atlas. The output is a matrix containing the reliability coefficients for every ROI in the atlas for every analysis.

Bug report, suggestions & questions

Please do not hesitate to email us (nils.kroemer@uni-tuebingen.de & juliane.froehner@tu-dresden.de) regarding questions, feature requests or bug reports. We are happy to receive your feedback to improve fmreli. In case something does not work or you find a bug or error in the toolbox, please let us know via the issues page.