Documentation

Software documentation

CONN toolbox manual

Working with cloud or HPC environments

Using CONN in HPC (High Performance Computing) clusters or multi-processor environments: CONN grid, Cluster configuration

Using CONN to securely interact with and analyze data in the cloud or in any network-accessible host : Remote projects, CONN remotely, command-line cheat-sheet

Scripting in CONN

   Main user-facing scripting functions in CONN: 

>> conn_batch

create and manage all aspects of your functional connectivity projects using scripts 

>> conn_module

integrate modular CONN functions into your own preprocessing or analysis pipelines (for arbitrary / non-connectivity data)

see conn extensions for a description of all modules currently available in CONN: conn module PREP, conn module GLM, conn module EL, conn module FL

   Examples of use: see all "How to..." sections in fMRI Methods book

Other useful scripting / command-line functions in CONN:  (sorted by topic)

File input/output/conversion:

>> conn_surf_read / conn_surf_write

reads/writes surface- level data (FreeSurfer fsaverage level-8 163K tessellation, with 163842 vertices and 327680 faces) from NIFTI format file

type "help conn_surf_write" in Matlab for additional details

e.g. :

>> conn_surf_write('data.surf.nii', rand([163842*2, 1]) ); % creates surface NIFTI file (e.g. 2 hemispheres 1 datapoint)

>> data = conn_surf_read('data.surf.nii'); % reads surface NIFTI file

>> conn_mtx_read / conn_mtx_write

reads/writes matrix-level data (arbitrary square matrix) from NIFTI format file

type "help conn_mtx_write" in Matlab for additional details

e.g. :

>> conn_mtx_write('data.mtx.nii', rand([10, 10, 1]) ); % creates matrix NIFTI file (e.g. 10 ROIs 1 datapoint)

>> data = conn_mtx_read('data.mtx.nii'); % reads matrix NIFTI file

>> conn_vol_read / conn_vol_write

reads/writes volume-level data (arbitrary 3D/4D matrix) from NIFTI format file

type "help conn_vol_write" in Matlab for additional details

e.g. :

>> conn_vol_write('data.nii', rand([64, 64, 64, 1]) ); % creates volume NIFTI file (e.g. single 3D image)

>> data = conn_vol_read('data.nii'); % reads volume NIFTI file

>> conn_dcm2nii : converts DICOM (.dcm ) to NIFTI format (.nii)

>> conn_gz2nii : converts compressed NIFTI (.gz ) to NIFTI format (.nii)

>> conn_matc2nii : converts CONN (.matc ) to NIFTI format (.nii)

>> conn_mgh2nii : converts MGH (.mgh .mgz ) to NIFTI format (.nii)

>> conn_annot2nii : converts FREESURFER annotation (lh*.annot rh*.annot) to surface NIFTI format (.surf.nii)

>> conn_surf_curv2nii : converts FREESURFER paint (lh. rh.) to surface NIFTI format (.surf.nii)

>> conn_surf_gii2nii : converts FREESURFER surface GIFTI (lh*.gii rh*.gii) to surface NIFTI format (.surf.nii)

>> conn_surf_nii2curv : converts surface NIFTI (.surf.nii) to FREESURFER paint format (lh. rh.)

>> conn_surf_nii2gii : converts surface NIFTI (.surf.nii) to FREESURFER surface GIFTI format (lh*.gii rh*.gii)

>> conn_cat2mgh : converts CAT12 output files to FREESURFER format

>> conn_surf_surf2vol :  projects surface NIFTI (.surf.nii) to volume NIFTI (.nii) format (in MNI-space)

>> conn_surf_vol2surf :  projects volume NIFTI (.nii) (in MNI-space) to surface NIFTI format (.surf.nii)

>> conn_cache pull/push

manages local cache (for fast repeated access to files from slow or remote drives)

type "help conn_cache" in Matlab for additional details

e.g. :

>> file = conn_cache('pull','/Volumes/ext/data.txt'); % creates local copy of remote/network file

>> edit(file); % change local copy

>> conn_cache('push','/Volumes/ext/data.txt'); % updates remote/network file with changes if necessary

Display functions:

>> conn_display <SPM.mat>

displays group-level analysis results (including analyses of standard 3D volume-level data, as well as surface- and matrix-level analyses)

type "help conn_display" in Matlab for additional details

e.g. :

>> hf = conn_display('/results/analysis01/SPM.mat'); % displays 2nd-level analysis results

>> conn_display(hf, 'export_mask', 'mask.nii'); % exports supra-threshold mask

>> conn_mesh_display <file.nii>

displays MNI-space 3D volume or surface data projected onto a 3D cortical reference surface (Freesurfer tessellation of ICBM 2009c Nonlinear Asymmetric template)

type "help conn_mesh_display" in Matlab for additional details

e.g. :

>> fh = conn_mesh_display('/results/analysis01/spmT_0001.nii'); % displays T-stat map projected on cortical surface

>> fh('print', 4, 'figure01.jpg', '-nogui'); % prints 4-view mosaic

>> conn_slice_display <file.nii>

displays MNI-space 3D volume data overlaid on individual slices of a reference structural image (ICBM 2009c Nonlinear Asymmetric template)

type "help conn_slice_display" in Matlab for additional details

e.g. :

>> fh = conn_slice_display('/results/analysis01/spmT_0001.nii'); % displays T-stat map on individual slices

>> fh('pointer_mm',[0 0 10]); % sets display center coordinates to [0 0 10] mm

>> fh('multisliceset',true,16,8); % sets multi-slice display to 16 slices and 8mm interslice

>> fh('togglegui',true); % hide GUI buttons

>> fh('print', 'figure01.jpg', '-nogui'); % prints figure

>> conn_mtx_display <file.nii>

displays matrix-level data 

type "help conn_mtx_display" in Matlab for additional details

e.g. :

>> fh = conn_mtx_display('/results/analysis02/spmT_0001.nii'); % displays T-stat map from matrix group-analysis

>> fh('print', 'figure01.jpg', '-nogui'); % prints figure

>> conn_table_display (data)

displays contingency table data 

type "help conn_table_display" in Matlab for additional details

e.g. :

>> data = conn_roioverlaps( 'results.ROIs.nii', '/software/conn/rois/networks.nii'); % computes overlap between two groups of ROIs

>> fh = conn_table_display(a.overlap,'rlabel',a.rows_names,'clabel',a.cols_names); % displays table

>> fh('print', 'figure01.jpg', '-nogui'); % prints figure

Using CONN to process/analyze arbitrary -non-connectivity- data:

>> conn module preprocessing <filename.json>

preprocesses and/or denoises arbitrary functional data. Processing methods include all those described in the Preprocessing pipeline and Denoising pipeline sections

type "help conn_module" in Matlab for additional details

e.g. :

>> conn_module( 'preprocessing', 'PREPROC01.json'); % runs default preprocessing pipeline on raw functional/structural data

example PREPROC01.json: (preprocessing only)

{

 "structurals": [ "/data/anat.nii" ],

 "functionals": [ [

   "/data/run1/func.nii",

   "/data/run2/func.nii"

  ] ],

 "steps": "default_mni",

 "sliceorder": "interleaved (Siemens)"

}

example PREPROC02.json: (preprocessing + denoising)

{

 "structurals": [ "/data/anat.nii" ],

 "functionals": [ [

   "/data/run1/func.nii",

   "/data/run2/func.nii"

  ] ],

 "steps": ["default_mni", "functional_regression", "functional_bandpass" ],

 "sliceorder": "interleaved (Siemens)",

 "reg_names": ["realignment", "scrubbing", "White Matter", "CSF"],

 "reg_dimensions": [Inf,Inf,5,5],

 "reg_deriv": [1,0,0,0],

 "bp_filter": [0.008,Inf]

}

>> conn module glm <filename.json>

performs group-level analyses of arbitrary data (including analyses of standard 3D volume-level data, as well as surface- and matrix-level analyses). Analyses methods include all those described in the Cluster-level inferences section (e.g. RFT, randomization/permutation, TFCE, NBS, etc.)

type "help conn_module" in Matlab for additional details

e.g. :

>> conn_module( 'glm', 'ANALYSIS01.json'); % runs 2nd-level analysis of arbitrary input images

>> conn_display( '/results/analysis01' ); % displays results

example ANALYSIS01.json (defining a two-sample t-test with N=10):

{

 "data": [

  "/data/file01.nii",

  "/data/file02.nii",

  "/data/file03.nii",

  "/data/file04.nii",

  "/data/file05.nii",

  "/data/file06.nii",

  "/data/file07.nii",

  "/data/file08.nii",

  "/data/file09.nii",

  "/data/file10.nii"

 ],

 "design_matrix": [[1,0],[1,0],[1,0],[1,0],[1,0],[0,1],[0,1],[0,1],[0,1],[0,1]],

 "contrast_between": [-1,1],

 "contrast_within": 1,

 "folder": "/results/analysis01"

}

Managing/submitting jobs in HPC/cluster environments:

>> conn_batch(..., 'parallel.N', N)

submits a request for N nodes in your cloud or HPC environment to execute the steps specified in a conn_batch command (each node will process/analyze a different subset of subjects)

type "help conn_batch" in Matlab for additional details

e.g.:

>> conn_batch('Denoising.done', true, 'parallel.profile', 'slurm', 'parallel.N', 10); 

% runs CONN denoising step on your current CONN project using 10 simultaneous

      % nodes/jobs, and using the Slurm protocol to submit these jobs

>> conn jobmanager 

manages jobs submitted to cloud or HPC environments

type "help conn_jobmanager" in Matlab for additional details

e.g.:

>> conn_jobmanager profiles % displays a list of all available HPC profiles

e.g.:

>> conn_jobmanager % launches gui displaying the status of any pending or running jobs

e.g.:

>> conn_jobmanager restartif error % restarts any nodes/jobs which have stopped with errors

>> conn('submit', fcn_name  [, arg1, arg2, ... ] )

 submits a request for a single node in your cloud or HPC environment to run the Matlab function or script fcn_name with (optional) input arguments arg1, arg2., etc.

e.g.:

>> conn submit myscript01; % run myscript01 remotely and wait until job finishes

e.g.:

>> job = conn('submit', 'myscript01'); % run myscript01 remotely and return immediately (before job starts)

>> conn_jobmanager('statusjob', job, [], true, true); % checks the status of a submitted job

>> conn submit_spmbatch <matlabbatch>

 submits arbitrary SPM batch <matlabbatch> (.m or .mat file, or matlabbatch structure) as a single job to your cloud or HPC environment

e.g.:

>> matlabbatch{1}.spm... = ... % define matlabbatch structure

>> save batchfile.mat matlabbatch;  % saves matlabbatch information in .mat file

>> conn submit_spmbatch batchfile.mat; % submit SPM batch file and wait until job finishes

Working with CONN remotely (command-line options)

>> conn remotely on

starts SSH-based encrypted/secure connection with remote server (this is not needed if you are already connected to a remote server in CONN's GUI)

e.g.:

>> conn remotely on

  Server address []: yourinstitution.edu

  Username []: 

  ...

>> conn_server cmd

runs interactive Matlab commands in remote CONN server

e.g.:

>> conn_server cmd % interactive command-line prompt (commands executed in CONN server)

 yourinstitution.edu >> vpa(pi,100) % run vpa function in remote CONN server

  ans =

  3.141592653589793238462643383279502884197169399375105820974944592307816406286208998628034825342117068

 yourinstitution.edu >> quit % exits interactive commands 

>> [out1, out2, ...] = conn_server('run', fcn_name, arg1, arg2, ... )

runs in remote CONN server the Matlab function or script fcn_name with (optional) input arguments arg1, arg2., etc.

e.g.:

>> out = conn_server('run', 'vpa', pi, 1000); run vpa function in remote CONN server and imports output variable

see "help conn_remotely" and command-line remote connection guide for additional details