Cheat-Sheet

some useful Matlab commands

when working with a remote server

INITIALIZATION

>> conn remotely on

starts SSH-based encrypted/secure connection with remote server (this will automatically launch a new server if the last one is down)

>> conn remotely off

ends connection with remote server


REMOTE EXECUTION

>> conn_server cmd

starts remote command-line prompt (type "quit" to return to local command-line prompt)

>> [out1, out2, ...] = conn_server('cmd', expression);

evaluates expression in remote server

>> conn_server('run', fcn_name, fcn_arg1, fcn_arg2, …);

runs function/script in remote server (blocks execution until function finishes)

>> conn_server(‘run_immediatereturn', fcn_name, fcn_arg1, fcn_arg2, …);

runs function/script in remote server (does not block execution)

>> [out1, out2, …] = conn_server('run', fcn_name, fcn_arg1, fcn_arg2, …);

runs function in remote server, returns output arguments locally

>> [out1, out2, …] = conn_server(‘run_keep', fcn_name, fcn_arg1, fcn_arg2, …);

runs function in remote server, keeping output arguments remotely (returns link to remote variables)


REMOTE JOB SUBMISSION

>> conn('submit', fcn_name, fcn_arg1, fcn_arg2, ... );

runs function in remote server HPC/cluster environment (blocks execution)

>> status = conn('submit', fcn_name, fcn_arg1, fcn_arg2, ... );

runs function in remote server HPC/cluster environment (does not block execution)



https://www.conn-toolbox.org/resources/documentation

FILE TRANSFER

>> conn_server('push', file_local, file_remote);

copies local file to remote server

>> conn_server('pull', file_remote, file_local);

copies file in remote server to local computer

>> conn_server_ssh('folderpush', folder_local, folder_remote);

copies local directory to remote server using SCP

>> conn_server_ssh('folderpull', folder_remote, folder_local);

copies directory in remote server to local computer using SCP


CONFIGURATION

>> conn remotely setup;

installs remote-access options (run once in server)

>> edit ~/connserverinfo.json

edits remote-access options (run in server)

>> conn jobmanager settings;

change remote server HPC/cluster environment options


ADVANCED COMMUNICATIONS

>> conn_server('connect', IP , SERVER_ID);

connects to CONN server through existing channel (SERVER_ID = portnumberCONNprivatekey)

>> conn_server('start' [, PORT, PUBLICKEY]);

initializes CONN server and open communication channel (run in server)

>> conn_server disconnect

closes communication channel to remote server

>> conn_server exit

terminates remote server and closes communication channel

remote storage

FILE CACHE

(note: use /CONNSERVER/* format to refer to files in server)

>> file_local = conn_cache('pull', file_remote);

creates local-copy of a remote file

>> conn_cache('push', file_remote);

updates a remote file with any changes to its local-copy

>> file_local = conn_cache('new', file_remote);

creates local-copy of a yet-to-be-created remote file

>> conn_cache clearall

clears all locally-cached files


GENERAL FILE UTILITIES

(note: use /CONNSERVER/* format to refer to files in server)

>> data = conn_loadmatfile(filename, ...)

loads data from a MAT-file (same syntax as Matlab's load function)

>> conn_savematfile(filename, ...)

saves data to a MAT-file (same syntax as Matlab's save function)

>> conn_fileutils( fcn_name, ...)

miscelaneous file management utility functions (overloading original fcn_name functions to add support for files/folders located within remote server):

conn_fileutils('fileread', filename) : reads text file (see fileread)

conn_fileutils('filewrite', filename, str) : creates text file with cell array str (one line per element)

conn_fileutils('fileappend', filename, str) : appends text file with cell array str (one line per element)

conn_fileutils('textread', filename) : reads formatted data from text file (see textread)

conn_fileutils('copyfile', source, target) : copies file (within same-machine transfers only)

conn_fileutils('movefile', source, target) : moves file

conn_fileutils('renamefile', source, target) : renames file

conn_fileutils('deletefile', filename) : deletes file

conn_fileutils('emptyfile', filename) : creates empty file

conn_fileutils('mkdir', dirname) : creates new folder (see mkdir)

conn_fileutils('rmdir', dirname) : removes folder (see rmdir)

conn_fileutils('linkdir', dirname, link) : creates symbolic-link to folder

conn_fileutils('dir', dirname) : lists files in folder (see dir)

conn_fileutils('isdir', dirname) : returns true if dirname is a directory (see isdir)

conn_fileutils('cd', dirname) : changes current working directory (see cd)

conn_fileutils('homedir') : returns user-specific home directory

conn_fileutils('uigetfile', ...) : GUI to select a file (see uigetfile)

conn_fileutils('uiputfile', ...) : GUI to select a new filename (see uiputfile)

conn_fileutils('imread', filename,...) : reads image file (see imread)

conn_fileutils('imwrite', A, filename,...) : writes image file (see imwrite)

conn_fileutils('imopen', filename) : displays image file in OS-viewer


LOCAL/REMOTE PATHS

>> conn_server('mnt', mount_point);

mounts remote server root directory in mount_point local directory

default mount point name = /CONNSERVER

>> conn_server('util_isremotefile', filename)

returns 1 if filename is in remote-storage format (located within mount point)

>> conn_server('util_remotefile', filename)

conforms filename to remote-storage format (e.g. /data -> /CONNSERVER/data)

>> conn_server('util_localfile', filename)

conforms filename to local-storage format (e.g. /CONNSERVER/data -> /data)

>> conn_server('util_isremotevar', var)

returns 1 if variable var is a link to a remote variable


MRI FILE UTILITIES

(note: use /CONNSERVER/* format to refer to files in server)

>> conn_display(filepath)

displays SPM or CONN analysis results

>> conn_slice_display(filename, ...)

slice-display view of NIFTI file filename

>> conn_mesh_display(filename, ...)

3D-display view of NIFTI file filename

>> conn_mtx_display(filename, ...)

matrix-display view of NIFTI file filename

>> data = conn_vol_read(filename)

reads volume NIFTI file filename

>> conn_vol_write(filename, data)

writes volume NIFTI file filename

>> data = conn_surf_read(filename)

reads surface NIFTI file filename

>> conn_surf_write(filename, data)

writes surface NIFTI file filename

>> data = conn_mtx_read(filename)

reads matrix NIFTI file filename

>> conn_mtx_write(filename, data)

writes matrix NIFTI file filename

Matlab programming tips

Tips when adding remote-storage functionality to an existing function that is designed to work with local files only:


Tip #1:

Make the function run remotely if it needs to work with remote files

(this works best when the function call requires minimal transfer of information)


Original function syntax:

function fileout = myfunction(filein, varargin)

...

... % the code here works with files locally/normally

...

end

New function syntax:

function fileout = myfunction(filein, varargin)

% if the input file is remote, run this same function in server

if any(conn_server('util_isremotefile',filein)),

fileout = conn_server('util_remotefile',conn_server('run',mfilename,conn_server('util_localfile',filein),varargin{:}));

return

end

% otherwise continue normally


...

... % the code here works with files locally/normally

...


end

Tip #2:

Make the function bring/pull remote files to work with them locally, and update those remote files if necessary at the end

(this works best when the function needs to combine/mix remote and local files)


Original function syntax:

function varargout = fcn(filein, varargin)

...

... % the code here works with files locally/normally

...

end

New function syntax:

function varargout = fcn(filein, varargin)

% create local cache/copy of remote file if necessary

isremotefile=conn_server('util_isremotefile',filein);

if isremotefile,

remotefilename=filein;

filein=conn_cache('pull',remotefilename); % note: change 'pull' to 'new' if the remote file does not exist yet

end


...

... % the code here works with files locally/normally

...


% update remote file with any changes if necessary

if isremotefile, conn_cache('push',remotefilename); end

end

Tip #3:

If a function manages files using functions that follow template #1-3 conventions then it does not need any additional changes to work with remote-storage files

Tips when adding remote-execution functionality to an existing function that works with large variables:


Tip #4:

Make the function run remotely if it needs to work with remote variables

(this works best when the function is an intermediate function in a larger pipeline of functions)


Original function syntax:

function output = fcn(input, varargin)

...

... % the code here works with "input" and "output" variables locally/normally

...

end

New function syntax:

function output = fcn(input, varargin)

% if the input variable is a link to a remote variable, run this same function in server and keep output in server as well

if conn_server('util_isremotevar',input),

output=conn_server('run_keep',mfilename,input,varargin{:});

return

end

% otherwise continue normally


...

... % the code here works with "input" and "output" variables locally/normally

...


end