CONN remotely help

  Using CONN remotely   conn_remotely allows you to work and interact with CONN projects stored in any SSH-accessible  computer (e.g. projects stored in your own server, or in your institution's cluster-computing environment)  using the standard SSH protocol for tunneling all communications between your local computer and  the computer where the CONN project data is stored and analyzed
  To use it simply select in the CONN gui the menu 'Project. Connect to remote projects' and log-in to  your institution cluster-computing environment (or your personal server). Once connected CONN will be   able to work with projects and datasets stored in the server computer/network, and, if that server computer  is part of a larger cluster-computing environment, CONN will also be able to use that larger environment to  run preprocessing and analysis steps in parallel   Advanced Configuration/Installation information:    To use CONN remotely you only need a Linux computer (referred to as the 'host') that     a) it is located in a network-accessible location (e.g. fixed IP address)     b) it has an OpenSSH-compatible SSH server/daemon installed and running    c) it has Matlab/SPM/CONN (or standalone CONN) installed   You also need a second Linux/Mac/Windows computer (referred to as the 'client') that    a) it has regular network/internet access    b) it has an OpenSSH-compatible SSH client installed (e.g. this is already installed by default as        part of the operating system in most Linux/Mac/Windows computers)   Installation in the 'host' computer requires the standard installation of SPM/CONN, and also running  just once as part of the installation procedure the Matlab command "conn_remotely setup" (without quotes),  which will create a file in your home directory [~/connserverinfo.json] describing the location of  Matlab, CONN, and SPM packages (note: this command needs to be run for any user that wishes to access this  computer remotely; or alternatively a fixed ~/connserverinfo.json file may be manually copied to the home  directory of each user).   Installation in the 'client' computer simply requires the standard installation of SPM/CONN.   If the 'host' computer is part of a HPC or cluster environment, see     www.conn-toolbox.org/resources/cluster-configuration  for additional HPC/cluster installation instructions. While optional, setting up CONN's cluster/HPC  configuration in your 'host' computer allows conn_remotely to also submit jobs to your cluster remotely  (jobs submitted from the 'client' computer which will be run in the local network of the 'host' computer).    Advanced Usage information:    Manually start CONN from the 'client' computer using the Matlab command:      conn remotely   When prompted enter:     "Server address" prompt : Enter the IP-address of the 'host' computer (this may be the target computer                                    you want to connect to, or simply a login-node within a target network you                                    want to connect to (e.g. when connecting to a computer cluster environment)    "Username" prompt : Enter your username in the 'host' computer (used to establish an SSH-connection                                    with the host)    "Password" prompt : Enter your password in the 'host' computer (note: CONN does not read nor store                                    your password, this will be read by your system's ssh program directly from tty)   After this CONN will launch the standard CONN GUI and allow you to load and work with any CONN projects that may be  accessible from the 'host' computer.

  Additional Options:
     conn remotely setup : (run in 'host' computer just once to configure this host to accept incoming requests)                                 This step simply creates a file ~/connserverinfo.json with the following information:                                  CONNcmd : OS-level command used to start/run CONN on 'host' computer                                  host : name of 'host' computer (or login-node for HPC/cluster environments)                                  SERVERcmd : name of HPC profile to use when starting a CONN 'server'                                              see "conn_jobmanager profiles" to display a list of available profiles                                              see "help conn_jobmanager" for additional options                                     (e.g. SERVERcmd='Background' will run CONN server process in the same computer                                     as the 'server' computer that we are connecting to, while SERVERcmd='Slurm' will                                     run CONN server process in a different computer which will be requested using                                     the options defined in the HPC.configuration 'Slurm' profile)                                  SERVERpersistent : 1/0 to restart server after connection with client ends                                 When conn_remotely connects to a host it will try to first download this file from                                 ~/connserverinfo.json for information on how to proceed      conn remotely start <IP> : Same as 'conn remotely' but specifying the host IP address.                                 This step starts CONN from the 'client' computer (this will attempt first to create a                                  new connection to the same CONN server used in our last connection to this host, and                                  if that fails it will ask the host computer to start a new CONN server and then it                                  will connect to it)      conn remotely restart : re-starts CONN GUI after a dropped connection (this will not start a new CONN server                                 but it will only attempt to create a new connection to the CONN server used in our                                 last connection to this host)      conn isremotely : re-starts CONN GUI (this will not start a new CONN server and it will not create a                                 new connection)      conn remotely start local : Manually start conn client without SSH communications framework (see information                                 below in "Alternative (non-SSH) structure)" section)      conn remotely start server : Manually start conn server without SSH communications framework (see information                                  below in "Alternative (non-SSH) structure)" section)      conn remotely settings : Launches GUI for defining default client/server communication options

  Additional information:    When conn_remotely starts, the 'client' computer will attempt to connect using ssh to the 'host' computer and it will  establish a secure tunnel for communications. The host computer will then use its SERVERcmd profile in HPC/cluster  configuration settings to launch a new Matlab session in a 'server' computer, which may be the same as the 'host'  computer or a different computer within the same network (e.g. if using default HPC settings this session will be  simply run as a new background Matlab process in the server computer; if using non-default HPC setting this session  will be submitted to your cluster scheduler and run in a work-node as appropriate different from the login-node)  which will in turn wait for the client's connection. Once the connection is established, the 'host' computer will  only act as a gateway for communications between the 'client' and the 'server' computers. The 'server' computer  will wait for instructions from the 'client', while the 'client' runs CONN normally (typically for anything that  relates to GUI-interaction), querying the 'server' when needed (e.g. when needing to load data from remote files),  or requesting the 'server' to run longer or more complex steps when appropriate (e.g. when preprocessing the data  or running any analysis steps that requires significant interaction with the data). When the CONN GUI is closed in  the 'client' computer, the 'server' will exit and the SSH tunnel will be closed.     Standard (SSH-based) structure of CONN REMOTELY components:                                  ----------------------------------------------------       -------- | -------- --------- |      | Client | INIT: -----------> | Host | INIT: -----------> | Server | |      |________| <-- TCP/SSH -----> |________| <-- TCP/SSH -----> |_________| |   (local CONN session) | (gateway) (remote CONN session) |                                 |____________________________________________________|                                              (remote network)   When working with a remote project, all of the elements in CONN's gui will be run locally by the 'client' computer  and therefore will be as responsive as usual, but some steps in the GUI do require largers amount of data being  transferred from the 'server' (e.g. when switching to a new tab, or when launching a 3D-view renderer) so these  may take longer than usual to get started while that data is being transferred, please be patient. Also when  working with a remote project and running preprocessing or analysis steps, you will be offered the option to have  those steps run on the server Matlab session directly, or have the remote session use instead its own HPC/Cluster  configuration options to submit a job to its HPC/Cluster environment (to clarify: there is no option to run those  processes locally on your 'client' computer since that would require large amounts of data being transfered from  the 'server' where the data is stored to the 'client' where the analysis would be run, which would make that option  impractical in most scenarios).   If the 'host' computer is NOT SSH-accessible (e.g. another computer in your home/office network), the  following workaround allows you to still use conn_remotely functionality while setting up the server and the  connection manually. Note that this will skip SSH entirely, so in this case your communication data will not be  tunneled nor encrypted. This procedure is only recommended for communications WITHIN a local/secured network:     step 1) in the 'host/server' computer, run the Matlab command below to launch a CONN server process:                 conn_remotely start server;             and then enter when prompted the desired one-time-use password and TCP-port number (leave empty to             use the automatically search for an available port)     step 2) in the 'client' computer, close any active CONN windows and run the command:                 conn_remotely start local;             and then enter when prompted the host/server IP address, port number, and password as defined above     Alternative (non-SSH) structure of CONN REMOTELY components:       -------------------------------------------------------      | -------- --------------- |      | | Client | <------TCP------> | Host/Server | |      | |________| |_______________| |      |(local CONN session) (remote CONN session) |      |_______________________________________________________|                          (local network)     Advanced functions for debugging and command-line use:    CONNECTION   conn_remotely on            : starts remote CONN server and connect to it (equivalent to running "conn remotely" but                                without launching CONN GUI)  conn_remotely off           : disconnects from remote CONN server (equivalent to closing GUI when using "conn remotely"                                syntax to start CONN)  conn_remotely offandon      : restarts communication with remote CONN server after dropped connection (equivalent                                to running "conn remotely restart" but without launching CONN GUI)   REMOTE EXECUTION   conn_remotely cmd               : interactive command-line execution in remote CONN server  [...]=conn_remotely('cmd',...) : single-command execution in remote CONN server                                     e.g. y=conn_remotely('cmd','fft',1:4);                                     e.g. conn_remotely('cmd','run','/usr/me/mybatch01.m')   REMOTE FILE COPY   conn_remotely('push',localfile,remotefolder) : copies local file 'localfile' into remote folder 'remotefolder'                                                                e.g. conn_remotely push /data/file1.nii /usr/me/data/  conn_remotely('folderpush',localfolder,remotefolder) : copies the local folder 'localfolder' (and all of its contents)                                                                into remote folder 'remotefolder'                                                                e.g. conn_remotely push /data /usr/me  conn_remotely('pull',remotefile,localfolder) : copies remote file 'localfile' into local folder 'localfolder'                                                                e.g. conn_remotely pull /usr/me/data/file1.nii /data/  conn_remotely('folderpull',remotefolder,localfolder) : copies the remote folder 'remotefolder' (and all of its contents)                                                                into local folder 'localfolder'                                                                e.g. conn_remotely pull /usr/me/data /  conn_remotely('push',localfile,remotefile) : copies local file 'localfile' to remote storage and rename it                                                                as 'remotefile'                                                                e.g. conn_remotely push /data/file1.nii /usr/me/data/file1.nii  conn_remotely('pull',remotefile,localfile) : copies remote file 'remotefile' to local storage and rename it                                                                as 'localfile'                                                                e.g. conn_remotely pull /usr/me/data/file1.nii /data/file1.nii   LOCAL EXECUTION   When connected to a remote CONN server and running locally any conn_* function, most CONN functions that take  files or directories as input arguments will accept filepaths of the form: '/CONNSERVER/[filepath]' to refer  to a file/folder [filepath] in the remote server storage. For example:   conn_loadmatfile /CONNSERVER/usr/me/file1.mat : loads the variables in the remote file1.mat file into                                                              your (local) workspace  conn_mesh_display /CONNSERVER/usr/me/data/file1.nii : displays (locally) the remote file file1.nii  conn_cache pull /CONNSERVER/usr/me/data/file1.nii : creates a local-cache copy of the remote file file1.nii  conn_display /CONNSERVER/usr/me/results/SPM.mat : displays (locally) the remote 2nd-level results  conn_fileutils mkdir /CONNSERVER/usr/me/newfolder : creates a new remote directory (equivalent to: conn_remotely                                                              cmd mkdir /usr/me/newfolder)  data=conn_surf_read('/CONNSERVER/usr/me/file1.nii'); : reads the 2D-surface nifti remote file file1.nii  data=conn_mtx_read('/CONNSERVER/usr/me/file1.nii'); : reads the 2D-matrix nifti remote file file1.nii  data=conn_fileutils('spm_read_vols','/CONNSERVER/usr/me/file1.nii'); : will read the 3D/4D nifti remote file file1.nii   When connected to a remote CONN server and running locally CONN_HPC functions, CONN will always use the remote-server  HPC options to submit jobs instead of the local HPC options. For example:   conn('submit','disp','hello world') : will have the remote-server use its own HPC options                                                              (e.g. slurm profile) to submit a job to the remote network                                                              where the function disp('hello world') will be executed  conn('submit','run','/usr/me/mybatch01.m') : will have the remote-server use its own HPC options                                                              (e.g. slurm profile) to submit a job to the remote network where                                                              the script mybatch01 will be executed  conn_jobmanager('getprofile') : lists the remote-server default HPC profile (just the                                                                same as str=conn_remotely('cmd','conn_jobmanager','getprofile'))

Page updated

Google Sites

Report abuse