MATLAB and MDCS

Using MATLAB and MDCS

  1. Start MATLAB with graphical interface. Remember that you need to either:
    1. login with X11 forwarding enabled (or see guide for connecting from Windows)
      ssh -X username@abisko.hpc2n.umu.se
    2. login using thinlinc.
  2. Then load the module
    module load matlab

    and start MATLAB

    matlab &
  3. To setup work-path and do some configuration before using MDCS, do:
    configCluster 
  4. In order to set the project that you will be running your jobs in, set:
    ClusterInfo.setProjectName('SNICXXXX-YY-ZZ')

    NOTE change to your project.

  5. Set the walltime to some suitable time. This example sets it to 1 hour:
    ClusterInfo.setWallTime('01:00:00')
  6. Other useful things can be set the same way, for example:
    • Memory (in MB):
      ClusterInfo.setMemUsage('<option>')
    • Asking for exclusive use of the node (not sharing):
      ClusterInfo.setRequireExclusiveNode(true)
    • Setting the email address to get info about the job:
      ClusterInfo.setEmailAddress('<your@email.com>')
    • Setting the number of processors per node:
      ClusterInfo.setProcsPerNode('#numprocsnode')
    • etc.
  7. A value can be cleared by assigning it an empty value ('',[],false) or by calling the clear mething to clear all values:
    ClusterInfo.clear

Interactive jobs

  • Getting a MATLAB pool with #workers (the number of workers you want to use):
    p=parpool(#workers)

    Example, getting a parallel pool of 11 workers, which you can use in your parallel programs (11 workers + 1 master, meaning 12 tasks total - usually meaning 12 CPU cores):

    p=parpool(11)
  • You can now run parallel jobs which will use those workers. Example:
    my_parallel_example(11)

Serial batch jobs

This is for running serial MATLAB jobs on the cluster, submitted to the batch system from inside MATLAB.

  • Define a cluster object, c:
    c=parcluster;
  • Submit a job, j with:
    j=c.batch(@myfcn, N, {x1,x2,x3,....});

    where 'myfcn' is a command or serial MATLAB program (myfcn.m), N is number of output arguments from the evaluated function, and x1, x2, x3,... are input arguments to your program.

  • Wait for the job to finish:
    j.wait;
  • Get the results when the job has finished:
    j.fetchOutputs{:}
  • You can delete the job with:
    j.delete;
  • Retrieve and view list of jobs:
    jobs=c.Jobs;

Parallel batch jobs

This is for running parallel MATLAB jobs on the cluster, submitted to the batch system from inside MATLAB.

  • Define a cluster object, c:
    c=parcluster;
  • Submit a job, j to the batch system on the cluster defined in c with:
    j=c.batch('myscript', N, {x1,x2,x3,....},'pool',#workers);

    where 'myscript' is a command or MATLAB program (your myscript.m but do not include .m), N is number of output arguments from the evaluated function, x1, x2, x3,... are input arguments to your program, 'pool' is the pool you specify (needed for parallle jobs), and #workers is the number of workers you want. Example:

    j=c.batch('myscript', 1, {1000}, 'pool', 11);

    would run 'myscript.m' with the input argument "1000" in the pool called "pool". You will get 11 workers + 1 master, for a total of 12 tasks.

  • Wait for the job to finish:
    j.wait;
  • Get the results when the job has finished:
    j.fetchOutputs{:};
  • You can delete the job with:
    j.delete;
  • Retrieve and view list of jobs:
    jobs=c.Jobs;
  • You can get the job ID with:
    id=j.ID

    this is useful for retrieving the results after quitting MATLAB. You can find a job again with:

    j=c.findJob('ID', <id-number>);
  • If you need to debug, you can retrieve the output/error log file with:
    j.Parent.getDebugLog(j)

There is more information about MATLAB batch jobs here: http://se.mathworks.com/help/distcomp/batch.html

Using batch submission from command line

  1. Start thinlinc as above
  2. Open a terminal: Click "Applications" -> "System Tools" -> MATE terminal
  3. Type:
    cd pfs

    to change to your pfs directory. If you don't have a pfs link in your home directory, create one:

    ln -s /pfs/nobackup$HOME $HOME/pfs
  4. Go to the directory you have your files in
  5. Create script.
  6. Submit with:
    sbatch <script>

    as usual

There is an example for parallel batch jobs here: https://www.hpc2n.umu.se/resources/software/matlab

Updated: 2017-04-20, 16:56