Scripts (Job Submission Files)

Scripts (Job Submission Files)

The official name for batch scripts in SLURM is Job Submission Files, but here we will use both names interchangeably.

A job submission file can contain any of the commands that you would otherwise issue yourself from the command line. It is, for example possible to both compile and run a program and also to set any necessary environment values (though remember that SLURM exports the environment variables in your shell per default, so you can also just set them all there before submitting the job).
The results from compiling or running your programs can generally be seen after the job has completed, though as SLURM will write to the output file during the run, some results will be available quicker.

Outputs and any errors will per default be placed in the directory you are running from. Note that this directory should preferrably be placed under the parallel filesystem, since that is the only filesystem the batch system have access to easily.
Both output and errors will, by default, be combined into a file named slurm-<jobid>.out. You can send them to other/separate files with these commands:

#SBATCH --error=job.%J.err 
#SBATCH --output=job.%J.out

In the example above, you get files named job.<jobid>.err and job.<jobid>.out, you can of course give them other names, but if you are running several batch jobs, remember to name them so they are not over-written by later jobs.

A job submission file can be very simple, with most of the job attributes speciified on the command line, or it may consist of SLURM directives, comments and executable statements. A SLURM directive provides a way of specifying job attributes in addition to the command line options.

Naming: You can name your script anything, including the suffix. It does not matter. Just name it something that makes sense to you and helps you remember what the script is for. The standard is to name it with a suffix of .sbatch.

Note that you have to always include #!/bin/bash at the beginning of the script, since bash is the only supported shell. Some things may work under other shells, but not everything.

Example:

#!/bin/bash
# The name of the account you are running in, mandatory.
#SBATCH -A <account> 
# Asking for two nodes
#SBATCH -N 2
# and two tasks
#SBATCH -n 2
# and two cores per task
#SBATCH -c 2
# Request exclusive access to the node. 
# This means the job cannot share nodes with any other running 
# jobs - it is the opposite of --share 
#SBATCH --exclusive 
# Request 5 minutes of runtime for the job 
#SBATCH --time=00:05:00 
# Set the names for the error and output files 
#SBATCH --error=job.%J.err 
#SBATCH --output=job.%J.out

mpirun ./my_program

One (or more) # in front of a text line means it is a comment. #SBATCH is used to signify a SLURM directive. In order to comment out these, you need to put one more # in front of the #SBATCH.

Note: It is important to use capital letters for #SBATCH. Otherwise the line will be considered a comment, and ignored.

Note that <account> is of the form SNICXXXX-YY-ZZZ. You can find your project account with the command projinfo.

  • -N: number of nodes. If this is not given, enough will be allocated to fullfill the requirements of -n and/or -c. A range can be given. If you ask for, say, 1-1, then you will get 1 and only 1 node, no matter what you ask for otherwise. It will also assure that all the processors will be allocated on the same node.
  • -n: number of tasks.
  • -c: cores per task. Request that a specific number of cores be allocated to each task. This can be useful if the job is multi-threaded and requires more than one core per task for optimal performance. The default is one core per task.

See man sbatch for more commands for the script.

When the job is submitted, the lines of the script file are scanned for directives. An initial line in the script that begins with the characters "#!" will be ignored and scanning will start with the next line. Scanning will continue until the first executable line, that is a line that is not blank, not a directive line, nor a line whose first non-white space character is "#". If directives occur on subsequent lines, they will be ignored. The remainder of the directive line consists of the options to slurm in the same syntax as they appear on the command line. The option character is to be preceded with the "-" character, or "--" in the long form. If an option is present in both a directive and on the command line, that option and its argument, if any, will be ignored in the directive. The command line takes precedence!

If an option is present in a directive and not on the command line, that option and its argument, if any, will be processed as if it had occurred on the command line.

There are several examples of SLURM job submission files here.

Updated: 2017-10-04, 11:18