Multinode Job scripts
LL script file syntax
In order to submit a batch execution, you need to create a file, e.g. job.cmd, which contains
- LL directives specifying how many resources (walltime and processors) you wish to allocate to your job.
- shell commands and programs which you wish to execute. The file paths can be absolute or relative to the directory from which you will submit the job.
Note that if your job tries to use more memory or time than requested, it will be killed by LL. On the contrary, if you require larger resources than needed, you will pay more and/or you will wait longer before your job is taken into consideration. Design your applications in such a way that it can be easily restarted and request the right amount of resources.
The first part of the script file contains directives for LL specifying the resources needed by the job. Some of them are general keyworks.
The following rules dictate the form of your Job Command file:
- Keyword statements begin with # @. There can be any number of blanks between the # and the @.
- Comments begin with a # just as they do for shell scripts.
- Statement components are seperated by blanks.
- The backslash \ is the line continuation character. The continued line must not begin with # @.
- Keywords are case insensitive.
All jobs should be submitted to the compute nodes from the login node (i.e. from the front end node), as batch jobs via IBM's LoadLeveler. We are currently running LoadLeveler version 3.5.1.10 .
Job scheduling policy
General Job limitations (per user)
- maximum running jobs are 40
- maximum number of idle jobs are 5, jobs above this maximum are placed in the NotQueued (NQ) state
Jobs that are rejected
- the CPU account requested has run out of quota
- jobs running non-priority quota when number of idle jobs exceeds 25
- resources (ConsumableMemory and/or ConsumableCpus) are not specified
- the memory requested per node is higher than memory available
- total number of cores per node requested is more than available (16 without SMT specified)
Namd
Using Intel Compilers
