How can I use more than one node or core on Argo?

Important note: If your program or software is not parallel or distributive, specifying more than one node and/or more than one processor does absolutely nothing other than waste resources that could be used by some other client. 

How do I know if my program or software is parallel or distributive? If you've written your own program using a programming language like C or Fortran (or any other language) and you haven't included a paradigm such as MPI, your program is neither parallel nor distributive. If you are not familiar with the terms MPI, OpenMPI, MPICH2, or LAM or if you don't know what parallel or distributive mean, then your program is not parallel/distributive. If your software doesn't specifically mention that it parallel/distributive, it is not. Gaussian is an example of a parallel/distributive package; GaussView is not. If your program is not parallel/distributive, then it is SERIAL. That means it execute on a single processor on a single node.

Serial jobs

If the software or program is serial, do not include the nodes and ppn components on the qsub command. Including them will not make your program run faster.

The number of nodes you request to run your job is specified, like the destination, either on the qsub line or in the script (the number of requested nodes is a resource) If you are running a serial job (a serial job uses one node and one core on it), which is the default, you don't have to request one node. The following request for a single node is unnecessary:

qsub -V -l nodes=1 -q staff my_script

It's the same as doing:

qsub -V -q staff my_script

Parallel jobs

If you will run a parallel job using multiple nodes, then you must request (hardcode) the number nodes you want allocated. Put the number of nodes on the qsub line: 

qsub -V -l nodes=4 -q staff my_script 

or put the number of nodes in the script file:

#PBS -l nodes=4

and submit the script without the number of nodes on the qsub line:

qsub -V -q staff my_script

In both cases, the user is requesting four nodes.

Multiple cores

You want multiple cores. Cores are indicated by the ppn operand that follows the request for the number of nodes. Same as before: you may use one of the two formats to make the request. For the purposes of brevity, I will show only first method; you should be able to construct the second method from previous examples.

Use three nodes but only one core per node:

qsub -V -l nodes=3:ppn=1 -q staff my_script

Use three nodes and two core per node:

qsub -V -l nodes=3:ppn=2 -q staff my_script

Use one node and two cores on it:

qsub -V -l nodes=1:ppn=2 -q staff my_script

As stated before, one core is the default. If you don't specify the number of cores, then only one is assigned. The following two command invocations do the same thing; both request one core (the first includes the request; the second, lets the system default to one core):

qsub -V -l nodes=1:ppn=1 -q staff my_script

is the same as

qsub -V -l nodes=1 -q staff my_script

The requested cores will be on the same machine. If you request one machine with four cores, the system will look for a single machine having four cores. If it fails to find a matching machine, it will not attempt to allocate two machines and two cores on each of the two machines. Instead, your job does not run (queues) because the system will be unable to satisfy your request. The resources (not just cores but other requested resources including but not limited to nodes) may be owned by another job and, when that job completes and the resource(s) become available, they will be assigned to your job.

However, if you request resources that will never be available (I want eight cores on one node is an example of a nonexistent resource), your job will never run.

Need help?

Last updated: 

August 29, 2012

Browse by tag