Running AMUSE scripts on the PARA cluster

AMUSE can be installed and run on the PARA cluster at the Leiden University. All prerequisite software is already installed on the cluster, so you only need to download and build AMUSE (see  Install Amuse)

> svn co http://www.amusecode.org/svn/trunk amuse-svn
> cd amuse-svn
> ./configure
> make

Next you need to create a shell script to start your python script (see  Torque implementation at Leiden Observatory for a complete description of all options)

#!/bin/tcsh -f
#PBS -l nodes=1:ppn=4
#
setenv AMUSEDIR path/to/amuse

cd path/to/the/python/script

mpiexec -n 1 ${AMUSEDIR}/amuse.sh your_script.py

You need to specify -n 1 to mpiexec. This will start one version of the python script. In your script you can then start the codes (which will run on different nodes of the para cluster).

The number of nodes times the number of processors per node reserved for your script (with PBS -l nodes=1:ppn=4) must be the number of instances of the codes you will start in the script plus one for the script itself. So if you're script contains this line:

code = Athena(number_of_workers=4)

You need to reserve 5 or more processors (nodes=1:ppn=5) or (nodes=2:ppn=3).

WATCH OUT: Amuse by default will start a code using a redirection script and this script needs arguments. Unfortunately, the TORQUE/MPICH2 combination on the para cluster seems to have an error in spawning codes with arguments. To fix, create a amuserc file in your working directory (same directory as the cd earlier), and set this content.

[channel]
redirection=none

After you have written you're start script you can submit your script with (we assume, you've named the start script start_script.sh)

> qsub start_script.sh