The tutorial assumes you have already worked through the Execute a Job Tutorial. Therefore, the instructions here are abbreviated but will follow the same format so you may easily consult the extended tutorial.
Table of Contents
📝 Note: Do not execute jobs on the login nodes; only use the login nodes to access your compute nodes. Processor-intensive, memory-intensive, or otherwise disruptive processes running on login nodes will be killed without warning.
Open a Bash terminal (or MobaXterm for Windows users).
ssh [email protected].
When prompted, enter your password.
Here is an example sbatch script for running a batch job on an HPC like Onyx.
#!/bin/bash#SBATCH -n 16#SBATCH -o test_%A.out#SBATCH --error test_%A.err#SBATCH --mail-user $CHANGE_TO_YOUR_EMAIL#SBATCH --mail-type ALLmodule purgemodule load gnu/5.4.0module load spack-appsmodule load openmpi-3.0.0-gcc-5.4.0-clbdgmfmodule load python-3.6.3-gcc-5.4.0-ctlzpuvmodule load py-mpi4py-3.0.0-gcc-5.4.0-wh6rtv7module listmpirun python hello_world.py
Use nano or Vim (we use Vim here) to create and edit your sbatch script.
Create your sbatch script within Vim by typing
insert mode or paste the contents of your sbatch script into Vim.
Save your file by typing
:wq! and return to the Bash shell.
#!/usr/bin/env pythonimport sysfrom mpi4py import MPIsize = MPI.COMM_WORLD.Get_size()rank = MPI.COMM_WORLD.Get_rank()name = MPI.Get_processor_name()print("Hello, World! I am process ",rank," of ",size," on ",name)
Use Vim (
vim) to create your python source file within your working directory.
Paste the hello world python code into Vi.
Save your file and return to the Bash shell.
Python does not need to be compiled.
Before proceeding, ensure that you are still in your working directory (using
pwd) and that you still have the openmpi module loaded (using
We need to be in the same path/directory as our sbatch script and our python script. Use
ls -al to confirm their presence.
sbatch to schedule your batch job in the queue.
This command will automatically queue your job using slurm and produce a job number. You can check the status of your job at any time with the
squeue --job <jobnumber>
You can also stop your job at any time with the
scancel --job <jobnumber>
View your results.
You can view the contents of these files using the
less command followed by the file name.
Your output should look something like this (the output is truncated.):
Hello, World! I am process 11 of 20 on compute_nodeHello, World! I am process 13 of 20 on compute_nodeHello, World! I am process 12 of 20 on compute_nodeHello, World! I am process 0 of 20 on compute_nodeHello, World! I am process 15 of 20 on compute_nodeHello, World! I am process 19 of 20 on compute_nodeHello, World! I am process 7 of 20 on compute_nodeHello, World! I am process 16 of 20 on compute_nodeHello, World! I am process 3 of 20 on compute_nodeHello, World! I am process 9 of 20 on compute_node...
Download your results (using the
scp command or an SFTP client) or move them to persistent storage. See our moving data section for help.