> For the complete documentation index, see [llms.txt](https://doane-ccla.gitbook.io/docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://doane-ccla.gitbook.io/docs/using-the-hpc/execute-a-job/python.md). # Working with Python The tutorial assumes you have already worked through the [Execute a Job Tutorial](/docs/using-the-hpc/execute-a-job.md). Therefore, the instructions here are abbreviated but will follow the same format so you may easily consult the extended tutorial. **Table of Contents** * [Step 1: Access the Onyx HPC](/docs/using-the-hpc/execute-a-job/python.md#step-1-access-the-onyx-hpc) * [Step 2: Create an sbatch Script](/docs/using-the-hpc/execute-a-job/python.md#step-2-create-a-sbatch-script) * [Example sbatch Script](/docs/using-the-hpc/execute-a-job/python.md#example-sbatch-script) * [sbatch Procedure](/docs/using-the-hpc/execute-a-job/python.md#sbatch-procedure) * [Step 3: Create a Python Program](/docs/using-the-hpc/execute-a-job/python.md#step-3-create-a-python-program) * [MPI Hello World Code](/docs/using-the-hpc/execute-a-job/python.md#mpi-hello-world-code) * [Python Procedure](/docs/using-the-hpc/execute-a-job/python.md#python-procedure) * [Step 4: Run the Job](/docs/using-the-hpc/execute-a-job/python.md#step-4-run-the-job) 📝 **Note:** Do not execute jobs on the login nodes; only use the login nodes to access your compute nodes. Processor-intensive, memory-intensive, or otherwise disruptive processes running on login nodes will be killed without warning. ## Step 1: Access the Onyx HPC 1. Open a Bash terminal (or MobaXterm for Windows users). 2. Execute `ssh doaneusername@onyx.doane.edu`. 3. When prompted, enter your password. ## Step 2: Create an sbatch Script ### Example sbatch Script Here is an example sbatch script for running a batch job on an HPC like Onyx. ```bash #!/bin/bash #SBATCH -n 16 #SBATCH -o test_%A.out #SBATCH --error test_%A.err #SBATCH --mail-user $CHANGE_TO_YOUR_EMAIL #SBATCH --mail-type ALL module purge module load gnu/5.4.0 module load spack-apps module load openmpi-3.0.0-gcc-5.4.0-clbdgmf module load python-3.6.3-gcc-5.4.0-ctlzpuv module load py-mpi4py-3.0.0-gcc-5.4.0-wh6rtv7 module list mpirun python hello_world.py ``` ### sbatch Procedure 1. Use nano or Vim (we use Vim here) to create and edit your sbatch script. ```bash vim slurm_py_example.job ``` 2. Create your sbatch script within Vim by typing `i` for `insert` mode or paste the contents of your sbatch script into Vim. 3. Save your file by typing `:wq!` and return to the Bash shell. ## Step 3: Create a Python Program ### MPI Hello World Code ```python #!/usr/bin/env python import sys from mpi4py import MPI size = MPI.COMM_WORLD.Get_size() rank = MPI.COMM_WORLD.Get_rank() name = MPI.Get_processor_name() print("Hello, World! I am process ",rank," of ",size," on ",name) ``` ### Python Procedure 1. Use Vim (`vim`) to create your python source file within your working directory. ```bash vim hello_world.py ``` 2. Paste the hello world python code into Vi. 3. Save your file and return to the Bash shell. *Python does not need to be compiled.* ## Step 4: Run the Job 1. Before proceeding, ensure that you are still in your working directory (using `pwd`) and that you still have the openmpi module loaded (using `module list`). * We need to be in the same path/directory as our sbatch script and our python script. Use `ls -al` to confirm their presence. 2. Use `sbatch` to schedule your batch job in the queue. ```bash sbatch slurm_py_example.job ``` This command will automatically queue your job using slurm and produce a job number. You can check the status of your job at any time with the `squeue` command. ```bash squeue --job ``` You can also stop your job at any time with the `scancel` command. ```bash scancel --job ``` 3. View your results.\ You can view the contents of these files using the `less` command followed by the file name.
```bash less test_.out ``` Your output should look something like this (*the output is truncated.*): ```bash Hello, World! I am process 11 of 20 on compute_node Hello, World! I am process 13 of 20 on compute_node Hello, World! I am process 12 of 20 on compute_node Hello, World! I am process 0 of 20 on compute_node Hello, World! I am process 15 of 20 on compute_node Hello, World! I am process 19 of 20 on compute_node Hello, World! I am process 7 of 20 on compute_node Hello, World! I am process 16 of 20 on compute_node Hello, World! I am process 3 of 20 on compute_node Hello, World! I am process 9 of 20 on compute_node . . . ``` 4. Download your results (using the `scp` command or an SFTP client) or move them to persistent storage. See our [moving data](https://github.com/Doane-CCLA/docs/tree/87eef1ceec793702788e253029927a3fd4c17519/data-transfer-storage/moving-data.md) section for help. #### Additional Examples * [Working with C](/docs/using-the-hpc/execute-a-job.md) * [Working with C++](/docs/using-the-hpc/execute-a-job/cpp.md) * [Working with Fortran](/docs/using-the-hpc/execute-a-job/fortran.md) * [Working with Makefiles](/docs/using-the-hpc/execute-a-job/makefile.md) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://doane-ccla.gitbook.io/docs/using-the-hpc/execute-a-job/python.md?ask= ``` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.