2026SEL3-project-Brittle_Star_Controller_Modularity/docs/HPC.md

HPC Guide

Full documentation: https://docs.hpc.ugent.be/

Cluster Selection

Choose the appropriate cluster before submitting a job with module swap cluster/<name>. The default login cluster is doduo.

Check the current queue load at https://shieldon.ugent.be:8083/pbsmon-web-users/.

Initial Environment Setup

Run once from a login node after cloning the repository:

# 1. Connect via the web portal → HPC Login → Interactive Apps > Shell (tmux)
#    Set cluster to "donphan (interactive/debug)"

# 2. Clone the project (if not done yet)
git clone <repo-url>
cd 2026SEL3-project-BrittleStar

# 3. Run the install script
bash scripts/hpc/install.sh

Note: virtual environments are cluster-specific. Re-run the script if you switch to a new cluster.

Interactive Debugging on donphan

The donphan cluster provides quick access to a fractional A100 GPU and is ideal for verifying your environment end-to-end before submitting batch jobs.

Option A — Interactive shell session

# Swap to the debug cluster (from the login node)
module swap cluster/donphan

# Request an interactive job (1 node, 4 cores)
qsub -I -l nodes=1:ppn=4 -l walltime=1:00:00

# Once inside the job, activate your environment
cd "$PBS_O_WORKDIR"   # or cd to your project root if not set
module load vsc-venv
source vsc-venv --activate --modules env/hpc/modules.txt

# Run the smoke test
export MUJOCO_GL=egl
python src/train.py --config configs/hpc_smoke_test.yaml

Option B — JupyterLab session (HPC web portal)

1. In the web portal go to Interactive Apps → JupyterLab RHEL9

2. Set the following options:

   Option	Value
   Cluster	donphan (interactive/debug)
   Number of nodes	1
   Number of cores	4
   JupyterLab version	4.2.5 GCCcore-13.3.0
   Custom code	(leave blank — vsc-venv handles modules)

3. Click Launch, wait for the session to start, then Connect.

4. In JupyterLab, select the kernel SEL3 (<cluster>).

5. Verify GPU access:

   import jax
   print(jax.default_backend())  # expected: 'gpu'
   print(jax.devices())          # expected: [CudaDevice(id=0)]
   

Warning: JAX can only be loaded by one kernel at a time. Shut down other kernels before switching notebooks.

Submitting Batch Training Jobs

# Default cluster (joltik — one A100 GPU slice)
qsub scripts/hpc/train.pbs

# Switch to a different GPU cluster first
module swap cluster/accelgor
qsub scripts/hpc/train.pbs

Monitor your jobs:

qstat          # list your jobs
qstat -f <id>  # detailed info for a specific job
qdel <id>      # cancel a job

Logs are written to runs/pbs_<jobid>.out and runs/pbs_<jobid>.err.

---

Managing Dependencies

env/hpc/requirements.txt is the source of truth for pip packages on the HPC. It is automatically regenerated by CI whenever pyproject.toml changes. To regenerate locally:

uv run scripts/export_hpc_requirements.py

Do not edit env/hpc/requirements.txt by hand — edit pyproject.toml instead.