1
Fork 0

refactor: comment cleanups

This commit is contained in:
Tibo De Peuter 2026-04-04 16:25:16 +00:00 committed by Tibo De Peuter
parent 953e102483
commit 08d54b73b4
3 changed files with 1 additions and 37 deletions

View file

@ -1,4 +1,4 @@
# HPC Guide (Ghent University Tier-2)
# HPC Guide
Full documentation: <https://docs.hpc.ugent.be/>
@ -6,19 +6,8 @@ Full documentation: <https://docs.hpc.ugent.be/>
Choose the appropriate cluster before submitting a job with `module swap cluster/<name>`. The default login cluster is **doduo**.
| Cluster | Type | Use case |
|-----------|-------------------------|-----------------------------------------------|
| `donphan` | Interactive / debug GPU | First-time setup, interactive debugging |
| `doduo` | CPU (default login) | Rapid iteration, CPU-only smoke tests |
| `joltik` | GPU (A100 ¼-slice) | Standard training runs |
| `accelgor`| GPU (A100 full) | Large-scale / long experiments |
| `litleo` | GPU | Alternative GPU option |
> **Rule:** use at most **1 GPU per group at a time** on shared GPU clusters.
> Check the current queue load at <https://shieldon.ugent.be:8083/pbsmon-web-users/>.
---
## Initial Environment Setup
Run **once** from a login node after cloning the repository:
@ -35,16 +24,8 @@ cd 2026SEL3-project-BrittleStar
bash scripts/hpc/install.sh
```
This uses the official [`vsc-venv`](https://docs.hpc.ugent.be/Linux/setting_up_python_virtual_environments/#vsc-venv-python-virtual-environment-wrapper-script) wrapper to:
- Load the EasyBuild modules listed in `env/hpc/modules.txt` (JAX, Flax, WandB, …)
- Create a per-cluster virtual environment in `$VSC_DATA`
- Pip-install the remaining packages from `env/hpc/requirements.txt`
- Register a Jupyter kernel named `sel3_<cluster>`
> **Note:** virtual environments are cluster-specific. Re-run the script if you switch to a new cluster.
---
## Interactive Debugging on donphan
The `donphan` cluster provides quick access to a fractional A100 GPU and is ideal for verifying your environment end-to-end before submitting batch jobs.
@ -93,8 +74,6 @@ python src/train.py --config configs/hpc_smoke_test.yaml
> **Warning:** JAX can only be loaded by one kernel at a time. Shut down other kernels before switching notebooks.
---
## Submitting Batch Training Jobs
```bash

View file

@ -12,18 +12,12 @@
set -euo pipefail
echo "==> Loading vsc-venv module..."
module load vsc-venv
echo "==> Activating virtual environment (created per-cluster in \$VSC_DATA)..."
# vsc-venv transparently creates and manages a per-cluster venv in $VSC_DATA,
# loading the modules listed in env/hpc/modules.txt and pip-installing anything
# in env/hpc/requirements.txt that is not already satisfied.
source vsc-venv --activate \
--modules env/hpc/modules.txt \
--requirements env/hpc/requirements.txt
echo "==> Registering Jupyter kernel..."
python -m ipykernel install --user --name="sel3_${VSC_INSTITUTE_CLUSTER}" \
--display-name "SEL3 (${VSC_INSTITUTE_CLUSTER})"

View file

@ -6,27 +6,18 @@
#
# To target a specific GPU cluster (default: joltik):
# module swap cluster/accelgor && qsub scripts/hpc/train.pbs
#
# Available GPU clusters: joltik, accelgor, litleo
# Debug / interactive: doduo (CPU) or donphan (interactive GPU)
# ---------------------------------------------------------------------------
# PBS job directives
# ---------------------------------------------------------------------------
#PBS -N brittlestar-ppo
#PBS -l nodes=1:ppn=8:gpus=1
#PBS -l walltime=24:00:00
#PBS -l mem=32gb
#PBS -o runs/pbs_${PBS_JOBID}.out
#PBS -e runs/pbs_${PBS_JOBID}.err
# ---------------------------------------------------------------------------
set -euo pipefail
cd "$PBS_O_WORKDIR"
# Activate the project virtual environment via the official vsc-venv wrapper.
# This loads the modules listed in env/hpc/modules.txt and the per-cluster venv.
module load vsc-venv
source vsc-venv --activate --modules env/hpc/modules.txt