doc(hpc): resource specifiers
This commit is contained in:
parent
24ae7a2399
commit
3db714475f
3 changed files with 32 additions and 10 deletions
30
docs/HPC.md
30
docs/HPC.md
|
|
@ -13,13 +13,37 @@ Run **once** after cloning the repository. This script handles all modules, mirr
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
# Option A: Interactive (on a compute node)
|
# Option A: Interactive (on a compute node)
|
||||||
module swap cluster/donphan # or joltik
|
module swap cluster/donphan # Debug cluster (CPU only)
|
||||||
qsub -I -l nodes=1:gpus=1
|
# OR for GPU clusters:
|
||||||
|
# module swap cluster/joltik
|
||||||
|
# module swap cluster/accelgor
|
||||||
|
# module swap cluster/litleo
|
||||||
|
|
||||||
|
qsub -I -l nodes=1:gpus=1 # Only for GPU clusters
|
||||||
cd "${PBS_O_WORKDIR}"
|
cd "${PBS_O_WORKDIR}"
|
||||||
bash scripts/hpc/install.sh
|
bash scripts/hpc/install.sh
|
||||||
|
|
||||||
# Option B: Batch (Run in background)
|
# Option B: Batch (Run in background)
|
||||||
qsub scripts/hpc/install.sh
|
# NOTE: GPU clusters (joltik/accelgor/litleo) require -l gpus=1 at runtime
|
||||||
|
qsub -l gpus=1 scripts/hpc/install.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Production vs. Debug Clusters
|
||||||
|
|
||||||
|
Our scripts are cluster-agnostic and do **not** have hardcoded GPU requirements. Instead, you must request GPUs at runtime using the `-l gpus=1` flag when submitting to a production GPU cluster.
|
||||||
|
|
||||||
|
### Debugging (Donphan)
|
||||||
|
The `donphan` cluster does not support GPUs. Simply run the scripts without extra resource flags:
|
||||||
|
```bash
|
||||||
|
module swap cluster/donphan
|
||||||
|
qsub scripts/hpc/train.pbs
|
||||||
|
```
|
||||||
|
|
||||||
|
### Production (Joltik, Accelgor, Litleo)
|
||||||
|
These clusters provide GPU acceleration and **require** a GPU request at runtime:
|
||||||
|
```bash
|
||||||
|
module swap cluster/joltik # or accelgor/litleo
|
||||||
|
qsub -l gpus=1 scripts/hpc/train.pbs
|
||||||
```
|
```
|
||||||
|
|
||||||
## Interactive Debugging
|
## Interactive Debugging
|
||||||
|
|
|
||||||
|
|
@ -8,7 +8,7 @@
|
||||||
# qsub scripts/hpc/install.sh
|
# qsub scripts/hpc/install.sh
|
||||||
|
|
||||||
#PBS -N brittlestar-install
|
#PBS -N brittlestar-install
|
||||||
#PBS -l walltime=01:00:00
|
#PBS -l walltime=00:15:00
|
||||||
|
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,13 +1,11 @@
|
||||||
#!/bin/bash
|
# Production training (requires GPU at runtime):
|
||||||
# scripts/hpc/train.pbs
|
# qsub -l gpus=1 scripts/hpc/train.pbs
|
||||||
#
|
# Debug/CPU training:
|
||||||
# Usage (from project root):
|
|
||||||
# qsub scripts/hpc/train.pbs
|
# qsub scripts/hpc/train.pbs
|
||||||
|
|
||||||
#PBS -N brittlestar-ppo
|
#PBS -N brittlestar-ppo
|
||||||
#PBS -l nodes=1:ppn=8:gpus=1
|
#PBS -l nodes=1:ppn=8
|
||||||
#PBS -l walltime=24:00:00
|
#PBS -l walltime=24:00:00
|
||||||
#PBS -l mem=32gb
|
|
||||||
#PBS -o runs/brittlestar-ppo.o$PBS_JOBID
|
#PBS -o runs/brittlestar-ppo.o$PBS_JOBID
|
||||||
#PBS -e runs/brittlestar-ppo.e$PBS_JOBID
|
#PBS -e runs/brittlestar-ppo.e$PBS_JOBID
|
||||||
|
|
||||||
|
|
|
||||||
Reference in a new issue