tdpeuter/2026SEL3-project-Brittle_Star_Controller_Modularity
tdpeuter/2026SEL3-project-Brittle_Star_Controller_Modularity
Archived
1
Fork 0
Code Issues Releases Activity
This repository has been archived on 2026-08-15. You can view files and clone it, but you cannot make any changes to it's state, such as pushing and creating new issues, pull requests or comments.
2026SEL3-project-Brittle_St.../docs/HPC.md
Tibo De Peuter 70dd8c3161 fix(hpc): numpy
2026-04-04 22:58:47 +02:00

2.3 KiB
Raw Blame History

HPC Guide

Full documentation: https://docs.hpc.ugent.be/

Storage Overview

  • Run Outputs: Written to $VSC_SCRATCH during the job (fast I/O) and copied to $VSC_DATA at the end for persistence.
  • Virtual Environments: Managed on $VSC_DATA by mirroring configuration files. This avoids the 3GB home quota without requiring symlinks in the project root.

Initial Environment Setup

Run once after cloning the repository. This script handles all modules, mirroring, and environment synchronization.

# Option A: Interactive (on a compute node)
module swap cluster/donphan  # or joltik
qsub -I -l nodes=1:gpus=1
cd "${PBS_O_WORKDIR}"
bash scripts/hpc/install.sh

# Option B: Batch (Run in background)
qsub scripts/hpc/install.sh

Interactive Debugging

To activate your environment for interactive work, simply run the same install.sh script.

qsub -I -l nodes=1:ppn=4 -l walltime=1:00:00
cd "$PBS_O_WORKDIR"
bash scripts/hpc/install.sh

Verification Commands

After installation, run these commands to ensure your environment is set up correctly:

  1. Verify Quota Safety: 
    ls -d venvs 2>/dev/null && echo "FAIL" || echo ">>> PASS: Project root is clean."
  2. Verify Library Versions (NumPy Fix): 
    python -c "import numpy; print(f'NumPy: {numpy.__version__}')"
# Expected: 2.x.x (Venv version), not 1.2x (System version)
  3. Verify GPU Access: 
    python -c "import torch, jax; print(f'GPU: {torch.cuda.is_available()}'); print(f'JAX: {jax.devices()}')"

PR Verification (Quick Start)

If you are a reviewer verifying a PR, run this single block:

git checkout <pr-branch>
module swap cluster/donphan
qsub -I -l nodes=1:gpus=1

# Inside the interactive session:
cd "$PBS_O_WORKDIR"
bash scripts/hpc/install.sh
python -c "import torch, jax; print(torch.cuda.is_available()); print(jax.devices())"
exit

# Verify batch submission
qsub scripts/hpc/train.pbs

Managing Dependencies

env/hpc/requirements.txt is auto-generated from pyproject.toml. To regenerate:

uv run scripts/export_hpc_requirements.py

Modules listed in env/hpc/modules.txt are automatically excluded from the pip requirements to save space and use HPC-optimized binaries.