tdpeuter/2026SEL3-project-Brittle_Star_Controller_Modularity
tdpeuter/2026SEL3-project-Brittle_Star_Controller_Modularity
Archived
1
Fork 0
Code Issues Releases Activity

fix(hpc): numpy

Browse source
This commit is contained in:
Tibo De Peuter 2026-04-04 22:58:47 +02:00
parent 520d429bac
commit 70dd8c3161
3 changed files with 66 additions and 43 deletions
Download patch file Download diff file Expand all files Collapse all files

68
docs/HPC.md
View file

@ -9,29 +9,25 @@ Full documentation: <https://docs.hpc.ugent.be/>
## Initial Environment Setup
Run **once** after cloning the repository. Ensure you are logged into a **compute node** on `donphan` or `joltik`.
> [!IMPORTANT]
> To avoid the **3GB home directory quota limit**, the installation script mirrors your configuration files to **`$VSC_DATA`** (25GB+ quota). The `vsc-venv` tool then automatically creates and manages the environment on the larger partition.
Run **once** after cloning the repository. This script handles all modules, mirroring, and environment synchronization.
```bash
# 1. Start an interactive session (donphan for debug, joltik for training)
qsub -I -l nodes=1:ppn=8:gpus=1
# 2. Run the streamlined install script
# Option A: Interactive (on a compute node)
module swap cluster/donphan # or joltik
qsub -I -l nodes=1:gpus=1
cd "${PBS_O_WORKDIR}"
bash scripts/hpc/install.sh
# Option B: Batch (Run in background)
qsub scripts/hpc/install.sh
```
## Interactive Debugging
You can use the same `install.sh` script to quickly activate your environment for interactive work.
To activate your environment for interactive work, simply run the same `install.sh` script.
```bash
# Request an interactive job
qsub -I -l nodes=1:ppn=4 -l walltime=1:00:00
# Change to project directory and run install.sh to sync and activate
cd "$PBS_O_WORKDIR"
bash scripts/hpc/install.sh
```
@ -40,39 +36,39 @@ bash scripts/hpc/install.sh
After installation, run these commands to ensure your environment is set up correctly:
1. **Verify Location**:
1. **Verify Quota Safety**:
```bash
# Confirm that NO 'venvs' folder appeared in your project root
ls -d venvs 2>/dev/null # Should return 'not found'
# Confirm the environment is on the data partition
python -c "import torch; print(torch.__file__)"
# Expected: /kyukon/data/gent/vsc... or similar
ls -d venvs 2>/dev/null && echo "FAIL" || echo ">>> PASS: Project root is clean."
```
2. **Verify Library Versions (NumPy Fix)**:
```bash
python -c "import numpy; print(f'NumPy: {numpy.__version__}')"
# Expected: 2.x.x (Venv version), not 1.2x (System version)
```
3. **Verify GPU Access**:
```bash
python -c "import torch, jax; print(f'GPU: {torch.cuda.is_available()}'); print(f'JAX: {jax.devices()}')"
```
2. **Verify GPU Access**:
```bash
python -c "import torch; import jax; print(f'Torch CUDA: {torch.cuda.is_available()}'); print(f'JAX Devices: {jax.devices()}')"
```
*Expected output: `Torch CUDA: True` and `JAX Devices: [CudaDevice(id=0)]`.*
## PR Verification (Quick Start)
3. **Verify Home Quota**:
```bash
df -h ~ # Should show low usage (< 1GB typically)
```
## Submitting Batch Training Jobs
If you are a reviewer verifying a PR, run this single block:
```bash
# Submit to the default cluster (joltik)
git checkout <pr-branch>
module swap cluster/donphan
qsub -I -l nodes=1:gpus=1
# Inside the interactive session:
cd "$PBS_O_WORKDIR"
bash scripts/hpc/install.sh
python -c "import torch, jax; print(torch.cuda.is_available()); print(jax.devices())"
exit
# Verify batch submission
qsub scripts/hpc/train.pbs
# To choose a different cluster (e.g. donphan debug) without touching code
module swap cluster/donphan && qsub scripts/hpc/train.pbs
```
The `train.pbs` script automatically handles its own activation using the mirrored configurations on `$VSC_DATA`.
## Managing Dependencies
`env/hpc/requirements.txt` is auto-generated from `pyproject.toml`. To regenerate: