1
Fork 0
Commit graph

16 commits

Author SHA1 Message Date
2d08023abb fix(hpc): modules 2026-04-04 20:38:30 +02:00
a512682edd fix(hpc): additional insights 2026-04-04 20:26:09 +02:00
4339978cc6 fix(hpc): relax unbound variables 2026-04-04 20:04:05 +02:00
74e807472b feat(hpc): update PBS script with renamed --env-config-path flag 2026-04-04 19:07:05 +02:00
798a8ee9e6 feat(hpc): correct Tyro flags in PBS submission script 2026-04-04 19:03:38 +02:00
d1ca7a0f9f feat(hpc): pass scratch directory as run_dir in PBS script 2026-04-04 19:02:26 +02:00
92de38608f chore(hpc): finalize hpc settings and scripts
- Update train.pbs to default to smoke test for safety
- Update .gitignore and ruff.toml
- Sync uv.lock (removing python 3.11 remnants)
- Refine smoke test config
2026-04-04 18:59:38 +02:00
34a887cd58 refactor(hpc): use explicit PIP_CACHE_DIR/UV_CACHE_DIR, drop prose comments
Match comment style from previous cleanup commit:
- No section divider bars, no echo progress lines, no prose comments
  on obvious commands
- Replace XDG_CACHE_HOME with explicit PIP_CACHE_DIR and UV_CACHE_DIR
- Drop MPLCONFIGDIR (matplotlib cache is negligible in size)
2026-04-04 18:48:06 +02:00
7bcfa66685 feat(hpc): route caches to $VSC_SCRATCH, outputs to $VSC_DATA
- Install: redirect XDG_CACHE_HOME, UV_CACHE_DIR, MPLCONFIGDIR to
  $VSC_SCRATCH before any pip/module activity ($VSC_HOME is only ~3 GB)
- train.pbs:
  - Set same cache env vars at job start
  - Write run outputs and WandB cache to $VSC_SCRATCH during the run (fast I/O)
  - Copy results to $VSC_DATA/runs/<jobid>/ on completion for persistence
  - Redirect PBS stdout/stderr to $VSC_DATA/runs/<jobid>/job.{out,err}
  - Add cd $PBS_O_WORKDIR as first step (HPC best practice)
2026-04-04 18:39:45 +02:00
08d54b73b4 refactor: comment cleanups 2026-04-04 18:30:10 +02:00
9dce017525 refactor(hpc): migrate to PBS/vsc-venv, reorganise scripts under scripts/hpc/
- Replace Slurm headers with PBS (#PBS) directives
- Replace manual module+venv logic with vsc-venv --activate
- Cluster selection via 'module swap cluster/<name>' before qsub
- MUJOCO_GL=egl retained: required for headless physics sim and video rendering
- Delete deprecated scripts/hpc_install.sh and scripts/hpc_train.slurm
2026-04-04 18:11:38 +02:00
23f0b45614 feat(hpc): add structured env/hpc config and requirements export script
- env/hpc/modules.txt: curated list of HPC EasyBuild modules loaded by vsc-venv
- env/hpc/requirements.txt: pip packages not covered by HPC modules (auto-generated)
- scripts/export_hpc_requirements.py: developer utility to regenerate requirements.txt
  from pyproject.toml; to be called by CI when pyproject.toml changes
2026-04-04 18:10:38 +02:00
f2139fb419 fix(hpc): enable wandb tracking by default on compute nodes 2026-04-04 17:37:47 +02:00
205b284b96 feat(hpc): parse missing dependencies dynamically from pyproject 2026-04-04 17:37:47 +02:00
e81708106d ci(hpc): add slurm batch script for training 2026-04-01 19:34:47 +02:00
62e2adf97d ci(hpc): add optimized installation script 2026-04-01 19:34:47 +02:00