refactor: set wandb entity and simplify READMEs
This commit is contained in:
parent
3e992a3377
commit
7e7c5bf27c
7 changed files with 21 additions and 218 deletions
|
|
@ -2,91 +2,22 @@
|
|||
|
||||
This directory contains configuration files for training experiments.
|
||||
|
||||
## Quick Start
|
||||
## Usage
|
||||
|
||||
### 1. Choose a Template
|
||||
Use `--config` with `src/train.py` to run an experiment:
|
||||
|
||||
**For Development/Testing:**
|
||||
```bash
|
||||
cp configs/dev_test.yaml configs/my_dev.yaml
|
||||
python src/train.py --config configs/default_ppo.yaml
|
||||
```
|
||||
|
||||
**For Production Training:**
|
||||
You can overriding settings via CLI:
|
||||
```bash
|
||||
cp configs/production_training.yaml configs/my_experiment.yaml
|
||||
python src/train.py --config configs/default_ppo.yaml --learning-rate 0.001
|
||||
```
|
||||
|
||||
### 2. Configure Your Settings
|
||||
## Available Configurations
|
||||
|
||||
Edit your config file and **set your wandb entity**:
|
||||
```yaml
|
||||
# ⚠️ IMPORTANT: Set this to your WandB username or team name
|
||||
wandb_entity: "your-wandb-username"
|
||||
track: true # Enable WandB logging
|
||||
```
|
||||
|
||||
### 3. Run Training
|
||||
|
||||
**Using config file:**
|
||||
```bash
|
||||
python src/train.py --config configs/my_experiment.yaml
|
||||
```
|
||||
|
||||
**Override specific parameters:**
|
||||
```bash
|
||||
python src/train.py --config configs/my_experiment.yaml --learning-rate 0.001 --num-envs 32
|
||||
```
|
||||
|
||||
**Pure CLI (no config file):**
|
||||
```bash
|
||||
python src/train.py --track --wandb-entity your-username --total-timesteps 1000000
|
||||
```
|
||||
|
||||
## Features
|
||||
|
||||
### 📊 WandB Integration
|
||||
- Real-time metrics logging
|
||||
- Model checkpoints as artifacts
|
||||
- Run comparison and collaboration
|
||||
|
||||
### 🔧 Flexible Configuration
|
||||
- YAML files for reproducible experiments
|
||||
- CLI overrides for quick adjustments
|
||||
- Team collaboration without code changes
|
||||
|
||||
## Configuration Templates
|
||||
|
||||
### `dev_test.yaml`
|
||||
- Fast iteration for development
|
||||
- Short runs (100K timesteps)
|
||||
- Frequent checkpoints
|
||||
- Small environment count
|
||||
|
||||
### `production_training.yaml`
|
||||
- Full-scale training (50M timesteps)
|
||||
- Optimized hyperparameters
|
||||
- Production-ready settings
|
||||
|
||||
### `default_ppo.yaml`
|
||||
- Baseline configuration template
|
||||
- Balanced settings for most use cases
|
||||
|
||||
## Team Collaboration
|
||||
|
||||
Each team member should create their own config file:
|
||||
|
||||
```yaml
|
||||
# configs/alice_experiment.yaml
|
||||
exp_name: "alice_locomotion_v2"
|
||||
track: true
|
||||
wandb_project_name: "PPO-Modularity"
|
||||
wandb_entity: "alice-research" # Alice's WandB username
|
||||
total_timesteps: 20000000
|
||||
# ... other settings
|
||||
```
|
||||
|
||||
This allows everyone to:
|
||||
- Use their own WandB account
|
||||
- Run different experiments simultaneously
|
||||
- Share configurations via version control
|
||||
- Avoid conflicts in run names
|
||||
- `default_ppo.yaml`: Baseline config.
|
||||
- `dev_test.yaml`: Fast iteration for development.
|
||||
- `production_training.yaml`: Full-scale training.
|
||||
- `personal_template.yaml`: Template for team members to customize.
|
||||
|
|
|
|||
|
|
@ -15,7 +15,7 @@ seed: 1
|
|||
# Tracking settings
|
||||
track: false # Set to true to enable WandB logging
|
||||
wandb_project_name: "PPO-Modularity"
|
||||
wandb_entity: null # Set to your WandB username or team name
|
||||
wandb_entity: "SEL3-2026-Groep-4" # Set to your WandB username or team name
|
||||
|
||||
# Model saving
|
||||
save_model: true
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ seed: 123
|
|||
# Tracking settings - IMPORTANT: Set your own wandb_entity!
|
||||
track: true
|
||||
wandb_project_name: "PPO-Modularity-Dev"
|
||||
wandb_entity: null # ⚠️ SET THIS TO YOUR WANDB USERNAME OR TEAM
|
||||
wandb_entity: "SEL3-2026-Groep-4" # ⚠️ SET THIS TO YOUR WANDB USERNAME OR TEAM
|
||||
|
||||
# Model saving
|
||||
save_model: true
|
||||
|
|
|
|||
|
|
@ -9,7 +9,7 @@ seed: 42
|
|||
# WandB settings - ⚠️ IMPORTANT: Set your credentials!
|
||||
track: true # Enable WandB tracking
|
||||
wandb_project_name: "PPO-Modularity"
|
||||
wandb_entity: "YOUR_WANDB_USERNAME" # ⚠️ CHANGE THIS to your WandB username/team
|
||||
wandb_entity: "SEL3-2026-Groep-4" # ⚠️ CHANGE THIS to your WandB username/team
|
||||
|
||||
# Quick experiment settings (modify as needed)
|
||||
total_timesteps: 500000 # 500K for quick results
|
||||
|
|
|
|||
|
|
@ -10,7 +10,7 @@ seed: 42
|
|||
# Tracking settings - IMPORTANT: Set your own wandb_entity!
|
||||
track: true
|
||||
wandb_project_name: "PPO-Modularity"
|
||||
wandb_entity: null # ⚠️ SET THIS TO YOUR WANDB USERNAME OR TEAM
|
||||
wandb_entity: "SEL3-2026-Groep-4" # ⚠️ SET THIS TO YOUR WANDB USERNAME OR TEAM
|
||||
|
||||
# Model saving
|
||||
save_model: true
|
||||
|
|
|
|||
|
|
@ -26,7 +26,7 @@ class PPOArgs:
|
|||
wandb_project_name: str = "PPO-Modularity"
|
||||
|
||||
# the entity (team) of wandb's project
|
||||
wandb_entity: str | None = None
|
||||
wandb_entity: str | None = "SEL3-2026-Groep-4"
|
||||
|
||||
# whether to capture videos of the agent performances (check out `videos` folder)
|
||||
capture_video: bool = False
|
||||
|
|
|
|||
|
|
@ -1,145 +1,17 @@
|
|||
# Experiment Logger
|
||||
|
||||
A lightweight, standalone logging framework for machine learning experiments with multi-backend support.
|
||||
A lightweight logging framework supporting Weights & Biases, local JSON, and stdout.
|
||||
|
||||
## Features
|
||||
|
||||
- **Multi-backend logging**: Simultaneously log to WandB, local disk (JSON), and stdout
|
||||
- **Data preservation**: All metrics saved locally, even if WandB is unavailable
|
||||
- **Checkpoint management**: Save model checkpoints with metadata
|
||||
- **WandB integration**: Optional artifact upload for model versioning
|
||||
- **Graceful degradation**: Works without WandB installed
|
||||
- **Simple API**: Minimal configuration required
|
||||
|
||||
## Installation
|
||||
|
||||
This package is included in the project. To use it in your code:
|
||||
|
||||
```python
|
||||
from experiment_logger import UnifiedLogger
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from experiment_logger import UnifiedLogger
|
||||
|
||||
# Initialize logger
|
||||
logger = UnifiedLogger(
|
||||
run_name="my_experiment",
|
||||
config={"learning_rate": 0.001, "batch_size": 32},
|
||||
project_name="MyProject",
|
||||
entity="my-wandb-username", # Optional
|
||||
use_wandb=True, # Set to False to disable WandB
|
||||
)
|
||||
logger = UnifiedLogger(run_name="my_experiment", config={"lr": 0.001})
|
||||
|
||||
# Log metrics
|
||||
for step in range(100):
|
||||
logger.log({
|
||||
"loss": 1.0 / (step + 1),
|
||||
"accuracy": step * 0.01,
|
||||
}, step=step)
|
||||
|
||||
# Save checkpoint
|
||||
logger.save_checkpoint(
|
||||
params=model_params,
|
||||
step=100,
|
||||
metadata={"epoch": 1, "val_acc": 0.95},
|
||||
)
|
||||
|
||||
# Save final model
|
||||
logger.save_final_model(
|
||||
params=final_params,
|
||||
metadata={"final_accuracy": 0.98},
|
||||
)
|
||||
|
||||
# Finalize (flushes remaining metrics)
|
||||
logger.log({"loss": 0.5}, step=1)
|
||||
logger.save_checkpoint(params=model_params, step=1)
|
||||
logger.finish()
|
||||
```
|
||||
|
||||
## Context Manager
|
||||
|
||||
Use as a context manager for automatic cleanup:
|
||||
|
||||
```python
|
||||
with UnifiedLogger(run_name="my_exp", config={}) as logger:
|
||||
logger.log({"metric": 1.0})
|
||||
# Automatically calls finish() on exit
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
### Constructor Parameters
|
||||
|
||||
- `run_name` (str): Unique name for this run
|
||||
- `config` (dict): Configuration dictionary with hyperparameters
|
||||
- `project_name` (str): WandB project name (default: "PPO-Modularity")
|
||||
- `entity` (str, optional): WandB entity (team/user name)
|
||||
- `base_dir` (str): Base directory for local storage (default: "runs")
|
||||
- `use_wandb` (bool): Enable WandB logging (default: True)
|
||||
- `save_code` (bool): Save code to WandB (default: True)
|
||||
|
||||
### Directory Structure
|
||||
|
||||
```
|
||||
runs/
|
||||
└── my_experiment/
|
||||
├── config.json # Saved configuration
|
||||
├── metrics/
|
||||
│ └── metrics.jsonl # Line-delimited JSON metrics
|
||||
├── checkpoints/
|
||||
│ ├── checkpoint_step_100.flax
|
||||
│ └── checkpoint_step_100_metadata.json
|
||||
└── final_model.flax
|
||||
```
|
||||
|
||||
## API Reference
|
||||
|
||||
### `log(metrics, step=None, commit=True)`
|
||||
|
||||
Log metrics to all backends.
|
||||
|
||||
**Parameters:**
|
||||
- `metrics` (dict): Dictionary of metric name -> value
|
||||
- `step` (int, optional): Global step counter (auto-incremented if None)
|
||||
- `commit` (bool): Whether to commit to WandB immediately
|
||||
|
||||
### `save_checkpoint(params, step, prefix="checkpoint", metadata=None)`
|
||||
|
||||
Save model checkpoint to disk and optionally to WandB.
|
||||
|
||||
**Parameters:**
|
||||
- `params`: Model parameters (Flax params or any serializable object)
|
||||
- `step` (int): Current training step
|
||||
- `prefix` (str): Prefix for checkpoint filename
|
||||
- `metadata` (dict, optional): Additional metadata to save
|
||||
|
||||
### `save_final_model(params, metadata=None)`
|
||||
|
||||
Save the final trained model.
|
||||
|
||||
**Parameters:**
|
||||
- `params`: Model parameters
|
||||
- `metadata` (dict, optional): Metadata about the final model
|
||||
|
||||
### `finish()`
|
||||
|
||||
Finalize logging and cleanup. Flushes remaining metrics to disk.
|
||||
|
||||
## Usage in Projects
|
||||
|
||||
This logger is designed to be:
|
||||
- **Project-agnostic**: Use in any ML project, not just this one
|
||||
- **Framework-agnostic**: Works with JAX, PyTorch, TensorFlow, etc.
|
||||
- **Minimal dependencies**: Only requires `wandb` (optional), `flax` (for serialization), and `numpy`
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
1. **Never lose data**: All metrics saved locally, regardless of WandB availability
|
||||
2. **Simple API**: Minimal boilerplate, easy to integrate
|
||||
3. **Fail gracefully**: Missing WandB shouldn't break experiments
|
||||
4. **Reproducibility**: Save full configuration with every run
|
||||
|
||||
## License
|
||||
|
||||
Part of the 2026SEL3-project-BrittleStar repository.
|
||||
Logs and checkoints are saved in the `runs/` directory. If `track=True` (or `use_wandb=True`), everything is additionally synced to Weights & Biases.
|
||||
|
|
|
|||
Reference in a new issue