1
Fork 0

docs: describe normalization

This commit is contained in:
Tibo De Peuter 2026-04-28 10:44:55 +02:00
parent a1db544e76
commit 0032496073
Signed by: tdpeuter
SSH key fingerprint: SHA256:u/h/LVoqKF1Iz02uOyxe6hcjmoZASCGV2HM0TG9ZMoU

View file

@ -10,9 +10,12 @@ decentralized control architecture in mind, we divide these inputs into global a
Global inputs, always broadcasted to all nodes:
- Vertical orientation/tilt: A single, simplified metric representing the tilt or vertical alignment of the agent's
central body/disk, derived from the environment's raw disk rotation 3D vector $[roll, pitch, yaw]$:
$$tilt = sqrt(roll^2 + pitch^2)$$. This represents the deviation from the global Z-axis.
- Vertical orientation/tilt: A single, simplified metric representing the tilt/vertical alignment of the agent's
central body/disk, a.k.a. the deviation from the global Z-axis. Its value is derived from the environment's raw disk
rotation 3D vector $[roll, pitch, yaw]$:
$$
tilt = sqrt(roll^2 + pitch^2)
$$
- Goal vector: Instead of just a scalar distance, the goal is represented asa a vector (distance and ange/direction) to
the target.
@ -29,6 +32,16 @@ The action space defines how the agent interacts with the environment.
- Joint offsets: *absolute* target positions (offsets) for the joints, i.e. the exact angle the joint should move to.
## Normalization and Scaling
Both the input (observation) and output (action) spaces are rescaled to the range **$[-1, 1]$**.
For the input space, all raw physical values (angles, velocities, forces, distances) are normalized based on their
defined physical bounds. If a value exceeds these bounds during simulation, it is clipped to the $[-1, 1]$ range.
For the output space, the neural network's tanh-activated outputs (which naturally fall in $[-1, 1]$) are linearly
mapped to the physical joint limits defined in the robot's morphology.
## Rationale
When designing the state space, we must ask: *Could a human operator perform this task given only these inputs?*
@ -59,6 +72,17 @@ When designing the state space, we must ask: *Could a human operator perform thi
random walks or spiraling) to deduce the direction, drastically increasing the difficulty of the learning task.
- Contact sensors: Segment contact detects external ground interaction and is biologically vital for timing gait
transitions.
- **Zero-Centered Rescaling ($[-1, 1]$):** Using a zero-centered range is standard best practice for continuous control
tasks. It provides several mathematical and physical advantages:
- **Improved Gradient Flow:** Neural networks optimize faster when inputs are zero-centered. If all inputs were
positive (e.g., $[0, 1]$), the gradients during backpropagation would be forced to the same sign, causing
inefficient "zig-zag" weight updates.
- **Meaningful "Neutral" State:** In robotics, $0.0$ naturally represents a resting state (zero velocity, centered
position, no force). In a $[-1, 1]$ system, this physical rest maps to a neutral $0.0$ signal in the network.
- **Robustness to Amputation:** In this project, amputated limbs are padded with $0.0$. In a $[-1, 1]$ system, this
correctly communicates a "neutral/dead" signal. In a $[0, 1]$ system, $0.0$ would represent the absolute minimum
physical limit, causing the network to misinterpret missing limbs as being at their extreme limits.
Specifically, we do not include some available inputs:
@ -80,6 +104,10 @@ Alternative state and action formulations include:
- Scalar Goal Distance: Giving the agent only the scalar distance to the target would force it to learn a localized
searching behavior (e.g., spiraling or random walks) to determine the correct direction. While biologically plausible
for simpler organisms following chemical gradients, it drastically increases the difficulty of the learning task.
- **$[0, 1]$ Rescaling:** While some domains (like computer vision) use $[0, 1]$ scaling, it is generally avoided in
robotics. Scaling to $[0, 1]$ would mean that a resting joint (velocity = 0) maps to an input of $0.5$. This
constant positive bias forces the network to waste capacity learning to ignore or subtract this baseline signal just
to stand still. Furthermore, it breaks the "dead signal" interpretation of zero-padding used for amputations.
## MuJoCo