1
Fork 0

style: Fix Markdown indentation

This commit is contained in:
Tibo De Peuter 2026-03-15 23:03:09 +01:00
parent bc9419ca53
commit 94f505920c
Signed by: tdpeuter
GPG key ID: 38297DE43F75FFE2
3 changed files with 13 additions and 13 deletions

View file

@ -31,8 +31,8 @@ Choosing NerveNet introduces a scalability issue as the morphology grows. In Ner
segment or node per propagation step. When dealing with long arms (e.g., > 5 segments), this requires a large number of
propagation steps to transmit information from one tip of an arm to another.
If we would alternatively use SMP - which is possible - the inner states of nodes are shared across the entire graph in
just two passes. For very large or long morphologies, this would be much more scalable.
If we were to use SMP instead - which is possible - the inner states of nodes are shared across the entire graph in just
two passes. For very large or long morphologies, this would be much more scalable.
By rejecting SMP, we accept that our model might learn slower or require more computational power for highly segmented,
extended morphologies.

View file

@ -6,16 +6,16 @@ once, whereas modular controllers divide the brains across the body, inspired by
We define four architectures to compare:
1. **Centralized, monolithic**: A single Multi Layer Perceptron per robot that receives all observations and outputs
all actions.
all actions.
2. **Fully connected arm-level**: Each arm contains an MLP that processes the inputs for that arm, an MLP that processes
the communicated inner-states, and an MLP that outputs the actions for that arm. One policy for these MLPs is shared
across the arms. The controllers in each arm are connected to each other and form a fully connected graph. There is no
central disk, but the controllers are fully connected.
the communicated inner-states, and an MLP that outputs the actions for that arm. One policy for these MLPs is shared
across the arms. The controllers in each arm are connected to each other and form a fully connected graph. There is
no central disk, but the controllers are fully connected.
3. **Ring arm-level**: Identical setup to the fully connected arm-level, but the controllers are connected in a ring
structure. This setup is considered less centralized than the fully connected graph.
structure. This setup is considered less centralized than the fully connected graph.
4. **Segment-level**: Each segment contains the three MLPs discussed above. The base segments, attached to the body,
form a ring structure, with the remaining segments attached as extended "strings". Segments can only communicate with
segments that are physically connected to it.
form a ring structure, with the remaining segments attached as extended "strings". Segments can only communicate with
segments that are physically connected to it.
## Rationale

View file

@ -17,11 +17,11 @@ failures.
Alternative learning algorithms include:
- **Twin Delayed DDPG (Fujimoto et al., 2018)**: TD3 is a strong off-policy alternative used in the SMP paper (Huang et
al., 2020). It is highly sample-efficient and reportedly excels at zero-shot adaptations. However, this approach would be
more complex and error-prone than with PPO.
al., 2020). It is highly sample-efficient and reportedly excels at zero-shot adaptations. However, this approach would
be more complex and error-prone than with PPO.
- **Evolution strategies (ES)**: Evolution strategies are useful for optimizing Central Pattern Generators (CPGs), e.g.
CMA-ES, OpenAI-ES. While this method is easier to distribute and parallelize, ES typically scales worse with
exceptionally large observation spaces compared to gradient-based RL methods like PPO.
CMA-ES, OpenAI-ES. While this method is easier to distribute and parallelize, ES typically scales worse with
exceptionally large observation spaces compared to gradient-based RL methods like PPO.
**References**