style: Fix Markdown indentation
This commit is contained in:
parent
bc9419ca53
commit
94f505920c
3 changed files with 13 additions and 13 deletions
|
|
@ -31,8 +31,8 @@ Choosing NerveNet introduces a scalability issue as the morphology grows. In Ner
|
|||
segment or node per propagation step. When dealing with long arms (e.g., > 5 segments), this requires a large number of
|
||||
propagation steps to transmit information from one tip of an arm to another.
|
||||
|
||||
If we would alternatively use SMP - which is possible - the inner states of nodes are shared across the entire graph in
|
||||
just two passes. For very large or long morphologies, this would be much more scalable.
|
||||
If we were to use SMP instead - which is possible - the inner states of nodes are shared across the entire graph in just
|
||||
two passes. For very large or long morphologies, this would be much more scalable.
|
||||
|
||||
By rejecting SMP, we accept that our model might learn slower or require more computational power for highly segmented,
|
||||
extended morphologies.
|
||||
|
|
|
|||
|
|
@ -6,16 +6,16 @@ once, whereas modular controllers divide the brains across the body, inspired by
|
|||
We define four architectures to compare:
|
||||
|
||||
1. **Centralized, monolithic**: A single Multi Layer Perceptron per robot that receives all observations and outputs
|
||||
all actions.
|
||||
all actions.
|
||||
2. **Fully connected arm-level**: Each arm contains an MLP that processes the inputs for that arm, an MLP that processes
|
||||
the communicated inner-states, and an MLP that outputs the actions for that arm. One policy for these MLPs is shared
|
||||
across the arms. The controllers in each arm are connected to each other and form a fully connected graph. There is no
|
||||
central disk, but the controllers are fully connected.
|
||||
the communicated inner-states, and an MLP that outputs the actions for that arm. One policy for these MLPs is shared
|
||||
across the arms. The controllers in each arm are connected to each other and form a fully connected graph. There is
|
||||
no central disk, but the controllers are fully connected.
|
||||
3. **Ring arm-level**: Identical setup to the fully connected arm-level, but the controllers are connected in a ring
|
||||
structure. This setup is considered less centralized than the fully connected graph.
|
||||
structure. This setup is considered less centralized than the fully connected graph.
|
||||
4. **Segment-level**: Each segment contains the three MLPs discussed above. The base segments, attached to the body,
|
||||
form a ring structure, with the remaining segments attached as extended "strings". Segments can only communicate with
|
||||
segments that are physically connected to it.
|
||||
form a ring structure, with the remaining segments attached as extended "strings". Segments can only communicate with
|
||||
segments that are physically connected to it.
|
||||
|
||||
## Rationale
|
||||
|
||||
|
|
|
|||
|
|
@ -17,11 +17,11 @@ failures.
|
|||
Alternative learning algorithms include:
|
||||
|
||||
- **Twin Delayed DDPG (Fujimoto et al., 2018)**: TD3 is a strong off-policy alternative used in the SMP paper (Huang et
|
||||
al., 2020). It is highly sample-efficient and reportedly excels at zero-shot adaptations. However, this approach would be
|
||||
more complex and error-prone than with PPO.
|
||||
al., 2020). It is highly sample-efficient and reportedly excels at zero-shot adaptations. However, this approach would
|
||||
be more complex and error-prone than with PPO.
|
||||
- **Evolution strategies (ES)**: Evolution strategies are useful for optimizing Central Pattern Generators (CPGs), e.g.
|
||||
CMA-ES, OpenAI-ES. While this method is easier to distribute and parallelize, ES typically scales worse with
|
||||
exceptionally large observation spaces compared to gradient-based RL methods like PPO.
|
||||
CMA-ES, OpenAI-ES. While this method is easier to distribute and parallelize, ES typically scales worse with
|
||||
exceptionally large observation spaces compared to gradient-based RL methods like PPO.
|
||||
|
||||
**References**
|
||||
|
||||
|
|
|
|||
Reference in a new issue