Apply suggestions from code review

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

docs/design/communication.md

@@ -39,5 +39,5 @@ extended morphologies.
 
 **References**
 
-- Wang, Tingwu, Renjie Liao, Jimmy Ba, en S. Fidler. ‘NerveNet: Learning Structured Policy with Graph Neural Networks’. Conference paper presented bij International Conference on Learning Representations. 15 februari 2018. https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613.
-- Huang, Wenlong, Igor Mordatch, en Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 juli 2020. https://doi.org/10.48550/arXiv.2007.04976.
+- Wang, Tingwu, Renjie Liao, Jimmy Ba, and S. Fidler. ‘NerveNet: Learning Structured Policy with Graph Neural Networks’. Conference paper presented at International Conference on Learning Representations. 15 February 2018. https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613.
+- Huang, Wenlong, Igor Mordatch, and Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 July 2020. https://doi.org/10.48550/arXiv.2007.04976.

docs/design/controllers.md

@@ -12,7 +12,7 @@ the communicated inner-states, and an MLP that outputs the actions for that arm.
 across the arms. The controllers in each arm are connected to each other and form a fully connected graph. There is no
 central disk, but the controllers are fully connected.
 3. **Ring arm-level**: Identical setup to the fully connected arm-level, but the controllers are connected in a ring
-structure. This setup is considered less centralised than the fully connected graph.
+structure. This setup is considered less centralized than the fully connected graph.
 4. **Segment-level**: Each segment contains the three MLPs discussed above. The base segments, attached to the body,
 form a ring structure, with the remaining segments attached as extended "strings". Segments can only communicate with
 segments that are physically connected to it.
@@ -22,7 +22,7 @@ segments that are physically connected to it.
 To fairly compare decentralized modularity against centralized control, the decentralized models should not be allowed
 to contain a central organ acting as a bottleneck or coordinator. By removing the central disk in the decentralized
 models and replacing it with a ring topology, we closely approximate the biological reality of the brittle star and test
-a decentralized morphology. 
+a decentralized morphology.
 
-The fully connected graph functions as an intermediate step in between a fully centralised and a decentralised ring. We
-use it to test if the scaling of our models to more complex structures.
+The fully connected graph functions as an intermediate step in between a fully centralized and a decentralized ring. We
+use it to test whether our models scale to more complex structures.

docs/design/learning_algorithm.md

@@ -17,7 +17,7 @@ failures.
 Alternative learning algorithms include:
 
 - **Twin Delayed DDPG (Fujimoto et al., 2018)**: TD3 is a strong off-policy alternative used in the SMP paper (Huang et
-al., 2020). It is highly sample-efficient and reportedly excels at zero-shot adaptions. However, this approach would be
+al., 2020). It is highly sample-efficient and reportedly excels at zero-shot adaptations. However, this approach would be
 more complex and error-prone than with PPO.
 - **Evolution strategies (ES)**: Evolution strategies are useful for optimizing Central Pattern Generators (CPGs), e.g.
 CMA-ES, OpenAI-ES. While this method is easier to distribute and parallelize, ES typically scales worse with
@@ -25,6 +25,6 @@ exceptionally large observation spaces compared to gradient-based RL methods lik
 
 **References**
 
-- Fujimoto, Scott, Herke Hoof, en David Meger. ‘Addressing Function Approximation Error in Actor-Critic Methods’. Proceedings of the 35th International Conference on Machine Learning, 3 juli 2018, 1587-96. https://proceedings.mlr.press/v80/fujimoto18a.html.
-- Huang, Wenlong, Igor Mordatch, en Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 juli 2020. https://doi.org/10.48550/arXiv.2007.04976.
-- Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, en Oleg Klimov. ‘Proximal Policy Optimization Algorithms’. arXiv:1707.06347. Preprint, arXiv, 28 augustus 2017. https://doi.org/10.48550/arXiv.1707.06347.
+- Fujimoto, Scott, Herke Hoof, and David Meger. ‘Addressing Function Approximation Error in Actor-Critic Methods’. Proceedings of the 35th International Conference on Machine Learning, 3 July 2018, 1587-96. https://proceedings.mlr.press/v80/fujimoto18a.html.
+- Huang, Wenlong, Igor Mordatch, and Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 July 2020. https://doi.org/10.48550/arXiv.2007.04976.
+- Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. ‘Proximal Policy Optimization Algorithms’. arXiv:1707.06347. Preprint, arXiv, 28 August 2017. https://doi.org/10.48550/arXiv.1707.06347.

docs/design/reward_function.md

@@ -4,9 +4,9 @@ The robot needs to know whether its movements contribute to the ultimate goal of
 inputs must be distributed fairly to guarantee an objective comparison between different architectures.
 
 - The distance from the robot to the target and/or the light intensity are treated as global inputs.
-- Positions and joints, normalized to floating-point values between 0 and 1 are considered local inputs.
+- Positions and joints, which are normalized to floating-point values between 0 and 1, are considered local inputs.
 - The reward function is centered around minimizing the distance to the goal or maximizing the movement towards the goal
-within a finite number of timesteps $T$.
+  within a finite number of timesteps $T$.
 
 ## Rationale