Deployed 9759810 with MkDocs version: 1.6.1
This commit is contained in:
parent
281bae4df2
commit
1d2a68dc54
21 changed files with 128 additions and 32 deletions
4
404.html
4
404.html
|
|
@ -1078,6 +1078,10 @@
|
|||
|
||||
<script src="/assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="/javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1310,6 +1310,10 @@
|
|||
|
||||
<script src="../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1461,6 +1461,10 @@
|
|||
|
||||
<script src="../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1426,6 +1426,10 @@ bash<span class="w"> </span>scripts/hpc/install.sh
|
|||
|
||||
<script src="../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1530,6 +1530,10 @@ uv<span class="w"> </span>run<span class="w"> </span>python<span class="w"> </sp
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1253,6 +1253,10 @@ such as camera locations, simulation time and the task.</li>
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1364,6 +1364,10 @@
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1273,6 +1273,10 @@
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1267,7 +1267,7 @@
|
|||
<p>All runs are recorded locally in the <code>runs/</code> directory (or the directory specified in <code>experiment.base_run_dir</code>). You can view scalars and other metrics with TensorBoard:</p>
|
||||
<div class="highlight"><pre><span></span><code>tensorboard<span class="w"> </span>--logdir<span class="w"> </span>runs/
|
||||
</code></pre></div>
|
||||
<p>Access the interface at <code>http://localhost:6006</code>.</p>
|
||||
<p>Access the interface at <a href="http://localhost:6006">http://localhost:6006</a>.</p>
|
||||
<h3 id="cli-exploration-tool">CLI Exploration Tool</h3>
|
||||
<p>For quick diagnostics or to export data to CSV without launching the full TensorBoard UI, you can use the <code>explore_tensorboard.py</code> script:</p>
|
||||
<div class="highlight"><pre><span></span><code>uv<span class="w"> </span>run<span class="w"> </span>python<span class="w"> </span>scripts/analysis/explore_tensorboard.py<span class="w"> </span>runs/your_run_name/
|
||||
|
|
@ -1327,6 +1327,10 @@
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1332,6 +1332,10 @@
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1136,12 +1136,12 @@ estimation (Critic).</p>
|
|||
<p><strong>Centralized Architecture (Baseline)</strong></p>
|
||||
<p>This pipeline treats the agent as a single entity and uses standard Proximal Policy Optimization (PPO).</p>
|
||||
<ul>
|
||||
<li>Centralized Actor: Composed of two chained MLPs (Sensor $\rightarrow$ Motor) passing a hidden state between them. The
|
||||
<li>Centralized Actor: Composed of two chained MLPs (Sensor <span class="arithmatex">\(\rightarrow\)</span> Motor) passing a hidden state between them. The
|
||||
centralized sensor receives the concatenated global state vector of all limbs at once and processes it into a hidden
|
||||
state. The centralized motor receives this hidden state and outputs the joint offsets for all actuators
|
||||
simultaneously. This is mathematically equivalent to using one large MLP with hidden layers, but splitting makes the
|
||||
implementation easier by allowing us to reuse the same components for the decentralized modules.</li>
|
||||
<li>Centralized Critic: Composed of two sequential MLPs (Feature Extractor $\rightarrow$ Critic). Because PPO evaluates
|
||||
<li>Centralized Critic: Composed of two sequential MLPs (Feature Extractor <span class="arithmatex">\(\rightarrow\)</span> Critic). Because PPO evaluates
|
||||
the state-value function, this network only receives the concatenated global state vector (no actions). It outputs a
|
||||
single scalar estimating the expected future reward for the entire agent.</li>
|
||||
</ul>
|
||||
|
|
@ -1172,10 +1172,10 @@ variant.</p>
|
|||
<li>Decentralized Actor, split into three distinct models:</li>
|
||||
<li>Sensor: A local model at each node. It receives its local state plus the goal vector directly, processing them into
|
||||
an initial hidden state.</li>
|
||||
<li>Propagator: Nodes synchronously compute and exchange messages with connected neighbors for $N$ steps to update
|
||||
<li>Propagator: Nodes synchronously compute and exchange messages with connected neighbors for <span class="arithmatex">\(N\)</span> steps to update
|
||||
their hidden states. See <a href="../communication/">communication.md</a> for details.</li>
|
||||
<li>Motor: A local model uses its final updated hidden state to output the joint offset strictly for its own actuator.</li>
|
||||
<li>Centralized Critic: Composed of two sequential MLPs (Feature Extractor $\rightarrow$ Critic). During training, it
|
||||
<li>Centralized Critic: Composed of two sequential MLPs (Feature Extractor <span class="arithmatex">\(\rightarrow\)</span> Critic). During training, it
|
||||
acts globally by taking the concatenated state vectors from all sensors to output a single, global state-value scalar
|
||||
evaluating the entire agent's pose.</li>
|
||||
</ul>
|
||||
|
|
@ -1217,7 +1217,7 @@ critic for all nodes at once, for the following reasons:</p>
|
|||
|
||||
Prop -.->|"message passing"|Prop</code></pre>
|
||||
<h2 id="implementation-details-network-depth">Implementation Details (Network Depth)</h2>
|
||||
<p>Inspired by: https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/</p>
|
||||
<p>Inspired by: <a href="https://iclr-blog-track.github.io/2022/03/25/ppo-implementation-details/">PPO Implementation Details</a></p>
|
||||
<p>The MLPs used in both pipelines are defined with specific hidden layer configurations to balance learning capability
|
||||
and computational cost. As of right now, though this might change as we make progress in our experiments, we use:</p>
|
||||
<ul>
|
||||
|
|
@ -1233,9 +1233,9 @@ and computational cost. As of right now, though this might change as we make pro
|
|||
by previous research to maintain learning stability.</p>
|
||||
<p><strong>References</strong></p>
|
||||
<ul>
|
||||
<li>Ha, D. (2017, October 29). A Visual Guide to Evolution Strategies. 大トロ ・ Machine Learning. https://blog.otoro.net/2017/10/29/visual-evolution-strategies/</li>
|
||||
<li>Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. ‘Proximal Policy Optimization Algorithms’. arXiv:1707.06347. Preprint, arXiv, 28 August 2017. https://doi.org/10.48550/arXiv.1707.06347.</li>
|
||||
<li>Wang, Tingwu, Renjie Liao, Jimmy Ba, and S. Fidler. ‘NerveNet: Learning Structured Policy with Graph Neural Networks’. Conference paper presented at International Conference on Learning Representations. 15 February 2018. https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613.</li>
|
||||
<li>Ha, D. (2017, October 29). A Visual Guide to Evolution Strategies. 大トロ ・ Machine Learning. <a href="https://blog.otoro.net/2017/10/29/visual-evolution-strategies/">https://blog.otoro.net/2017/10/29/visual-evolution-strategies/</a></li>
|
||||
<li>Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. ‘Proximal Policy Optimization Algorithms’. arXiv:1707.06347. Preprint, arXiv, 28 August 2017. <a href="https://doi.org/10.48550/arXiv.1707.06347">https://doi.org/10.48550/arXiv.1707.06347</a>.</li>
|
||||
<li>Wang, Tingwu, Renjie Liao, Jimmy Ba, and S. Fidler. ‘NerveNet: Learning Structured Policy with Graph Neural Networks’. Conference paper presented at International Conference on Learning Representations. 15 February 2018. <a href="https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613">https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613</a>.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
|
@ -1291,6 +1291,10 @@ by previous research to maintain learning stability.</p>
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1183,8 +1183,8 @@ two passes. For very large or long morphologies, this would be much more scalabl
|
|||
extended morphologies.</p>
|
||||
<p><strong>References</strong></p>
|
||||
<ul>
|
||||
<li>Wang, Tingwu, Renjie Liao, Jimmy Ba, and S. Fidler. ‘NerveNet: Learning Structured Policy with Graph Neural Networks’. Conference paper presented at International Conference on Learning Representations. 15 February 2018. https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613.</li>
|
||||
<li>Huang, Wenlong, Igor Mordatch, and Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 July 2020. https://doi.org/10.48550/arXiv.2007.04976.</li>
|
||||
<li>Wang, Tingwu, Renjie Liao, Jimmy Ba, and S. Fidler. ‘NerveNet: Learning Structured Policy with Graph Neural Networks’. Conference paper presented at International Conference on Learning Representations. 15 February 2018. <a href="https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613">https://www.semanticscholar.org/paper/NerveNet:-Learning-Structured-Policy-with-Graph-Wang-Liao/249408527106d7595d45dd761dd53c83e5a02613</a>.</li>
|
||||
<li>Huang, Wenlong, Igor Mordatch, and Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 July 2020. <a href="https://doi.org/10.48550/arXiv.2007.04976">https://doi.org/10.48550/arXiv.2007.04976</a>.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
|
@ -1240,6 +1240,10 @@ extended morphologies.</p>
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1207,6 +1207,10 @@ use it to test whether our models scale to more complex structures.</p>
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1205,11 +1205,11 @@ decentralized control architecture in mind, we divide these inputs into global a
|
|||
<ul>
|
||||
<li>Vertical orientation/tilt: A single, simplified metric representing the tilt/vertical alignment of the agent's
|
||||
central body/disk, a.k.a. the deviation from the global Z-axis. Its value is derived from the environment's raw disk
|
||||
rotation 3D vector $[roll, pitch, yaw]$:
|
||||
rotation 3D vector <span class="arithmatex">\([roll, pitch, yaw]\)</span>:
|
||||
$$
|
||||
tilt = sqrt(roll^2 + pitch^2)
|
||||
$$</li>
|
||||
<li>Goal vector: A 2D unit vector representing the <em>egocentric</em> direction to the target. A value of $[1.0, 0.0]$
|
||||
<li>Goal vector: A 2D unit vector representing the <em>egocentric</em> direction to the target. A value of <span class="arithmatex">\([1.0, 0.0]\)</span>
|
||||
indicates that the target is directly in front of the agent (angle 0).</li>
|
||||
</ul>
|
||||
<p>Local inputs, routed directly to specific nodes:</p>
|
||||
|
|
@ -1225,10 +1225,10 @@ decentralized control architecture in mind, we divide these inputs into global a
|
|||
<li>Joint offsets: <em>absolute</em> target positions (offsets) for the joints, i.e. the exact angle the joint should move to.</li>
|
||||
</ul>
|
||||
<h2 id="normalization-and-scaling">Normalization and Scaling</h2>
|
||||
<p>Both the input (observation) and output (action) spaces are rescaled to the range <strong>$[-1, 1]$</strong>. </p>
|
||||
<p>Both the input (observation) and output (action) spaces are rescaled to the range <strong><span class="arithmatex">\([-1, 1]\)</span></strong>. </p>
|
||||
<p>For the input space, all raw physical values (angles, velocities, forces, distances) are normalized based on their
|
||||
defined physical bounds. If a value exceeds these bounds during simulation, it is clipped to the $[-1, 1]$ range.</p>
|
||||
<p>For the output space, the neural network's tanh-activated outputs (which naturally fall in $[-1, 1]$) are linearly
|
||||
defined physical bounds. If a value exceeds these bounds during simulation, it is clipped to the <span class="arithmatex">\([-1, 1]\)</span> range.</p>
|
||||
<p>For the output space, the neural network's tanh-activated outputs (which naturally fall in <span class="arithmatex">\([-1, 1]\)</span>) are linearly
|
||||
mapped to the physical joint limits defined in the robot's morphology.</p>
|
||||
<h2 id="rationale">Rationale</h2>
|
||||
<p>When designing the state space, we must ask: <em>Could a human operator perform this task given only these inputs?</em></p>
|
||||
|
|
@ -1254,7 +1254,7 @@ mapped to the physical joint limits defined in the robot's morphology.</p>
|
|||
exactly where the target is relative to its current orientation, allowing for directed and efficient locomotion.</li>
|
||||
</ul>
|
||||
<p>The goal representation is explicitly divided into a directional vector and a scalar distance. Keeping the direction
|
||||
as a normalized unit vector bounds the values to the $[-1, 1]$ range, which stabilizes neural network training.
|
||||
as a normalized unit vector bounds the values to the <span class="arithmatex">\([-1, 1]\)</span> range, which stabilizes neural network training.
|
||||
Providing only a scalar "distance to the goal" would force the agent to learning localized searching behaviors (e.g.
|
||||
random walks or spiraling) to deduce the direction, drastically increasing the difficulty of the learning task.</p>
|
||||
<p><strong>NOTE:</strong> We later dropped the "distance to vector", switching to only a direction as the input. Our reasoning is
|
||||
|
|
@ -1262,18 +1262,18 @@ mapped to the physical joint limits defined in the robot's morphology.</p>
|
|||
simplification that decreases the model input size.</p>
|
||||
<p>The environment provides a raw <code>unit_xy_direction_to_target</code> (global), which we transform into a calculated
|
||||
<code>robot_direction_to_target</code> (egocentric) before passing it to the MLPs. This vector consists of the X and Y
|
||||
direction, where a value of $[1.0, 0.0]$ (mapping to an angle of $0$) means the robot is facing directly towards the
|
||||
direction, where a value of <span class="arithmatex">\([1.0, 0.0]\)</span> (mapping to an angle of <span class="arithmatex">\(0\)</span>) means the robot is facing directly towards the
|
||||
target.
|
||||
- Contact sensors: Segment contact detects external ground interaction and is biologically vital for timing gait
|
||||
transitions.
|
||||
- Zero-Centered Rescaling ($[-1, 1]$): Using a zero-centered range is standard best practice for continuous control
|
||||
- Zero-Centered Rescaling (<span class="arithmatex">\([-1, 1]\)</span>): Using a zero-centered range is standard best practice for continuous control
|
||||
tasks. It provides several mathematical and physical advantages:
|
||||
- Improved Gradient Flow: Neural networks optimize faster when inputs are zero-centered. If all inputs were positive
|
||||
(e.g., $[0, 1]$), the gradients during backpropagation would be forced to the same sign, causing inefficient
|
||||
(e.g., <span class="arithmatex">\([0, 1]\)</span>), the gradients during backpropagation would be forced to the same sign, causing inefficient
|
||||
"zig-zag" weight updates.
|
||||
- Meaningful Neutral State: In robotics, $0.0$ naturally represents a resting state (zero velocity, centered
|
||||
position, no force). In a $[-1, 1]$ system, this physical rest maps to a neutral $0.0$ signal in the network.
|
||||
This also correctly communicaties a "neutral/dead" signal for amputated limbs that are padded with $0.0$ values.</p>
|
||||
- Meaningful Neutral State: In robotics, <span class="arithmatex">\(0.0\)</span> naturally represents a resting state (zero velocity, centered
|
||||
position, no force). In a <span class="arithmatex">\([-1, 1]\)</span> system, this physical rest maps to a neutral <span class="arithmatex">\(0.0\)</span> signal in the network.
|
||||
This also correctly communicaties a "neutral/dead" signal for amputated limbs that are padded with <span class="arithmatex">\(0.0\)</span> values.</p>
|
||||
<p>Specifically, we do not include some available inputs:</p>
|
||||
<ul>
|
||||
<li>Global position: Absolute spatial coordinates can cause the agent to overfit to a specific coordinate frame or map,
|
||||
|
|
@ -1292,13 +1292,13 @@ mapped to the physical joint limits defined in the robot's morphology.</p>
|
|||
<li>Scalar Goal Distance: Giving the agent only the scalar distance to the target would force it to learn a localized
|
||||
searching behavior (e.g., spiraling or random walks) to determine the correct direction. While biologically plausible
|
||||
for simpler organisms following chemical gradients, it drastically increases the difficulty of the learning task.</li>
|
||||
<li><strong>$[0, 1]$ Rescaling:</strong> While some domains (like computer vision) use $[0, 1]$ scaling, it is generally avoided in
|
||||
robotics. Scaling to $[0, 1]$ would mean that a resting joint (velocity = 0) maps to an input of $0.5$. This
|
||||
<li><strong><span class="arithmatex">\([0, 1]\)</span> Rescaling:</strong> While some domains (like computer vision) use <span class="arithmatex">\([0, 1]\)</span> scaling, it is generally avoided in
|
||||
robotics. Scaling to <span class="arithmatex">\([0, 1]\)</span> would mean that a resting joint (velocity = 0) maps to an input of <span class="arithmatex">\(0.5\)</span>. This
|
||||
constant positive bias forces the network to waste capacity learning to ignore or subtract this baseline signal just
|
||||
to stand still. Furthermore, it breaks the "dead signal" interpretation of zero-padding used for amputations.</li>
|
||||
</ul>
|
||||
<h2 id="mujoco">MuJoCo</h2>
|
||||
<p>This is what the filtered input vectors look like in MuJoCo, with $J$ joints and $S$ segments:</p>
|
||||
<p>This is what the filtered input vectors look like in MuJoCo, with <span class="arithmatex">\(J\)</span> joints and <span class="arithmatex">\(S\)</span> segments:</p>
|
||||
<ul>
|
||||
<li><code>joint_position</code>: shape=(J,), dtype=float64</li>
|
||||
<li><code>joint_velocity</code>: shape=(J,), dtype=float64</li>
|
||||
|
|
@ -1307,7 +1307,7 @@ mapped to the physical joint limits defined in the robot's morphology.</p>
|
|||
<li><code>robot_direction_to_target</code>: shape=(2,), dtype=float64, egocentric</li>
|
||||
<li><code>disk_z_tilt</code>: shape=(1,), dtype=float64, derived from <code>disk_rotation</code></li>
|
||||
</ul>
|
||||
<p>This brings the entire input space down to $3J + S + 4$ float64's, compared to $4J + S + 15$ float64's for the
|
||||
<p>This brings the entire input space down to <span class="arithmatex">\(3J + S + 4\)</span> float64's, compared to <span class="arithmatex">\(4J + S + 15\)</span> float64's for the
|
||||
unfiltered inputs.</p>
|
||||
<p>For reference, these are all the inputs that are available in the MuJoCo environment:</p>
|
||||
<div class="highlight"><pre><span></span><code>obs keys: ['joint_position', 'joint_velocity', 'joint_actuator_force', 'actuator_force', 'disk_position', 'disk_rotation', 'disk_linear_velocity', 'disk_angular_velocity', 'tendon_position', 'tendon_velocity', 'segment_contact', 'unit_xy_direction_to_target', 'xy_distance_to_target']
|
||||
|
|
@ -1395,6 +1395,10 @@ xy_distance_to_target: shape=(1,), dtype=float64, size=1
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1172,9 +1172,9 @@ failures.</p>
|
|||
</ul>
|
||||
<p><strong>References</strong></p>
|
||||
<ul>
|
||||
<li>Fujimoto, Scott, Herke Hoof, and David Meger. ‘Addressing Function Approximation Error in Actor-Critic Methods’. Proceedings of the 35th International Conference on Machine Learning, 3 July 2018, 1587-96. https://proceedings.mlr.press/v80/fujimoto18a.html.</li>
|
||||
<li>Huang, Wenlong, Igor Mordatch, and Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 July 2020. https://doi.org/10.48550/arXiv.2007.04976.</li>
|
||||
<li>Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. ‘Proximal Policy Optimization Algorithms’. arXiv:1707.06347. Preprint, arXiv, 28 August 2017. https://doi.org/10.48550/arXiv.1707.06347.</li>
|
||||
<li>Fujimoto, Scott, Herke Hoof, and David Meger. ‘Addressing Function Approximation Error in Actor-Critic Methods’. Proceedings of the 35th International Conference on Machine Learning, 3 July 2018, 1587-96. <a href="https://proceedings.mlr.press/v80/fujimoto18a.html">https://proceedings.mlr.press/v80/fujimoto18a.html</a>.</li>
|
||||
<li>Huang, Wenlong, Igor Mordatch, and Deepak Pathak. ‘One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Control’. arXiv:2007.04976. Preprint, arXiv, 9 July 2020. <a href="https://doi.org/10.48550/arXiv.2007.04976">https://doi.org/10.48550/arXiv.2007.04976</a>.</li>
|
||||
<li>Schulman, John, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. ‘Proximal Policy Optimization Algorithms’. arXiv:1707.06347. Preprint, arXiv, 28 August 2017. <a href="https://doi.org/10.48550/arXiv.1707.06347">https://doi.org/10.48550/arXiv.1707.06347</a>.</li>
|
||||
</ul>
|
||||
|
||||
|
||||
|
|
@ -1230,6 +1230,10 @@ failures.</p>
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1180,7 +1180,7 @@ inputs must be distributed fairly to guarantee an objective comparison between d
|
|||
<li>The distance from the robot to the target and/or the light intensity are treated as global inputs.</li>
|
||||
<li>Positions and joints, which are normalized to floating-point values between 0 and 1, are considered local inputs.</li>
|
||||
<li>The reward function is centered around minimizing the distance to the goal or maximizing the movement towards the goal
|
||||
within a finite number of timesteps $T$.</li>
|
||||
within a finite number of timesteps <span class="arithmatex">\(T\)</span>.</li>
|
||||
<li>To motivate efficient movement, the amount of timesteps taken to reach the goal will be used as penalty.</li>
|
||||
<li>An extra penalty based on movement relative to the current step and the previous is used to penalize a movement away from the target.</li>
|
||||
</ul>
|
||||
|
|
@ -1252,6 +1252,10 @@ this down as potential future research.</p>
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
|
|
@ -1148,6 +1148,8 @@
|
|||
|
||||
|
||||
<h1 id="documentation">Documentation</h1>
|
||||
<p>Welcome to the Brittle Star project documentation. This codebase contains the implementations and research for the scientific evaluation of controller modularity in brittle-star-like robots trained using Reinforcement Learning.</p>
|
||||
<p>For the core codebase, scripts, and contribution history, visit our <a href="https://github.com/SELab-3-2026/SEL3-2026-Groep-4">GitHub Repository</a>.</p>
|
||||
<h2 id="design-architecture-design">Design & architecture (<code>/design</code>)</h2>
|
||||
<p>If you are interested in the "why did you do it like this?"</p>
|
||||
<ul>
|
||||
|
|
@ -1222,6 +1224,10 @@
|
|||
|
||||
<script src="assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
18
javascripts/mathjax.js
Normal file
18
javascripts/mathjax.js
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
window.MathJax = {
|
||||
tex: {
|
||||
inlineMath: [["\\(", "\\)"]],
|
||||
displayMath: [["\\[", "\\]"]],
|
||||
processEscapes: true,
|
||||
processEnvironments: true
|
||||
},
|
||||
options: {
|
||||
ignoreHtmlClass: ".*|",
|
||||
processHtmlClass: "arithmatex"
|
||||
}
|
||||
};
|
||||
|
||||
document.addEventListener("DOMContentLoaded", () => {
|
||||
MathJax.startup.document.state(0);
|
||||
MathJax.typesetClear();
|
||||
MathJax.typesetPromise();
|
||||
});
|
||||
|
|
@ -1239,6 +1239,10 @@ python<span class="w"> </span>explore_tensorboard.py<span class="w"> </span><
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
File diff suppressed because one or more lines are too long
|
|
@ -1412,6 +1412,10 @@
|
|||
|
||||
<script src="../../assets/javascripts/bundle.79ae519e.min.js"></script>
|
||||
|
||||
<script src="../../javascripts/mathjax.js"></script>
|
||||
|
||||
<script src="https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js"></script>
|
||||
|
||||
|
||||
</body>
|
||||
</html>
|
||||
Reference in a new issue