Subjects reinforcement learning

Marvin Maze Rewards C51219

Step-by-step solutions with LaTeX - clean, fast, and student-friendly.

Use the AI math solver

1. **Problem Statement:** We need to redefine the rewards vector $r_b$ for Marvin's maze problem to reflect that Marvin gets no reward for entering cells 3 and 14, but instead receives rewards of +5 and +2 respectively when teleporting out of these cells. 2. **Original Reward Issue:** The original rewards vector assigned zero reward for being in cell 3 and 14 because it considered rewards only upon entering the state, not upon teleporting out. 3. **New Reward Structure:** - Reward for entering cell 3: $0$ - Reward for teleporting out of cell 3: $+5$ - Reward for entering cell 14: $0$ - Reward for teleporting out of cell 14: $+2$ - Other rewards remain the same. 4. **Adjusting the Rewards Vector:** - Since teleportation happens after being in cell 3 or 14, we add the teleportation reward to the reward of the state from which teleportation occurs. - For cell 3 (index 2 in zero-based indexing), reward remains $0$. - For cell 14 (index 13), reward remains $0$. - We add $+5$ to the reward of the state from which teleportation occurs out of cell 3 (cell 3 teleports to cell 8, so reward is assigned at cell 3's teleportation step). - Similarly, add $+2$ to the reward of the state from which teleportation occurs out of cell 14 (cell 14 teleports to cell 12). 5. **Final Rewards Vector $r_b$:** $$ r_b = \begin{bmatrix} -\frac{1}{2}, \frac{3}{4}, 0, \frac{3}{4}, -\frac{1}{2}, 1, -\frac{1}{4}, -\frac{1}{4}, -\frac{1}{4} + 5, -\frac{1}{8}, \frac{1}{4}, -\frac{1}{2}, -\frac{1}{2} + 2, 0 \end{bmatrix} $$ 6. **Simplify the vector:** - $-\frac{1}{4} + 5 = 4.75$ - $-\frac{1}{2} + 2 = 1.5$ So, $$ r_b = \begin{bmatrix} -0.5, 0.75, 0, 0.75, -0.5, 1, -0.25, -0.25, 4.75, -0.125, 0.25, -0.5, 1.5, 0 \end{bmatrix} $$