MarkMH.github.io/mountain_car.html at main · MarkMH/MarkMH.github.io · GitHub

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
<!DOCTYPE HTML>
<!--
	Basic structure of Strata from HTML5 UP html5up.net | @ajlkn
	Free for private and commercial use under CCA 3.0 license (html5up.net/license)
	Significant customizations by Mark Marner-Hausen
-->
<html>
	<head>
		<title>Webpage Marner-Hausen</title>
		<meta charset="utf-8" />
		<meta name="viewport" content="width=device-width, initial-scale=1, user-scalable=no" />
		<link rel="stylesheet" href="assets/css/main.css" />
	</head>
	<body class="is-preload">

		<!-- Header -->
			<header id="header">
				<div class="inner">
					<a class="image avatar"><img src="images/IMG_9947.jpg" alt="" /></a>
					<h1><strong>Hi 👋, I'm Mark,<br />
						a Senior ML Engineer <br />
						with scientific background</strong></a></h1>
				</div>
			</header>

		<!-- Main -->
			<div id="main">

				<!-- One -->
					<section id="one">
						<header class="major">
							<h1>Mountain Car & Neural Networks</h1>

							<p align="justify" style="text-align:justify"><b>
								In this project, I attempt to solve a classical benchmark problem for reinforcement learning algorithms using neural
								networks provided by the <i>torch.nn</i> module. In this attempt, I follow an actor-based on policy method where the policy
								is parameterized by a function approximation, namely, an approximation using neural networks. The advantage of this
								approach over Q-learning is that the stochasticity of the policy allows exploration.
							</b></p>

							<p align="justify" style="text-align:justify"><b><span class="image left"><img src="images/fulls/mountain_car.png" alt="" /></span>
								Mountain Car is a deterministic multidimensional problem where a car is at the bottom of a valley with the goal of
								reaching the flag on the right hill.
								The strategy space consists only of the acceleration of the car in either direction of the valley. The catch is, that
								the car is underpowered, so it will not be able to just go up the steep hill. Consequently, the car must learn to harness
								the potential energy by driving up the opposite hill before it can reach the objective at the top of the right-hand hill.
								It follows that the objective function is not straightforward and that exploration off path is very important.
							</b></p>

							<p align="justify" style="text-align:justify"><b>
								The gym module provides two environments for mountain car: one with discrete actions and one with continuous actions. In
								this project, the former is sought to be solved
								(<a href="https://mgoulao.github.io/gym-docs/environments/classic_control/mountain_car_continuous/">gym documentation</a>).
							</b></p>
						</header>
					</section>

					<section>
								<ul class="actions fit">
									<!--
									<li><a href="XX" class="button fit">Working Paper</a></li>
									<li><a href="XX" class="button fit">GitHub</a></li>
									-->
									<li><a href="index.html" class="button fit">Portfolio</a></li>

								</ul>
					</section>
			</div>

		<!-- Footer -->
			<footer id="footer">
				<div class="inner">
					<ul class="icons">
						<br>
						<li><a href="https://github.com/MarkMH" class="icon brands fa-github"><span class="label">Github</span></a></li>
						<li><a href="https://www.linkedin.com/in/mark-marner-hausen-854a3a117/" class="icon brands fa-linkedin"><span class="label">LinkedIn</span></a></li>
					</ul>
					<ul
					<li><br> Mark Marner-Hausen <br />
						</p>
					</ul>
				</div>
			</footer>

		<!-- Scripts -->
			<script src="assets/js/jquery.min.js"></script>
			<script src="assets/js/jquery.poptrox.min.js"></script>
			<script src="assets/js/browser.min.js"></script>
			<script src="assets/js/breakpoints.min.js"></script>
			<script src="assets/js/util.js"></script>
			<script src="assets/js/main.js"></script>

	</body>
</html>