From ed27c043c91dee197e5a2b4c12904c12bd97ff15 Mon Sep 17 00:00:00 2001
From: wutong88yuyu <2179751939@qq.com>
Date: Tue, 16 Dec 2025 21:24:20 +0800
Subject: [PATCH 1/5] =?UTF-8?q?=E8=BD=A6=E9=81=93=E4=BF=9D=E6=8C=81?=
=?UTF-8?q?=E8=BE=85=E5=8A=A9?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
---
src/Active-Lane-Keeping-Assistant/README.md | 206 ++++++-
src/Active-Lane-Keeping-Assistant/agent.py | 198 +++++++
src/Active-Lane-Keeping-Assistant/lane.py | 512 ++++++++++++++++++
src/Active-Lane-Keeping-Assistant/main.py | 297 ++++++++++
src/Active-Lane-Keeping-Assistant/recorder.py | 63 +++
.../requirements.txt | 4 +
src/Active-Lane-Keeping-Assistant/world.py | 196 +++++++
7 files changed, 1475 insertions(+), 1 deletion(-)
create mode 100644 src/Active-Lane-Keeping-Assistant/agent.py
create mode 100644 src/Active-Lane-Keeping-Assistant/lane.py
create mode 100644 src/Active-Lane-Keeping-Assistant/main.py
create mode 100644 src/Active-Lane-Keeping-Assistant/recorder.py
create mode 100644 src/Active-Lane-Keeping-Assistant/requirements.txt
create mode 100644 src/Active-Lane-Keeping-Assistant/world.py
diff --git a/src/Active-Lane-Keeping-Assistant/README.md b/src/Active-Lane-Keeping-Assistant/README.md
index 4f80bd0eea..ebd89fdb3e 100644
--- a/src/Active-Lane-Keeping-Assistant/README.md
+++ b/src/Active-Lane-Keeping-Assistant/README.md
@@ -1 +1,205 @@
-车道辅助系统的完成
\ No newline at end of file
+# Active Lane Keeping Assistant
+
+> Please note that the code and contributions in this GitHub repository were created through a collaborative effort by [@Irish-77](https://github.com/Irish-77) and [@Ronho](https://github.com/Ronho). Both contributors acknowledge that their work is of equal merit and that they each have a deep understanding of the other's contributions.
+
+The aim of this project is to control a vehicle. Computer vision methods are used for lane recognition. The final control of the vehicle is done with the help of different variants of the PID controller. The variants are also investigated and compared.
+
+## Table of contents
+1. [Installation](#installation)
+1. [Description](#description)
+ 1. [Introductory Questions](#introductory-questions)
+ 1. [Computer Vision to Identify the Track](#computer-vision-to-identify-the-track)
+ 1. [Control Technology for Tracking](#control-technology-for-tracking)
+1. [Closing Words](#closing-words)
+1. [Sources](#sources)
+
+## Installation
+
+First, the project must be cloned via git using the following command.
+```sh
+git clone https://github.com/C2G-BR/Active-Lane-Keeping-Assistant.git
+```
+To be able to test the project, you have to install [CARLA](https://github.com/carla-simulator/carla/releases) next. This project was created and tested with version 0.9.13 of CARLA.
+
+Additionally, a Python environment is required. Due to dependencies it is necessary to use Python 3.8. All necessary packages can be installed with the command
+```sh
+pip install -r requirements.txt
+```
+that must be executed inside the project folder.
+
+To run the program, CARLA must be started at the beginning. Afterwards a run can be started with
+```sh
+python src/main.py -id "test" -c pid -s 1000
+```
+within the project's folder. Within the CARLA interface, a car should now appear that starts to move. To find the vehicle, it may be necessary to look around within the world. All possible configuration settings can be viewed by
+```sh
+python main.py -h
+```
+If the error
+```sh
+RuntimeError: time-out of 5000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000
+```
+occurs, it can help to execute the command multiple times and restart CARLA.
+
+## Description
+
+### Introductory Questions
+In order to implement the system, the following questions first arose:
+
+What metric can we use to obtain information about the current deviation from the track?
+
+>Since the focus of this project is on image-based control of the vehicle, several transformation steps were applied to the image to find the track, which are described below. These transformations do not use machine learning techniques in the process. The deviation value from the center point can ultimately be derived from the detected line.
+
+Which algorithm can be used to keep the track optimal?
+
+>Different control engineering methods were selected for this project. The following algorithms were evaluated:
+>1. Simple countersteering with fixed steering angle
+>1. P controller
+>1. PD controller
+>1. PID controller
+
+### Computer Vision to Identify the Track
+As described earlier, no machine learning methods were used for track finding. There are two reasons for this:
+- many machine learning methods often still work according to a black box principle, i.e. the exact reason for prediction is usually not necessarily comprehensible.
+- on the other hand it is also interesting to see how only image transformations can already detect the roadway.
+
+**Approach**
+
+To extract the trace through image-based transformations, several steps were performed, which are described below.
+
+
+
+
+Original image that we get from the simulator. We get a new frame every 0.01 seconds.
+
+
+
+
+
+The image shows the original image in black and white. All colors that are within a defined color space are marked with white. All other pixels are replaced by black pixels. Additionally, the blur of the image was increased.
+
+
+
+
+
+The region of interest (RoI, outlined in pink here) indicates the area to be viewed. This means that only the current lane is considered. The RoI should be large enough to include curves, but small enough not to include other lane lines.
+
+
+
+
+
+Here the angle is changed to a bird's eye view. Only the region of interest is taken into account. You can already see the track better. Through the vanishing point the lines approach each other.
+
+
+
+
+
+The figure shows the transformed image in binary representation and the corresponding histogram. The peaks of the histogram correspond to the gradients (i.e. the change from black to white and vice versa) from the image above. From the histogram, the outer limits of the lane can be read. The center of the roadway corresponds to the center of the two histogram peaks.
+
+
+
+
+
+During this step, a small window is slid over the image. Based on the non-black pixels inside the sliding window, important line points are extracted.
+
+
+
+
+
+Using the calculated coordinates of the line points (from the previous image), a polynomial is approximated for each side.
+
+
+
+
+
+Original image with detected track.
+
+
+**Challenges during Implementation**
+
+> No or Incorrectly Detected Track
+
+Based on the detected tracks, the center point could be determined afterwards. However, there is a problem here. On the test track "Town04" (see below), which we have selected here, there are no solid lines (only the separating lane on the far left and right), which is why our method has difficulty recognizing a reasonable lane. The second figure below shows what such an incorrectly detected trace looks like. In the code, we caught the error by calculating the area of the detected trace each time. If the area falls below a predefined threshold, the detected trace is ignored and the center of the last valid trace is used as a starting point for further decisions.
+
+
+
+
+Town04 in CARLA
+
+
+
+
+
+The illustration shows a misidentified track. Within the red box (used here only as a visual aid), a narrow green stripe can be seen. However, the green stripe does not correspond to any lane and is therefore an error.
+
+
+> Highway Exit
+
+In the CARLA simulation, the vehicle is spawned on the right lane. However, the exits are also linked to the right lane. As soon as an exit comes up, our program has problems to detect a lane, because for a short time the right lane is missing (see below). To avoid such errors, a lane change to the left lane is performed at the beginning of each simulation.
+
+
+
+
+On the right lane, the dividing line is missing for a short time.
+
+
+> Collision Detection
+
+Another problem was the collision with the concrete sliding wall. If the vehicle does not detect a lane or the parameters are configured too sensitively, the vehicle leaves the lane and collides with a concrete sliding wall. Sometimes, however, a phenomenon occurs that shortly before the vehicle collides, it detects a lane for a short time. Accordingly, the vehicle stores the last deviation of the last valid time for the current error (see solution from chapter No or Incorrectly Detected Lane). During optimization, this phenomenon can cause problems. As a solution we have implemented a CollisionDetector. As soon as the vehicle collides with an object and therefore comes to a stop, the previous error is stored for the remaining time.
+
+### Control Technology for Tracking
+
+As already described in the introduction, four procedures were tested. The simple method countersteers with a constant steering angle as soon as it exceeds a threshold value. In this way, the agent manages to keep the vehicle on track without any problems. However, the abrupt and uneven steering movements also reduce driving comfort. Programming a simple procedure was nevertheless helpful, as it allowed the test environment to be tested initially.
+
+Based on the observations of the driving behavior controlled by a simple controller, a more complex controller is needed that blocks strong "wobble" and compensates instead by precise steering movements.
+
+A pure P-controller like the simple controller was already able to drive the track with confidence. In addition, the proportional part made the steering movements smoother than with just a constant factor. However, the vehicle only countersteers as soon as the error occurs from the other direction (vehicle must now approach the center point from the other side). The differential component counteracts overshooting of the track center point, thus reducing extreme overshoots. If, for example, further deviations or distortions occur due to the environment, which prevent the vehicle from reaching the center point, these influences must be taken into account. The integrating controller monitors the sum of all errors and influences the steering movement unless the errors have been reduced during a certain time.
+
+The parameters for the P and PD controllers can still be determined manually, since a maximum of two parameters have to be adjusted to each other. With three parameters, however, the adaptation effort increases, which is why the use of automated processes is the logical consequence. There are different methods with advantages and disadvantages. The random search offers the possibility to test many parameter variations, which normally would not be tested in this form. In order to find meaningful parameters, however, many iterations must be executed, which also requires many resources. A method that does not have this problem is the Twiddle algorithm, which was also used in the context of this project. The twiddle algorithm makes it possible to calculate reasonable values for the proportional, integrating and differentiating parts using a simple implementation.
+
+The twiddle algorithm proceeds as follows: At the beginning, the parameters to be optimized are initially defined in a list and the corresponding deltas are set up. The parameters from the parameter list are adjusted one after the other. For this purpose, it is first examined whether an increase of the current parameter results in a reduction of the error, which in this case corresponds to the deviation from the center of the track. If this is the case, the corresponding delta value is increased. Otherwise, the search is performed in the other direction by reducing the current parameter by twice the delta value and then recalculating the error. If there was an improvement with the reduced parameter value, the delta value is increased. Otherwise, the delta values and parameter values are updated. To determine the parameters of the PID controller, the combinations were tested to a maximum of 5000 steps using the twiddle method.
+
+Due to the simulation environment, we were not able to optimize the parameters outside the simulation, which is why we were bound to the real-time simulation and could not "fast-forward" the optimization.
+
+The final parameters are:
+| Method | Value |
+| ----------- | ----------- |
+| Simple Lane Detection | Constant steering angle: 0.75 |
+| PID-Controller | P: 0.65003358 I: 0.00000002 D: 0.03399347 |
+
+**Examples of Runs of the Simplified Procedure**
+
+[](https://youtu.be/WlGPokl0ZK8)
+
+(Video) Simulation with the simple method over 10k steps.
+
+
+
+
+
+Error depending on the time step for the simple method. It can be seen from step 1000 that the error occurs cyclically in similarity to the sinusoid. This can be explained by the change from side relative to the center of the detected trace. The straight sections connecting the curves show the moments when no trace was detected. Here, false detections occur again and again due to the small white lines in the spaces.
+
+
+**Examples of Runs of the PID Controller**
+
+[](https://youtu.be/YVZafMtOxzY)
+
+
+(Video) Simulation with a PID controller over 10k steps.
+
+
+
+
+
+Error depending on the time step for the PID controller. Larger deviations occur at the beginning, which, however, do not occur from step 2000 onwards. After that the controller shows a constant stable deviation to the center point.
+
+
+## Closing Words
+
+The vision of self-driving vehicles poses a great challenge both in science and in practice, which is why there is a great deal of interest in this topic, even from many newcomers. Accordingly, there are some sources on the Internet and in the literature about this, which have already implemented similar projects. Furthermore, there are also different approaches. For example, some projects encode the images in HSL or HSV color space, whereas other projects leave it at RGB encoding. It should be mentioned that we did not "invent" the performed transformation steps during the track detection, rather we tested different combinations of approaches and parameters that are most suitable for our specific use case.
+
+## Sources
+- https://towardsdatascience.com/lane-detection-with-deep-learning-part-1-9e096f3320b7
+- https://www.analyticsvidhya.com/blog/2020/05/tutorial-real-time-lane-detection-opencv/
+
+[Go back to table of contents](#table-of-contents)
diff --git a/src/Active-Lane-Keeping-Assistant/agent.py b/src/Active-Lane-Keeping-Assistant/agent.py
new file mode 100644
index 0000000000..091c3df28d
--- /dev/null
+++ b/src/Active-Lane-Keeping-Assistant/agent.py
@@ -0,0 +1,198 @@
+from __future__ import annotations
+
+import matplotlib.pyplot as plt
+import numpy as np
+import os
+
+class Agent():
+ """Agent that outputs the desired behaviour given
+ """
+
+ def __init__(self, tau_p:float = 0, tau_d:float = 0, tau_i:float = 0,
+ surface_lower_threshold:float = 20e6, throttle:float = 0.3,
+ suface_upper_threshold=30e6, controller:str = 'simple') -> None:
+ """Constructor
+
+ Args:
+ tau_p (float, optional): Parameter for the proportional part of the
+ PID-Controller. Defaults to 0.
+ tau_d (float, optional): Parameter for the derivative part of the
+ PID-Controller. Defaults to 0.
+ tau_i (float, optional): Parameter for the integral part of the
+ PID-Controller. Defaults to 0.
+ surface_lower_threshold (float, optional): Minimum amount of surface
+ that has to be detected in order to calculate new output. This
+ helps to find suitable lanes. Defaults to 20e6.
+ throttle (float, optional): Throttle to return at each time step.
+ Defaults to 0.3.
+ suface_upper_threshold (_type_, optional): Maximum amount of surface
+ that has to be detected in order to calculate new output. This
+ helps to find suitable lanes. Defaults to 30e6.
+ controller (str, optional): Identifies the controller to be used.
+ This can be one of the following:
+ - 'simple': hard coded controller that does not use any of
+ the tau parameters
+ - 'p': controller that only uses the proportional part
+ - 'pd': controller that only uses the proportional and
+ derivative part
+ - 'pid': pid-controller
+ Defaults to 'simple'.
+ """
+ self.tau_p = tau_p
+ self.tau_d = tau_d
+ self.tau_i = tau_i
+ self.surface_lower_threshold = surface_lower_threshold
+ self.surface_upper_threshold = suface_upper_threshold
+ self.prev_error = None
+ self.throttle = throttle
+ self.func = None
+ self.errors = []
+ self.controller_name = controller
+ self._select_controller_method(name=self.controller_name)
+
+ def _select_controller_method(self, name:str) -> None:
+ """Helper method to select the controller
+
+ Args:
+ name (str): Name of the controller.
+
+ Raises:
+ Exception: If the given name does not map to a controller method.
+ """
+ name = name.lower()
+ if name == 'simple':
+ self.func = Agent._simple_controller
+ elif name == 'p':
+ self.func = self._p_controller
+ elif name == 'pd':
+ self.func = self._pd_controller
+ elif name == 'pid':
+ self.func = self._pid_controller
+ else:
+ raise Exception(f'Controller name \'{name}\' is not applicable.')
+
+ def check_surface_area(self, detection_surface_area:float) -> bool:
+ """Checks Surface Area
+
+ Args:
+ detection_surface_area (float): _description_
+
+ Returns:
+ bool: Whether the given surface area is not inbetween the selected
+ interval.
+ """
+ lower = (detection_surface_area < self.surface_lower_threshold)
+ upper = (detection_surface_area > self.surface_upper_threshold)
+ return lower or upper
+
+ def show_error(self) -> None:
+ """Displays Error
+
+ Displays the difference to the center of all past steps.
+ """
+ plt.figure(1)
+ plt.clf()
+ x = range(len(self.errors))
+ y = self.errors
+ plt.plot(x, y, 'g', label=self.controller_name)
+ plt.plot(x, np.zeros(len(self.errors)), 'r', label='baseline')
+ plt.xlabel('Step')
+ plt.ylabel('Difference to Center')
+ plt.legend()
+ plt.pause(1e-10)
+
+ def save_error_fig(self, path:str, id:str) -> None:
+ """Save the Figure of the Error Plot
+
+ Args:
+ path (str): Folder to place the file in.
+ id (str): Unique identifier to prevent overwriting files.
+ """
+ plt.figure(1)
+ file_name = os.path.join(path, f'{id}_error.jpg')
+ plt.savefig(file_name)
+
+ def get_actions(self, detection_surface_area:float,
+ error:float) -> tuple[float, float]:
+ """Retrieve action based on Observation
+
+ Args:
+ error (float): Difference to the center of the detected lane.
+ detection_surface_area (float): Detected surface area.
+
+ Returns:
+ tuple[float, float]:
+ [0]: Steering angle to use.
+ [1]: Throttle to apply.
+ """
+ if (self.check_surface_area(detection_surface_area)):
+ steer = 0
+ if len(self.errors) > 0:
+ self.errors.append(self.errors[-1])
+ else:
+ self.errors.append(0)
+ else:
+ self.errors.append(error)
+ steer = self.func(error=error)
+ return steer, self.throttle
+
+ @staticmethod
+ def _simple_controller(error:float) -> float:
+ """Hard Coded Controller
+
+ Args:
+ error (float): Difference to the center of the detected lane.
+
+ Returns:
+ float: Steering angle to use.
+ """
+ if (abs(error) < 0.1):
+ steer = 0
+ elif error > 0:
+ steer = -0.75
+ else:
+ steer = 0.75
+ return steer
+
+ def _p_controller(self, error:float) -> float:
+ """Proportional Controller
+
+ Args:
+ error (float): Difference to the center of the detected lane.
+
+ Returns:
+ float: Steering angle to use.
+ """
+ steer = - self.tau_p * error
+ return steer
+
+ def _pd_controller(self, error:float) -> float:
+ """Proportional and Derivative Controller
+
+ Args:
+ error (float): Difference to the center of the detected lane.
+
+ Returns:
+ float: Steering angle to use.
+ """
+ if len(self.errors) > 2:
+ self.prev_error = self.errors[-2]
+ else:
+ self.prev_error = self.errors[-1]
+
+ deviation = error - self.prev_error
+ steer = - self.tau_d * deviation/0.1 + self._p_controller(error)
+ return steer
+
+ def _pid_controller(self, error:float) -> float:
+ """PID-Controller
+
+ Args:
+ error (float): Difference to the center of the detected lane.
+
+ Returns:
+ float: Steering angle to use.
+ """
+ sum_error = sum(self.errors)
+ steer = - self.tau_i * sum_error + self._pd_controller(error)
+ return steer
\ No newline at end of file
diff --git a/src/Active-Lane-Keeping-Assistant/lane.py b/src/Active-Lane-Keeping-Assistant/lane.py
new file mode 100644
index 0000000000..53f6211ac6
--- /dev/null
+++ b/src/Active-Lane-Keeping-Assistant/lane.py
@@ -0,0 +1,512 @@
+from __future__ import annotations
+
+import cv2
+import matplotlib.pyplot as plt
+import numpy as np
+
+from os.path import join
+
+class Lane:
+ """Detects Lanes in a given Image
+ """
+
+ def __init__(self, height:int, width:int, save:bool=False,
+ save_folder:str=join('img', 'examples')) -> None:
+ """Constructor
+
+ Args:
+ height (int): Height of the Image.
+ width (int): Width of the Image.
+ save (bool, optional): Whether to save the Images that will be
+ generated during the process. Defaults to False.
+ save_folder (str, optional): Folder to save the Images. This only
+ applies if save is true. Defaults to join('img', 'examples').
+ """
+ self.save = save
+ self.save_folder = save_folder
+ self.img_height = height
+ self.img_width = width
+ self.margin = int((1/12) * self.img_width)
+ self.DEGREE = 2
+
+ # Variables to store for lines
+ self.x_left = None
+ self.x_right = None
+ self.y_left = None
+ self.y_right = None
+
+ # Variables for fill
+ self.x_left_fill = None
+ self.x_right_fill = None
+ self.y_left_fill = None
+ self.y_right_fill = None
+
+ def get_lines(self, img:np.ndarray) -> np.ndarray:
+ """Get white Lines
+
+ Filters out lighter parts of the image.
+
+ Args:
+ img (np.ndarray): Image on which the changes are to be applied.
+
+ Returns:
+ np.ndarray: Image with changes applied.
+ """
+ hls = cv2.cvtColor(img, cv2.COLOR_BGR2HLS)
+
+ # Create threshold matrix to differentiate black and white based on the
+ # lightness (of HSL).
+ _, binary = cv2.threshold(hls[:, :, 1], 150, 255, cv2.THRESH_BINARY)
+ binary_blured = cv2.GaussianBlur(binary, (3, 3), 0)
+
+ if self.save:
+ cv2.imwrite(join(self.save_folder, 'lines.jpg'), binary_blured)
+
+ return binary_blured
+
+ def extract_roi(self, img:np.ndarray, bottom:int=350, top:int=260,
+ left:int=200, right:int=440) -> tuple[np.ndarray, np.ndarray]:
+ """Extract the Region of Interest
+
+ Extracts the region of interest and transforms it to fit the original
+ image size.
+
+ Args:
+ img (np.ndarray): Image on which the changes are to be applied.
+ Should generally be returned by get_lines.
+ bottom (int, optional): Bottom of the trapezoid. Defaults to 350.
+ top (int, optional): Top of the trapezoid. Defaults to 260.
+ left (int, optional): Top-Left of the trapezoid. Defaults to 200.
+ right (int, optional): Top-Right of the trapezoid. Defaults to 440.
+
+ Returns:
+ np.ndarray: Image with changes applied.
+ tuple[np.ndarray, np.ndarray]:
+ [0]: Image showing the warped perspective.
+ [1]: Matrix representing the inverse operation to undo the
+ transformation.
+ """
+ # Corners of the trapezoidal relevant area.
+ roi_points = np.float32([
+ [left, top],
+ [0, bottom], # Bottom-Left
+ [self.img_width, bottom], # Bottom-Right
+ [right, top]
+ ])
+
+ # Padding to apply on the transformed image.
+ padding = int(0.25 * self.img_width)
+ # Corners of the trapezoidal relevant area after transformation.
+ desired_roi_points = np.float32([
+ [padding, 0], # Top-left corner
+ [padding, self.img_height], # Bottom-left corner
+ [self.img_width-padding, self.img_height], # Bottom-right corner
+ [self.img_width-padding, 0] # Top-right corner
+ ])
+
+ perspective_transform = cv2.getPerspectiveTransform(roi_points,
+ desired_roi_points)
+ inverse_perspective_transform = cv2 \
+ .getPerspectiveTransform(desired_roi_points, roi_points)
+ perspective = cv2.warpPerspective(img, perspective_transform,
+ (self.img_width, self.img_height), flags=cv2.INTER_LINEAR)
+ _, warped_perspective_binary = cv2.threshold(perspective, 127, 255,
+ cv2.THRESH_BINARY)
+
+ if self.save:
+ COLOR_TRAPEZE = (147,20,255)
+ warped_perspective = cv2.polylines(warped_perspective_binary.copy(),
+ np.int32([desired_roi_points]), True, COLOR_TRAPEZE, 3)
+ cv2.imwrite(join(self.save_folder, 'warped_perspective.jpg'),
+ warped_perspective)
+
+ return warped_perspective_binary, inverse_perspective_transform
+
+ def get_hist(self,
+ img:np.ndarray) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
+ """Generates Historgramm of light Parts inside the image.
+
+ Args:
+ img (np.ndarray): Image on which the changes are to be applied.
+ Should generally be returned by extract_roi.
+
+ Returns:
+ tuple[np.ndarray, np.ndarray, np.ndarray]:
+ [0]: Values of the histogram.
+ [1]: Width-index of the location of the left lane.
+ [2]: Width-index of the location of the right lane.
+ """
+ hist = np.sum(img[int(img.shape[0]/2):,:], axis=0)
+ midpoint = np.int(hist.shape[0]/2)
+ left_lane = np.argmax(hist[:midpoint])
+ right_lane = np.argmax(hist[midpoint:]) + midpoint
+
+ if self.save:
+ _, (ax1, ax2) = plt.subplots(2, figsize=(8,8))
+ plt.subplots_adjust(hspace=0.25)
+ ax1.imshow(img, cmap='gray', aspect='auto')
+ ax1.set_title('Transformation of the Binary Representation')
+ ax2.plot(hist)
+ ax2.set_title('Histogram of the Deflections')
+ plt.savefig(join(self.save_folder, 'warped_hist.jpg'))
+
+ return hist, left_lane, right_lane
+
+ def get_line_fits(self, img:np.ndarray, max_left_idx:np.ndarray,
+ max_right_idx:np.ndarray,
+ number_windows:int=10) -> tuple[np.ndarray, np.ndarray]:
+ """Apply Sliding Windows and Retrieve Lines
+
+ Args:
+ img (np.ndarray): Image on which the changes are to be applied.
+ Should generally be the original image.
+ max_left_idx (np.ndarray): Index of the detected left line as
+ returned by get_hist.
+ max_right_idx (np.ndarray): Index of the detected right line as
+ returned by get_hist.
+ number_windows (int, optional): Number of windows to create.
+ Defaults to 10.
+
+ Returns:
+ tuple[np.ndarray, np.ndarray]:
+ [0]: Second order polynomial curve to represent the left line.
+ [1]: Second order polynomial curve to represent the right line.
+ """
+ # Number of pixels to identify line.
+ MINIMAL_AREA = int((1/24) * self.img_width)
+ WINDOW_HEIGHT = np.int(img.shape[0]/number_windows)
+
+ # Retrive x and y coordinates of white pixels (non-zero).
+ nonzero_y, nonzero_x = img.nonzero()
+
+ # Store the pixel indices for the left and right lane lines
+ left_lane_idxs = []
+ right_lane_idxs = []
+
+ WINDOW_COLOR = (255,255,255)
+ WINDOW_THICKNESS = 2
+
+ for window in range(number_windows):
+ y_low = img.shape[0] - (window + 1) * WINDOW_HEIGHT
+ y_high = img.shape[0] - window * WINDOW_HEIGHT
+ x_low_left = max_left_idx - self.margin
+ x_high_left = max_left_idx + self.margin
+ x_low_right = max_right_idx - self.margin
+ x_high_right = max_right_idx + self.margin
+
+ # Draw windows.
+ cv2.rectangle(img, (x_low_left, y_low), (x_high_left, y_high),
+ WINDOW_COLOR, WINDOW_THICKNESS)
+ cv2.rectangle(img, (x_low_right, y_low),(x_high_right, y_high),
+ WINDOW_COLOR, WINDOW_THICKNESS)
+
+ # Find the white pixels in x and y inside the window.
+ non_zero_left_idxs = ((nonzero_y >= y_low) & (nonzero_y < y_high)
+ & (nonzero_x >= x_low_left) & (nonzero_x < x_high_left)) \
+ .nonzero()[0]
+ non_zero_right_idxs = ((nonzero_y >= y_low) & (nonzero_y < y_high)
+ & (nonzero_x >= x_low_right) & (nonzero_x < x_high_right)) \
+ .nonzero()[0]
+ left_lane_idxs.append(non_zero_left_idxs)
+ right_lane_idxs.append(non_zero_right_idxs)
+
+ # Recenter next window on mean position.
+ if len(non_zero_left_idxs) > MINIMAL_AREA:
+ max_left_idx = np.int(np.mean(nonzero_x[non_zero_left_idxs]))
+ if len(non_zero_right_idxs) > MINIMAL_AREA:
+ max_right_idx = np.int(np.mean(nonzero_x[non_zero_right_idxs]))
+
+ left_lane_idxs = np.concatenate(left_lane_idxs)
+ right_lane_idxs = np.concatenate(right_lane_idxs)
+
+ # If valid lane has been found, update values for next image. This
+ # ensures that if no lane has been found, the detection within the next
+ # image are still possible. However, this requires to detect a lane
+ # within the first image after the lane object has been created.
+ is_invalid = (sum(nonzero_x[left_lane_idxs]) == 0) or \
+ (sum(nonzero_y[left_lane_idxs]) == 0)
+ is_false = (nonzero_x[left_lane_idxs].size == 0) or \
+ (nonzero_x[left_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.x_left = nonzero_x[left_lane_idxs].copy()
+
+ is_false = (nonzero_y[left_lane_idxs].size == 0) or \
+ (nonzero_y[left_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.y_left = nonzero_y[left_lane_idxs].copy()
+
+ is_invalid = (sum(nonzero_x[right_lane_idxs]) == 0) or \
+ (sum(nonzero_y[right_lane_idxs]) == 0)
+ is_false = (nonzero_x[right_lane_idxs].size == 0) or \
+ (nonzero_x[right_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.x_right = nonzero_x[right_lane_idxs].copy()
+
+ is_false = (nonzero_y[right_lane_idxs].size == 0) or \
+ (nonzero_y[right_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.y_right = nonzero_y[right_lane_idxs].copy()
+
+ # Fit a second order polynomial curve to represent the lines.
+ left_line = np.polyfit(self.y_left, self.x_left, self.DEGREE)
+ right_line = np.polyfit(self.y_right, self.x_right, self.DEGREE)
+
+ # Retrieve x and y coordinates by applying the ploynomical curves.
+ y = np.linspace(0, img.shape[0]-1, img.shape[0])
+ left_fit_x = left_line[0]*y**2 + left_line[1]*y + \
+ left_line[2]
+ right_fit_x = right_line[0]*y**2 + right_line[1]*y + \
+ right_line[2]
+
+ if self.save:
+ out_img = np.dstack((img, img, img)) * 255
+
+ left_lane_idxs = ((nonzero_x > (left_line[0]*(nonzero_y**2) +
+ left_line[1]*nonzero_y + left_line[2] - self.margin)) &
+ (nonzero_x < (left_line[0]*(nonzero_y**2) +
+ left_line[1]*nonzero_y + left_line[2] + self.margin)))
+ right_lane_idxs = ((nonzero_x > (right_line[0]*(nonzero_y**2) +
+ right_line[1]*nonzero_y + right_line[2] - self.margin)) &
+ (nonzero_x < (right_line[0]*(nonzero_y**2) +
+ right_line[1]*nonzero_y + right_line[2] + self.margin)))
+
+ # Apply color to the left and right line pixels.
+ out_img[nonzero_y[left_lane_idxs], nonzero_x[left_lane_idxs]] = \
+ [255, 0, 0]
+ out_img[nonzero_y[right_lane_idxs], nonzero_x[right_lane_idxs]] = \
+ [0, 0, 255]
+
+ figure, (ax1, ax2) = plt.subplots(2)
+ figure.tight_layout(pad=3.0)
+ ax1.imshow(img, cmap='gray')
+ ax2.imshow(out_img)
+ ax2.plot(left_fit_x, y, color='yellow')
+ ax2.plot(right_fit_x, y, color='yellow')
+ ax1.set_title('Sliding Windows of the Transformed Image')
+ ax2.set_title('Detected Track Lines')
+ plt.savefig(join(self.save_folder, 'sliding_windows.jpg'))
+
+ return left_line, right_line
+
+ def get_search_window(self, img:np.ndarray, left_line:np.ndarray,
+ right_line:np.ndarray) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
+ """Retrieve the Search Window for Lines
+
+ Args:
+ img (np.ndarray): Image on which the changes are to be applied.
+ Should generally be the original image.
+ left_line (np.ndarray): Second order polynomial curve to represent
+ the left line.
+ right_line (np.ndarray): Second order polynomial curve to represent
+ the right line.
+
+ Returns:
+ tuple[np.ndarray, np.ndarray, np.ndarray]:
+ [0]: Values returned by applying the polynomial curve to using
+ the y values for the left line.
+ [1]: Values returned by applying the polynomial curve to using
+ the y values for the right line.
+ [2]: Generated y values to get a continuous look.
+ """
+ # Retrive x and y coordinates of white pixels (non-zero).
+ nonzero_y, nonzero_x = img.nonzero()
+
+ # Get indices for he left and right line pixels.
+ left_lane_idxs = ((nonzero_x > (left_line[0]*(nonzero_y**2) +
+ left_line[1]*nonzero_y + left_line[2] - self.margin)) &
+ (nonzero_x < (left_line[0]*(nonzero_y**2) + left_line[1]*nonzero_y +
+ left_line[2] + self.margin)))
+ right_lane_idxs = ((nonzero_x > (right_line[0]*(nonzero_y**2) +
+ right_line[1]*nonzero_y + right_line[2] - self.margin)) &
+ (nonzero_x < (right_line[0]*(nonzero_y**2) +
+ right_line[1]*nonzero_y + right_line[2] + self.margin)))
+
+ # If valid lane has been found, update values for next image. This
+ # ensures that if no lane has been found, the detection within the next
+ # image are still possible. However, this requires to detect a lane
+ # within the first image after the lane object has been created.
+ is_invalid = (sum(nonzero_x[left_lane_idxs]) == 0) or \
+ (sum(nonzero_y[left_lane_idxs]) == 0)
+ is_false = (nonzero_x[left_lane_idxs].size == 0) or \
+ (nonzero_x[left_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.x_left_fill = nonzero_x[left_lane_idxs].copy()
+
+ is_false = (nonzero_y[left_lane_idxs].size == 0) or \
+ (nonzero_y[left_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.y_left_fill = nonzero_y[left_lane_idxs].copy()
+
+ is_invalid = (sum(nonzero_x[right_lane_idxs]) == 0) or \
+ (sum(nonzero_y[right_lane_idxs]) == 0)
+ is_false = (nonzero_x[right_lane_idxs].size == 0) or \
+ (nonzero_x[right_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.x_right_fill = nonzero_x[right_lane_idxs].copy()
+
+ is_false = (nonzero_y[right_lane_idxs].size == 0) or \
+ (nonzero_y[right_lane_idxs] is None)
+ if not (is_false or is_invalid):
+ self.y_right_fill = nonzero_y[right_lane_idxs].copy()
+
+ # Fit a second order polynomial curve to represent the outer lines.
+ left_fit = np.polyfit(self.y_left_fill, self.x_left_fill, self.DEGREE)
+ right_fit = np.polyfit(self.y_right_fill, self.x_right_fill,
+ self.DEGREE)
+
+ # Retrieve x and y coordinates by applying the ploynomical curves.
+ y = np.linspace(0, img.shape[0]-1, img.shape[0])
+ left_fit_x = left_fit[0]*y**2 + left_fit[1]*y + left_fit[2]
+ right_fit_x = right_fit[0]*y**2 + right_fit[1]*y + right_fit[2]
+
+ if self.save:
+ out_img = np.dstack((img, img, img))*255
+ window_img = np.zeros_like(out_img)
+
+ # Apply color to the left and right line pixels.
+ out_img[nonzero_y[left_lane_idxs], nonzero_x[left_lane_idxs]] = \
+ [255, 0, 0]
+ out_img[nonzero_y[right_lane_idxs], nonzero_x[right_lane_idxs]] = \
+ [0, 0, 255]
+
+ # Create a polygon to represent the search window area and convert
+ # the x and y points into a usable format for cv2.fillPoly().
+ left_line_window1 = np.array([np.transpose(
+ np.vstack([left_fit_x-self.margin, y]))])
+ left_line_window2 = np.array([np.flipud(np.transpose(
+ np.vstack([left_fit_x+self.margin, y])))])
+ left_line_pts = np.hstack((left_line_window1, left_line_window2))
+ right_line_window1 = np.array([np.transpose(
+ np.vstack([right_fit_x-self.margin, y]))])
+ right_line_window2 = np.array([np.flipud(np.transpose(
+ np.vstack([right_fit_x+self.margin, y])))])
+ right_line_pts = np.hstack((right_line_window1, right_line_window2))
+
+ # Draw the lane onto the warped blank image
+ cv2.fillPoly(window_img, np.int_([left_line_pts]), (0, 255, 0))
+ cv2.fillPoly(window_img, np.int_([right_line_pts]), (0, 255, 0))
+ result = cv2.addWeighted(out_img, 1, window_img, 0.3, 0)
+
+ _, ax= plt.subplots(1)
+ ax.imshow(result)
+ ax.plot(left_fit_x, y, color='yellow')
+ ax.plot(right_fit_x, y, color='yellow')
+ ax.set_title('Search Window on the Transformed Image')
+ plt.savefig(join(self.save_folder, 'search_window.jpg'))
+
+ return left_fit_x, right_fit_x, y
+
+ def show_lane(self, orig_img:np.ndarray, img:np.ndarray,
+ left_fit_x:np.ndarray, right_fit_x:np.ndarray, y:np.ndarray,
+ inverse_perspective_transform:np.ndarray) -> tuple[np.ndarray, int]:
+ """Draw Lane on original Image
+
+ Args:
+ orig_img (np.ndarray): Original image before any changes.
+ img (np.ndarray): Image after warping.
+ left_fit_x (np.ndarray): Values returned by applying the polynomial
+ curve to using the y values for the left line.
+ right_fit_x (np.ndarray): Values returned by applying the polynomial
+ curve to using the y values for the right line.
+ y (np.ndarray): Generated y values to get a continuous look.
+ inverse_perspective_transform (np.ndarray): Matrix representing the
+ inverse operation to undo the transformation.
+
+ Returns:
+ tuple[np.ndarray, int]:
+ [0]: Original image with lane drawn on.
+ [1]: Detected surface area multiplied by 255.
+ """
+
+ # Generate image to draw on.
+ warp_zero = np.zeros_like(img).astype(np.uint8)
+ color_warp = np.dstack((warp_zero, warp_zero, warp_zero))
+
+ # Converting the x and y points into a usable format for cv2.fillPoly().
+ pts_left = np.array([np.transpose(np.vstack([left_fit_x, y]))])
+ pts_right = np.array([np.flipud(np.transpose(
+ np.vstack([right_fit_x, y])))])
+ pts = np.hstack((pts_left, pts_right))
+
+ # Draw lane on warped image.
+ cv2.fillPoly(color_warp, np.int_([pts]), (0, 255, 0))
+
+ # Undo warp transformation.
+ newwarp = cv2.warpPerspective(color_warp, inverse_perspective_transform,
+ (orig_img.shape[1], orig_img.shape[0]))
+
+ # Draw lane on original image.
+ result = cv2.addWeighted(orig_img, 1, newwarp, 0.3, 0)
+
+ if self.save:
+ _, ax = plt.subplots(1)
+ ax.imshow(cv2.cvtColor(result, cv2.COLOR_BGR2RGB))
+ ax.set_title('Original Image with Recognised Track')
+ plt.savefig(join(self.save_folder, 'lane_overlay.jpg'))
+
+ return result, np.sum(color_warp)
+
+ def get_car_position(self, img:np.ndarray, left_line:np.ndarray,
+ right_line:np.ndarray) -> tuple[np.ndarray, float]:
+ """Retrieve the Error
+
+ Args:
+ img (np.ndarray): Original image with lane drawn on.
+ left_line (np.ndarray): Second order polynomial curve to represent
+ the left line.
+ right_line (np.ndarray): Second order polynomial curve to represent
+ the right line.
+
+ Returns:
+ tuple[np.ndarray, float]:
+ [0]: Original image with lane and offset drawn on.
+ [1]: Calculated offset to the center of the detected lane.
+ """
+ M_PER_PIX = 3.7 / 781 # Meters per pixel
+
+ # Retrieve the position of the car assuming it is the center of the
+ # image.
+ car_location = self.img_width / 2
+
+ # Retrieve x coordinates of bottom from the lane.
+ bottom_left = left_line[0]*self.img_height**2 + \
+ left_line[1]*self.img_height + left_line[2]
+ bottom_right = right_line[0]*self.img_height**2 + \
+ right_line[1]*self.img_height + right_line[2]
+
+ center_lane = (bottom_right - bottom_left)/2 + bottom_left
+ center_offset = (np.abs(car_location) - np.abs(center_lane)) * \
+ M_PER_PIX * 100
+
+ cv2.putText(img, (f'Difference to Center: '
+ f'{str(round(center_offset, 5))} cm'), (int((5/600)*self.img_width),
+ int((20/338)*self.img_height)), cv2.FONT_HERSHEY_SIMPLEX,
+ (float((0.5/600)*self.img_width)), (255,255,255),2,cv2.LINE_AA)
+
+ return img, center_offset
+
+ def pipe(self, img:np.ndarray) -> tuple[np.ndarray, float, int]:
+ """Pipes Image through every Step in right order.
+
+ Args:
+ img (np.ndarray): Original image to detect lane on.
+
+ Returns:
+ tuple[np.ndarray, float, int]:
+ [0]: Original image with lane and offset drawn on.
+ [1]: Calculated offset to the center of the detected lane.
+ [2]: Detected surface area multiplied by 255.
+ """
+ original_image = img.copy()
+ img = self.get_lines(img)
+ img, inverse_perspective_transform = self.extract_roi(img)
+ _, max_left_idx, max_right_idx = self.get_hist(img)
+ left_line, right_line = self.get_line_fits(img.copy(),
+ max_right_idx, max_left_idx)
+ left_fit_x, right_fit_x, y = self.get_search_window(img.copy(),
+ left_line, right_line)
+ img, detection_surface_area = self.show_lane(original_image,
+ img.copy(), left_fit_x, right_fit_x, y,
+ inverse_perspective_transform)
+ img, error = self.get_car_position(img, left_line, right_line)
+ return img, error, detection_surface_area
\ No newline at end of file
diff --git a/src/Active-Lane-Keeping-Assistant/main.py b/src/Active-Lane-Keeping-Assistant/main.py
new file mode 100644
index 0000000000..1b20920936
--- /dev/null
+++ b/src/Active-Lane-Keeping-Assistant/main.py
@@ -0,0 +1,297 @@
+from __future__ import annotations
+
+import argparse
+import logging
+import numpy as np
+import os
+import warnings
+
+from itertools import count
+from os.path import join
+from typing import Iterable
+
+from agent import Agent
+from recorder import Recorder
+from world import World
+
+def log_parameters(id:str, logger:logging.Logger, parameters: Iterable[float],
+ delta_parameters: Iterable[float], error:float, steps:int) -> None:
+ """Log one set of Parameters
+
+ Args:
+ id (str): An unique identifier used to identify the run.
+ logger (logging.Logger): Logger that writes logs.
+ parameters (Iterable[float]): Parameters of the controller.
+ delta_parameters (Iterable[float]): Delta parameters for adaption of the
+ parameters.
+ error (float): Summed error received by the environment.
+ steps (int): Number of steps taken within the run.
+ """
+ logger.info((f'id: {id} | parameters: {parameters} | delta_parameters: '
+ f'{delta_parameters} | error: {error:.5f} | steps: {steps:5d}'))
+
+def run(id:str, path:str, world:World, agent:Agent, steps:int = 100,
+ save:bool=False) -> tuple[float, int]:
+ """One Epsisode inside the World
+
+ Performs one run inside the world until the maximum steps is reached or a
+ collision was detected.
+
+ Args:
+ id (str): An unique identifier used to identify the run.
+ path (str): Path where all files for a run will be stored.
+ world (World): Environment for the simulation.
+ agent (Agent): Agent to decide on actions.
+ steps (int, optional): Number of maximum amount of steps to take until
+ termination. Defaults to 100.
+ save (bool, optional): Whether to create a video of the run. Defaults to
+ False.
+
+ Returns:
+ tuple[float, int]:
+ [0]: Sum of errors as returned by the world.
+ [1]: Number of steps taken.
+ """
+ error, detection_surface_area, _, _ = world.reset()
+
+ if save:
+ recorder = Recorder(folder=path)
+ recorder.init_new_video(id=id)
+
+ for i in range(steps):
+ steer, throttle = agent.get_actions(detection_surface_area, error)
+ error, detection_surface_area, img, collision_detected = world.step(
+ steer=steer, throttle=throttle)
+ agent.show_error()
+
+ if save:
+ recorder.add_image(img=img)
+
+ if collision_detected:
+ agent.errors = agent.errors + [max(agent.errors)] * (steps-i)
+ agent.show_error()
+ break
+
+ agent.save_error_fig(path, id)
+ if save:
+ recorder.close_recording()
+
+ return sum(agent.errors), i
+
+def check_supported_controller(name:str) -> str:
+ """Check supported controller
+
+ Checks if the provided controller string is allowed for the adaption mode.
+
+ Args:
+ name (str): Identifies the controller to be used for the run.
+
+ Returns:
+ str: Identifies the controller to be used for the run. This may be
+ different from the input name.
+ """
+ if name.lower() != 'pid':
+ warnings.warn(('Twiddle does not support other controllers than '
+ '\'pid\'. Setting controller to \'pid\''))
+ return 'pid'
+ return name
+
+def get_logger(id:str, path:str, debug:bool=False) -> logging.Logger:
+ """Retrieve logger
+
+ The returned logger 'pacman_rl' writes to a file inside the 'logs' folder
+ and streams the logs to the console.
+
+ Args:
+ id (str): An unique identifier used to identify the log file.
+ path (str): Path where all files for a run will be stored.
+ debug (bool, optional): Whether to use the debug mode for logging. If
+ False, only infos will be logged. Defaults to False.
+
+ Returns:
+ logging.Logger: Logger that streams and writes logs.
+ """
+ if debug:
+ level = logging.DEBUG
+ else:
+ level = logging.INFO
+
+ formatter = logging.Formatter('%(asctime)s [%(levelname)s] %(message)s')
+ logger = logging.getLogger('active_lane_keeping_assistant')
+ logger.setLevel(level=level)
+ fh = logging.FileHandler(join(path, f'alka_{id}.log'), mode='a')
+ sh = logging.StreamHandler()
+ sh.setFormatter(formatter)
+ fh.setFormatter(formatter)
+ logger.addHandler(fh)
+ logger.addHandler(sh)
+
+ return logger
+
+def twiddle(id:str, path:str, tolerance:float=0.2, controller:str='pid',
+ steps:int=100, debug:bool=False) -> None:
+ """Twiddle-Algorithm
+
+ Perform the twiddle algorithm to receive near optimal parameters for the
+ PID-controller.
+
+ Args:
+ id (str): An unique identifier used to identify the run.
+ path (str): Path where all files for a run will be stored.
+ tolerance (float, optional): Value that must be undershot for the
+ algorithm to be terminated. Defaults to 0.2.
+ controller (str, optional): Identifies the controller to be used for the
+ run. Defaults to 'pid'.
+ steps (int, optional): Number of maximum amount of steps to take until
+ termination. Defaults to 100.
+ debug (bool, optional): Whether to use the debug mode for logging.
+ Defaults to False.
+ """
+ logger = get_logger(id=id, path=path, debug=debug)
+ controller = check_supported_controller(name=controller)
+
+ logger.info(f'twiddle: {id}')
+ logger.info((f'tolerance: {tolerance} | controller: {controller} | steps: '
+ f'{steps}'))
+
+ parameters = np.zeros(3)
+ delta_parameters = np.ones(3)
+
+ world = World()
+ agent = Agent(controller=controller, tau_p=parameters[0],
+ tau_i=parameters[1], tau_d=parameters[2])
+
+ best_err, steps_taken = run(id, path, world, agent, steps, save=False)
+ best_parameters = parameters
+ best_id = id
+
+ log_parameters(id, logger, parameters, delta_parameters, best_err,
+ steps_taken)
+
+ for t in count():
+ if (sum(delta_parameters) < tolerance):
+ break
+ for i in range(len(parameters)):
+ new_id = f'{id}_{t}_{i}_a'
+ world.id = new_id
+ parameters[i] += delta_parameters[i]
+
+ agent = Agent(controller=controller, tau_p=parameters[0],
+ tau_i=parameters[1], tau_d=parameters[2])
+ error, steps_taken = run(new_id, path, world, agent, steps)
+ log_parameters(new_id, logger, parameters, delta_parameters, error,
+ steps_taken)
+
+ if abs(error) < abs(best_err):
+ logger.debug(f'Follow same direction for parameter {i}.')
+ best_err = error
+ best_parameters = parameters
+ best_id = new_id
+ delta_parameters[i] *= 1.1
+ else:
+ logger.debug(f'Trying opposite direction for parameter {i}.')
+ new_id = f'{id}_{t}_{i}_a'
+ world.id = new_id
+ parameters[i] -= 2*delta_parameters[i]
+
+ agent = Agent(controller=controller,
+ tau_p=parameters[0], tau_i=parameters[1],
+ tau_d=parameters[2])
+ error, steps_taken = run(new_id, path, world, agent, steps)
+ log_parameters(new_id, logger, parameters, delta_parameters,
+ error, steps_taken)
+
+ if abs(error) < abs(best_err):
+ logger.debug((f'Follow opposite direction for parameter '
+ f'{i}.'))
+ best_err = error
+ best_parameters = parameters
+ best_id = new_id
+ delta_parameters[i] *= 1.1
+ else:
+ logger.debug((f'Neither direction worked for {i}. '
+ f'Decreasing step size.'))
+ parameters[i] += delta_parameters[i]
+ delta_parameters[i] *= 0.9
+
+ logger.info((f'best run - id: {best_id} | parameters: {best_parameters} | '
+ f'error: {best_err:.5f}'))
+
+def no_adapt(id:str, path:str, steps:int=100, controller:str='simple') -> None:
+ """Run without adaption
+
+ Run the environment without the adaption of the parameters from the
+ controller.
+
+ Args:
+ id (str): Unique identifier for the run.
+ path (str): Path where all files for a run will be stored.
+ steps (int, optional): Number of steps to take within the environment.
+ Defaults to 100.
+ controller (str, optional): Identifies the controller to be used for the
+ run. Defaults to 'simple'.
+ """
+ world = None
+ try:
+ world = World(time_difference=0.05)
+ agent = Agent(controller=controller)
+ run(id=id, path=path, world=world, agent=agent, steps=steps, save=False)
+ finally:
+ if world is not None:
+ world.close()
+
+
+def make_path(folder:str, id:str) -> str:
+ """Create folder
+
+ Checks if a folder for the id already exists and creates the folder if
+ necessary.
+
+ Args:
+ folder (str): Name of the folder where the folder for the run will be
+ located.
+ id (str): Unique identifier for the run. This will be used to identify
+ the specific folder for this run.
+
+ Returns:
+ str: Path including the id of the run.
+ """
+ path = join(os.getcwd(), folder, id)
+
+ if not os.path.exists(path):
+ os.makedirs(path)
+ else:
+ warnings.warn('Path already exists. Files may be overwritten.')
+
+ return path
+
+if __name__ == '__main__':
+ parser = argparse.ArgumentParser(prog='Active Lane Keeping System',
+ description='')
+ parser.add_argument('-id', '--identifier', type=str, help=('Unique '
+ 'identifier used to identify the run.'), dest='id', required=True)
+ parser.add_argument('-c', '--controller', default='pid', type=str,
+ choices=['simple', 'p', 'pd', 'pid'], help=('The method used to '
+ 'control the car.'), dest='controller')
+ parser.add_argument('-s', '--steps', default=100, type=int, help=('The '
+ 'number of steps that the agent controls the car. This does not '
+ 'include the inital setup driving to the fourth lane.'), dest='steps')
+ parser.add_argument('-a', '--adapt', action='store_true',
+ help=('Whether to adapt the parameters. This only takes effect if not '
+ '\'simple\' is selected as controller.'), dest='adapt')
+ parser.add_argument('-t', '--tolerance', default=0.2, type=float,
+ help=('Tolerance range for the adaption of the parameters. This takes '
+ 'effect only if adapt is set to true.'), dest='tolerance')
+ parser.add_argument('-d', '--debug', action='store_true',
+ help=('Whether to use debug mode for logging. This takes effect only '
+ 'if adapt is set to true.'), dest='debug')
+ args = parser.parse_args()
+
+ path = make_path(folder='assets', id=args.id)
+
+ if args.adapt:
+ twiddle(id=args.id, path=path, tolerance=args.tolerance,
+ controller=args.controller, steps=args.steps, debug=args.debug)
+ else:
+ no_adapt(id=args.id, path=path, steps=args.steps,
+ controller=args.controller)
\ No newline at end of file
diff --git a/src/Active-Lane-Keeping-Assistant/recorder.py b/src/Active-Lane-Keeping-Assistant/recorder.py
new file mode 100644
index 0000000000..043a714df5
--- /dev/null
+++ b/src/Active-Lane-Keeping-Assistant/recorder.py
@@ -0,0 +1,63 @@
+import cv2
+import numpy as np
+
+from os.path import join
+
+class Recorder():
+ """The recorder records a run of a simulation and saves it as a video file
+ """
+
+ def __init__(self,
+ fps:int=30,
+ height:int=480,
+ width:int=640,
+ folder:str='videos') -> None:
+ """Constructor for the recorder
+
+ Args:
+ fps (int, optional): Number of fps for video clip. Defaults to 15.
+ height (int, optional): Height of video. Defaults to 480.
+ width (int, optional): Width of video. Defaults to 640.
+ folder (str, optional): Storage location. Defaults to 'videos'.
+ """
+
+ self.cameraCapture = cv2.VideoCapture(0)
+ self.height = height
+ self.width = width
+ self.fps = float(fps)
+ self.folder = folder
+
+ self.fourcc = cv2.VideoWriter_fourcc(*'mp4v')
+ self.video = None
+
+ def init_new_video(self,
+ id:str) -> None:
+ """Triggers the process of recording a new video
+
+ Args:
+ id (str): Id/name of the video file
+ """
+ self.video = cv2.VideoWriter(join(self.folder, f'{id}.mp4'),
+ self.fourcc,
+ self.fps,
+ (self.width, self.height))
+
+ def add_image(self,
+ img:np.ndarray) -> None:
+ """Appends a new image to the video
+
+ Args:
+ img (np.ndarray): Frame to append to the video.
+ """
+ self.video.write(img)
+
+ def close_recording(self) -> None:
+ """Stops the recording
+
+ Raises:
+ Exception:
+ If no recording has been initiated, the video can't be released.
+ """
+ if self.video is None:
+ raise Exception('No video initialized.')
+ self.video.release()
\ No newline at end of file
diff --git a/src/Active-Lane-Keeping-Assistant/requirements.txt b/src/Active-Lane-Keeping-Assistant/requirements.txt
new file mode 100644
index 0000000000..effa44305c
--- /dev/null
+++ b/src/Active-Lane-Keeping-Assistant/requirements.txt
@@ -0,0 +1,4 @@
+matplotlib==3.7.1
+numpy==1.20.0
+opencv-python==4.7.0.72
+carla==0.9.13
\ No newline at end of file
diff --git a/src/Active-Lane-Keeping-Assistant/world.py b/src/Active-Lane-Keeping-Assistant/world.py
new file mode 100644
index 0000000000..7f106f10ca
--- /dev/null
+++ b/src/Active-Lane-Keeping-Assistant/world.py
@@ -0,0 +1,196 @@
+from __future__ import annotations
+
+import carla
+import cv2
+import numpy as np
+import queue
+import random # <--- 新增:为了随机选择出生点
+
+from lane import Lane
+
+
+class World():
+ """Environment that wraps around the Carla-API
+ """
+
+ def __init__(self, server_ip: str = '127.0.0.1', port: int = 2000,
+ timeout: float = 5.0, map: str = 'Town04', image_height: int = 480,
+ image_width: int = 640, fov: int = 110,
+ time_difference: float = 0.01) -> None:
+ """Constructor
+ """
+
+ self.image_height = image_height
+ self.image_width = image_width
+
+ self.lane = Lane(height=self.image_height, width=self.image_width)
+
+ self.fov = fov
+
+ # 1. 连接客户端
+ self.client = carla.Client(server_ip, port)
+
+ # === 修复 1: 设置 30 秒超时,防止加载地图太慢报错 ===
+ self.client.set_timeout(30.0)
+ # ===============================================
+
+ # 2. 加载地图
+ self.world = self.client.load_world(map)
+
+ settings = self.world.get_settings()
+ settings.fixed_delta_seconds = time_difference
+ # Client and server work synchronously.
+ settings.synchronous_mode = True
+ self.world.apply_settings(settings)
+ self.blueprint_library = self.world.get_blueprint_library()
+
+ # Use Tesla Model 3 as Car
+ self.bp = self.blueprint_library.filter('model3')[0]
+
+ # === 修复 2: 随机选择出生点,防止车位被占导致无法生成 ===
+ spawn_points = self.world.get_map().get_spawn_points()
+ if spawn_points:
+ self.spawn_point = random.choice(spawn_points)
+ else:
+ self.spawn_point = carla.Transform() # 防止没有生成点的情况
+ # ====================================================
+
+ self.vehicle = None
+ self.sensor = None
+ self.collision_sensor = None
+
+ self.image_queue = queue.Queue()
+ self.collision_detected = False
+ self.initialized = False
+
+ def close(self) -> None:
+ """Destroys all currently used Actors
+ """
+ if self.vehicle is not None:
+ self.vehicle.destroy()
+ if self.sensor is not None:
+ self.sensor.destroy()
+ if self.collision_sensor is not None:
+ self.collision_sensor.destroy()
+
+ def reset(self) -> tuple[float, float, np.ndarray, bool]:
+ """Resets the Actors
+
+ Returns:
+ tuple[float, float, np.ndarray, bool]:
+ [0]: Difference to the center of the detected lane.
+ [1]: Detected surface area.
+ [2]: Image consisting including the detected surface area.
+ [3]: Whether a collision has been detected.
+ """
+ self.image_queue = queue.Queue()
+ self.close()
+
+ # 尝试生成车辆,如果失败(比如还是有碰撞),这行可能会报错,但有了随机点后概率大大降低
+ self.vehicle = self.world.spawn_actor(self.bp, self.spawn_point)
+
+ # Spawn camera
+ blueprint = self.blueprint_library.find('sensor.camera.rgb')
+ blueprint.set_attribute('image_size_x', f'{self.image_width}')
+ blueprint.set_attribute('image_size_y', f'{self.image_height}')
+ blueprint.set_attribute('fov', f'{self.fov}')
+
+ # Relative location to car
+ spawn_point = carla.Transform(carla.Location(x=2.5, z=0.7))
+ self.sensor = self.world.spawn_actor(blueprint, spawn_point,
+ attach_to=self.vehicle)
+ self.sensor.listen(self.image_queue.put)
+
+ # Spawn collision sensor
+ blueprint = self.world.get_blueprint_library() \
+ .find('sensor.other.collision')
+ self.collision_sensor = self.world.spawn_actor(blueprint,
+ carla.Transform(), attach_to=self.vehicle)
+ self.collision_detected = False
+ self.collision_sensor.listen(lambda e: {self._set_collision()})
+ self.initialized = True
+
+ return self._change_to_left_lane()
+
+ def _set_collision(self) -> None:
+ """Helper to set collision_detected to True.
+ """
+ self.collision_detected = True
+
+ def _change_to_left_lane(self) -> tuple[float, float, np.ndarray, bool]:
+ """Hard Code to make the Car start in the leftmost Lane
+
+ Returns:
+ tuple[float, float, np.ndarray, bool]:
+ [0]: Difference to the center of the detected lane.
+ [1]: Detected surface area.
+ [2]: Image consisting including the detected surface area.
+ [3]: Whether a collision has been detected.
+ """
+ for throttle, steer, steps in [
+ (0.2, -0.11, 850), (0.2, 0.17, 250), (-0.2, 0.17, 50)]:
+ for _ in range(steps):
+ error, detection_surface_area, transformed_image, _ = self.step(
+ throttle=throttle, steer=steer)
+
+ return error, detection_surface_area, transformed_image, \
+ self.collision_detected
+
+ def get_image(self) -> np.ndarray:
+ """Retrieve the Image in RGB
+
+ Returns:
+ np.ndarray: Image in RGB.
+ """
+ image = self.image_queue.get()
+ image = np.array(image.raw_data)
+ image = image.reshape((self.image_height, self.image_width, 4))
+ image = image[:, :, :3]
+ return image
+
+ @staticmethod
+ def show_image(image: np.ndarray) -> None:
+ """Display the Image
+
+ Args:
+ image (np.ndarray): Image to display.
+ """
+ cv2.imshow("", image)
+ cv2.waitKey(1)
+
+ def step(self, show: bool = True, throttle: float = 0,
+ steer: float = 0) -> tuple[float, float, np.ndarray, bool]:
+ """Simulate one Step
+
+ Args:
+ show (bool, optional): Whether to show the image of the car driving.
+ Defaults to True.
+ throttle (float, optional): Which throttle to apply. Defaults to 0.
+ steer (float, optional): Which steering to apply. Defaults to 0.
+
+ Raises:
+ Exception: Requires the reset method to be called before step.
+
+ Returns:
+ tuple[float, float, np.ndarray, bool]:
+ [0]: Difference to the center of the detected lane.
+ [1]: Detected surface area.
+ [2]: Image consisting including the detected surface area.
+ [3]: Whether a collision has been detected.
+ """
+
+ if not self.initialized:
+ raise Exception('Reset must be called before step.')
+
+ self.vehicle.apply_control(carla.VehicleControl(throttle=throttle,
+ steer=steer))
+ self.world.tick()
+ image = self.get_image()
+ transformed_image, error, detection_surface_area = self.lane \
+ .pipe(img=image)
+
+ if show:
+ World.show_image(image=transformed_image)
+
+ return error, detection_surface_area, transformed_image, \
+ self.collision_detected
\ No newline at end of file
From 91365a6f5281496b45101ced5407f47d9f8bd93b Mon Sep 17 00:00:00 2001
From: wutong88yuyu <2179751939@qq.com>
Date: Tue, 16 Dec 2025 21:42:43 +0800
Subject: [PATCH 2/5] =?UTF-8?q?=E8=BD=A6=E9=81=93=E4=BF=9D=E6=8C=81?=
=?UTF-8?q?=E8=BE=85=E5=8A=A9=E7=B3=BB=E7=BB=9F?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
---
src/Active-Lane-Keeping-Assistant/README.md | 261 +++++---------------
1 file changed, 56 insertions(+), 205 deletions(-)
diff --git a/src/Active-Lane-Keeping-Assistant/README.md b/src/Active-Lane-Keeping-Assistant/README.md
index ebd89fdb3e..fb471718cb 100644
--- a/src/Active-Lane-Keeping-Assistant/README.md
+++ b/src/Active-Lane-Keeping-Assistant/README.md
@@ -1,205 +1,56 @@
-# Active Lane Keeping Assistant
-
-> Please note that the code and contributions in this GitHub repository were created through a collaborative effort by [@Irish-77](https://github.com/Irish-77) and [@Ronho](https://github.com/Ronho). Both contributors acknowledge that their work is of equal merit and that they each have a deep understanding of the other's contributions.
-
-The aim of this project is to control a vehicle. Computer vision methods are used for lane recognition. The final control of the vehicle is done with the help of different variants of the PID controller. The variants are also investigated and compared.
-
-## Table of contents
-1. [Installation](#installation)
-1. [Description](#description)
- 1. [Introductory Questions](#introductory-questions)
- 1. [Computer Vision to Identify the Track](#computer-vision-to-identify-the-track)
- 1. [Control Technology for Tracking](#control-technology-for-tracking)
-1. [Closing Words](#closing-words)
-1. [Sources](#sources)
-
-## Installation
-
-First, the project must be cloned via git using the following command.
-```sh
-git clone https://github.com/C2G-BR/Active-Lane-Keeping-Assistant.git
-```
-To be able to test the project, you have to install [CARLA](https://github.com/carla-simulator/carla/releases) next. This project was created and tested with version 0.9.13 of CARLA.
-
-Additionally, a Python environment is required. Due to dependencies it is necessary to use Python 3.8. All necessary packages can be installed with the command
-```sh
-pip install -r requirements.txt
-```
-that must be executed inside the project folder.
-
-To run the program, CARLA must be started at the beginning. Afterwards a run can be started with
-```sh
-python src/main.py -id "test" -c pid -s 1000
-```
-within the project's folder. Within the CARLA interface, a car should now appear that starts to move. To find the vehicle, it may be necessary to look around within the world. All possible configuration settings can be viewed by
-```sh
-python main.py -h
-```
-If the error
-```sh
-RuntimeError: time-out of 5000ms while waiting for the simulator, make sure the simulator is ready and connected to 127.0.0.1:2000
-```
-occurs, it can help to execute the command multiple times and restart CARLA.
-
-## Description
-
-### Introductory Questions
-In order to implement the system, the following questions first arose:
-
-What metric can we use to obtain information about the current deviation from the track?
-
->Since the focus of this project is on image-based control of the vehicle, several transformation steps were applied to the image to find the track, which are described below. These transformations do not use machine learning techniques in the process. The deviation value from the center point can ultimately be derived from the detected line.
-
-Which algorithm can be used to keep the track optimal?
-
->Different control engineering methods were selected for this project. The following algorithms were evaluated:
->1. Simple countersteering with fixed steering angle
->1. P controller
->1. PD controller
->1. PID controller
-
-### Computer Vision to Identify the Track
-As described earlier, no machine learning methods were used for track finding. There are two reasons for this:
-- many machine learning methods often still work according to a black box principle, i.e. the exact reason for prediction is usually not necessarily comprehensible.
-- on the other hand it is also interesting to see how only image transformations can already detect the roadway.
-
-**Approach**
-
-To extract the trace through image-based transformations, several steps were performed, which are described below.
-
-
-
-
-Original image that we get from the simulator. We get a new frame every 0.01 seconds.
-
-
-
-
-
-The image shows the original image in black and white. All colors that are within a defined color space are marked with white. All other pixels are replaced by black pixels. Additionally, the blur of the image was increased.
-
-
-
-
-
-The region of interest (RoI, outlined in pink here) indicates the area to be viewed. This means that only the current lane is considered. The RoI should be large enough to include curves, but small enough not to include other lane lines.
-
-
-
-
-
-Here the angle is changed to a bird's eye view. Only the region of interest is taken into account. You can already see the track better. Through the vanishing point the lines approach each other.
-
-
-
-
-
-The figure shows the transformed image in binary representation and the corresponding histogram. The peaks of the histogram correspond to the gradients (i.e. the change from black to white and vice versa) from the image above. From the histogram, the outer limits of the lane can be read. The center of the roadway corresponds to the center of the two histogram peaks.
-
-
-
-
-
-During this step, a small window is slid over the image. Based on the non-black pixels inside the sliding window, important line points are extracted.
-
-
-
-
-
-Using the calculated coordinates of the line points (from the previous image), a polynomial is approximated for each side.
-
-
-
-
-
-Original image with detected track.
-
-
-**Challenges during Implementation**
-
-> No or Incorrectly Detected Track
-
-Based on the detected tracks, the center point could be determined afterwards. However, there is a problem here. On the test track "Town04" (see below), which we have selected here, there are no solid lines (only the separating lane on the far left and right), which is why our method has difficulty recognizing a reasonable lane. The second figure below shows what such an incorrectly detected trace looks like. In the code, we caught the error by calculating the area of the detected trace each time. If the area falls below a predefined threshold, the detected trace is ignored and the center of the last valid trace is used as a starting point for further decisions.
-
-
-
-
-Town04 in CARLA
-
-
-
-
-
-The illustration shows a misidentified track. Within the red box (used here only as a visual aid), a narrow green stripe can be seen. However, the green stripe does not correspond to any lane and is therefore an error.
-
-
-> Highway Exit
-
-In the CARLA simulation, the vehicle is spawned on the right lane. However, the exits are also linked to the right lane. As soon as an exit comes up, our program has problems to detect a lane, because for a short time the right lane is missing (see below). To avoid such errors, a lane change to the left lane is performed at the beginning of each simulation.
-
-
-
-
-On the right lane, the dividing line is missing for a short time.
-
-
-> Collision Detection
-
-Another problem was the collision with the concrete sliding wall. If the vehicle does not detect a lane or the parameters are configured too sensitively, the vehicle leaves the lane and collides with a concrete sliding wall. Sometimes, however, a phenomenon occurs that shortly before the vehicle collides, it detects a lane for a short time. Accordingly, the vehicle stores the last deviation of the last valid time for the current error (see solution from chapter No or Incorrectly Detected Lane). During optimization, this phenomenon can cause problems. As a solution we have implemented a CollisionDetector. As soon as the vehicle collides with an object and therefore comes to a stop, the previous error is stored for the remaining time.
-
-### Control Technology for Tracking
-
-As already described in the introduction, four procedures were tested. The simple method countersteers with a constant steering angle as soon as it exceeds a threshold value. In this way, the agent manages to keep the vehicle on track without any problems. However, the abrupt and uneven steering movements also reduce driving comfort. Programming a simple procedure was nevertheless helpful, as it allowed the test environment to be tested initially.
-
-Based on the observations of the driving behavior controlled by a simple controller, a more complex controller is needed that blocks strong "wobble" and compensates instead by precise steering movements.
-
-A pure P-controller like the simple controller was already able to drive the track with confidence. In addition, the proportional part made the steering movements smoother than with just a constant factor. However, the vehicle only countersteers as soon as the error occurs from the other direction (vehicle must now approach the center point from the other side). The differential component counteracts overshooting of the track center point, thus reducing extreme overshoots. If, for example, further deviations or distortions occur due to the environment, which prevent the vehicle from reaching the center point, these influences must be taken into account. The integrating controller monitors the sum of all errors and influences the steering movement unless the errors have been reduced during a certain time.
-
-The parameters for the P and PD controllers can still be determined manually, since a maximum of two parameters have to be adjusted to each other. With three parameters, however, the adaptation effort increases, which is why the use of automated processes is the logical consequence. There are different methods with advantages and disadvantages. The random search offers the possibility to test many parameter variations, which normally would not be tested in this form. In order to find meaningful parameters, however, many iterations must be executed, which also requires many resources. A method that does not have this problem is the Twiddle algorithm, which was also used in the context of this project. The twiddle algorithm makes it possible to calculate reasonable values for the proportional, integrating and differentiating parts using a simple implementation.
-
-The twiddle algorithm proceeds as follows: At the beginning, the parameters to be optimized are initially defined in a list and the corresponding deltas are set up. The parameters from the parameter list are adjusted one after the other. For this purpose, it is first examined whether an increase of the current parameter results in a reduction of the error, which in this case corresponds to the deviation from the center of the track. If this is the case, the corresponding delta value is increased. Otherwise, the search is performed in the other direction by reducing the current parameter by twice the delta value and then recalculating the error. If there was an improvement with the reduced parameter value, the delta value is increased. Otherwise, the delta values and parameter values are updated. To determine the parameters of the PID controller, the combinations were tested to a maximum of 5000 steps using the twiddle method.
-
-Due to the simulation environment, we were not able to optimize the parameters outside the simulation, which is why we were bound to the real-time simulation and could not "fast-forward" the optimization.
-
-The final parameters are:
-| Method | Value |
-| ----------- | ----------- |
-| Simple Lane Detection | Constant steering angle: 0.75 |
-| PID-Controller | P: 0.65003358 I: 0.00000002 D: 0.03399347 |
-
-**Examples of Runs of the Simplified Procedure**
-
-[](https://youtu.be/WlGPokl0ZK8)
-
-(Video) Simulation with the simple method over 10k steps.
-
-
-
-
-
-Error depending on the time step for the simple method. It can be seen from step 1000 that the error occurs cyclically in similarity to the sinusoid. This can be explained by the change from side relative to the center of the detected trace. The straight sections connecting the curves show the moments when no trace was detected. Here, false detections occur again and again due to the small white lines in the spaces.
-
-
-**Examples of Runs of the PID Controller**
-
-[](https://youtu.be/YVZafMtOxzY)
-
-
-(Video) Simulation with a PID controller over 10k steps.
-
-
-
-
-
-Error depending on the time step for the PID controller. Larger deviations occur at the beginning, which, however, do not occur from step 2000 onwards. After that the controller shows a constant stable deviation to the center point.
-
-
-## Closing Words
-
-The vision of self-driving vehicles poses a great challenge both in science and in practice, which is why there is a great deal of interest in this topic, even from many newcomers. Accordingly, there are some sources on the Internet and in the literature about this, which have already implemented similar projects. Furthermore, there are also different approaches. For example, some projects encode the images in HSL or HSV color space, whereas other projects leave it at RGB encoding. It should be mentioned that we did not "invent" the performed transformation steps during the track detection, rather we tested different combinations of approaches and parameters that are most suitable for our specific use case.
-
-## Sources
-- https://towardsdatascience.com/lane-detection-with-deep-learning-part-1-9e096f3320b7
-- https://www.analyticsvidhya.com/blog/2020/05/tutorial-real-time-lane-detection-opencv/
-
-[Go back to table of contents](#table-of-contents)
+Active Lane Keeping Assistant (主动式车道保持辅助系统)
+这是一个基于 CARLA 模拟器 的自动驾驶车道保持项目。该系统使用计算机视觉技术从摄像头图像中检测车道线,并利用 PID 控制器(或自适应 Twiddle 算法)控制车辆的转向,使其保持在车道中心行驶。
+
+
+项目简介
+该项目主要由三个核心模块组成:
+视觉感知 (Lane):使用 OpenCV 进行图像处理,包括 HLS 阈值过滤、透视变换(鸟瞰图)、滑动窗口车道检测和二阶多项式拟合。
+决策控制 (Agent):实现了 Simple、P、PD 和 PID 四种控制器,并包含 Twiddle 算法用于自动调整 PID 参数。
+仿真环境 (World):封装了 CARLA API,负责车辆生成、传感器管理(RGB 相机、碰撞传感器)以及同步仿真步进。
+
+
+环境依赖
+本项目依赖于 CARLA 模拟器环境。请确保您已安装以下依赖:Python 3.xCARLA Simulator (推荐版本 0.9.13 1)主要的 Python 库依赖如下(详见 requirements.txt 2):
+carla==0.9.13
+opencv-python==4.7.0.72
+numpy==1.20.0
+matplotlib==3.7.1
+安装步骤克隆本仓库到本地。启动 CARLA 模拟器(服务器端)。安装 Python 依赖:Bashpip install -r requirements.txt
+
+
+使用方法
+项目的入口文件是 main.py。你可以通过命令行参数来控制运行模式。
+基本运行
+运行默认的 PID 控制器,行驶 1000 步,并指定本次运行的 ID 为 test_run:Bashpython main.py -id test_run -c pid -s 1000
+命令行参数说明
+参数,简写,描述,默认值
+--identifier,-id,(必填) 本次运行的唯一标识符,用于生成日志和结果文件夹。,无
+--controller,-c,"选择控制器类型,可选值:simple, p, pd, pid。",pid
+--steps,-s,车辆运行的步数。,100
+--adapt,-a,是否开启参数自适应模式(Twiddle 算法调参)。,False
+--tolerance,-t,Twiddle 算法的容差值。,0.2
+--debug,-d,是否开启调试日志模式。,False
+进阶用法:参数自适应 (Twiddle)如果你想让 Agent 自动寻找最优的 PID 参数,可以使用 -a 参数开启 Twiddle 算法:Bashpython main.py -id auto_tune -c pid -s 500 -a -t 0.1
+注意:Twiddle 算法仅支持 pid 控制器。
+
+
+ 项目结构
+ Plaintextsrc/
+├── agent.py # 智能体类:包含 PID 控制器逻辑和 Twiddle 算法
+├── lane.py # 车道检测类:处理图像,计算车道线偏差 (Error)
+├── main.py # 主程序:处理参数,运行仿真循环
+├── recorder.py # 录像机类:将仿真画面保存为视频文件
+├── world.py # 环境类:管理 CARLA 客户端、地图加载和 Actor 生成
+└── requirements.txt # 依赖列表
+核心算法流程图像获取:
+World 类从 CARLA 的 RGB 摄像头获取原始图像。
+预处理:Lane.get_lines() 将图像转换为 HLS 空间并提取白色车道线。
+透视变换:Lane.extract_roi() 将图像转换为鸟瞰图(BEV),以便更准确地测量曲率。
+车道拟合:Lane.get_line_fits() 使用滑动窗口算法定位车道像素,并拟合二阶多项式曲线。
+误差计算:计算车辆中心与车道中心的偏差值(Error)。
+控制输出:Agent 根据偏差值,通过 PID 控制器计算转向角度(Steer)。
+执行控制:将转向和油门指令发送回 CARLA 车辆。
+
+
+结果输出
+运行后,程序会在 assets// 目录下生成以下内容:日志文件 (alka_.log):记录运行过程中的参数和误差。误差曲线图 (_error.jpg):展示行驶过程中的横向偏差。中间过程图片 (如果在代码中开启 save=True):包括二值化图、滑动窗口图、原始图像叠加图等。
\ No newline at end of file
From 7a658e8e517845ee2c2efdffd36aa2a29114afc0 Mon Sep 17 00:00:00 2001
From: wutong88yuyu <2179751939@qq.com>
Date: Wed, 17 Dec 2025 16:52:47 +0800
Subject: [PATCH 3/5] =?UTF-8?q?=E5=B0=8F=E8=BD=A6=E8=90=BD=E7=82=B9?=
=?UTF-8?q?=E4=BD=8D=E7=BD=AE=E4=BC=98=E5=8C=96?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
---
src/Active-Lane-Keeping-Assistant/world.py | 23 ++++++++++++++++------
1 file changed, 17 insertions(+), 6 deletions(-)
diff --git a/src/Active-Lane-Keeping-Assistant/world.py b/src/Active-Lane-Keeping-Assistant/world.py
index 7f106f10ca..13348785e4 100644
--- a/src/Active-Lane-Keeping-Assistant/world.py
+++ b/src/Active-Lane-Keeping-Assistant/world.py
@@ -47,12 +47,23 @@ def __init__(self, server_ip: str = '127.0.0.1', port: int = 2000,
# Use Tesla Model 3 as Car
self.bp = self.blueprint_library.filter('model3')[0]
- # === 修复 2: 随机选择出生点,防止车位被占导致无法生成 ===
- spawn_points = self.world.get_map().get_spawn_points()
- if spawn_points:
- self.spawn_point = random.choice(spawn_points)
- else:
- self.spawn_point = carla.Transform() # 防止没有生成点的情况
+ # === 修改: 将出生点设置为当前上帝视角相机的位置 ===
+ # 1. 获取上帝视角的观察者对象
+ spectator = self.world.get_spectator()
+
+ # 2. 获取观察者现在的坐标和朝向
+ transform = spectator.get_transform()
+
+ # 3.稍微把位置抬高一点(+2米),防止相机贴地导致车卡在地里
+ transform.location.z += 2.0
+
+ # 4. 稍微把位置往前挪一点(+5米),让你能直接看到车出现在眼前
+ # (利用朝向向量计算前方的位置)
+ forward_vector = transform.get_forward_vector()
+ transform.location.x += forward_vector.x * 5.0
+ transform.location.y += forward_vector.y * 5.0
+
+ self.spawn_point = transform
# ====================================================
self.vehicle = None
From 2ff8abe2cb41b26bdedd81e7d5f1b3dfccd4b6b2 Mon Sep 17 00:00:00 2001
From: wutong88yuyu <2179751939@qq.com>
Date: Wed, 17 Dec 2025 22:06:33 +0800
Subject: [PATCH 4/5] =?UTF-8?q?=E5=B0=8F=E8=BD=A6=E7=94=9F=E6=88=90?=
=?UTF-8?q?=E5=90=8E=E7=9B=B4=E6=8E=A5=E5=B7=A6=E8=BD=AC=E6=92=9E=E8=BD=A6?=
=?UTF-8?q?=E9=97=AE=E9=A2=98=E8=A7=A3=E5=86=B3?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
---
src/Active-Lane-Keeping-Assistant/world.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/Active-Lane-Keeping-Assistant/world.py b/src/Active-Lane-Keeping-Assistant/world.py
index 13348785e4..2c4315f9ec 100644
--- a/src/Active-Lane-Keeping-Assistant/world.py
+++ b/src/Active-Lane-Keeping-Assistant/world.py
@@ -121,7 +121,7 @@ def reset(self) -> tuple[float, float, np.ndarray, bool]:
self.collision_sensor.listen(lambda e: {self._set_collision()})
self.initialized = True
- return self._change_to_left_lane()
+ return 0.0, 0.0, np.zeros((self.image_height, self.image_width, 3)), False
def _set_collision(self) -> None:
"""Helper to set collision_detected to True.
From 6ff123833a6ecd935c0d64cbbbdf623f0caaacb5 Mon Sep 17 00:00:00 2001
From: wutong88yuyu <2179751939@qq.com>
Date: Wed, 17 Dec 2025 22:39:57 +0800
Subject: [PATCH 5/5] =?UTF-8?q?=E8=A7=A3=E5=86=B3carla=E5=8D=A1=E9=A1=BF?=
=?UTF-8?q?=E9=97=AE=E9=A2=98?=
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
---
src/Active-Lane-Keeping-Assistant/main.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/Active-Lane-Keeping-Assistant/main.py b/src/Active-Lane-Keeping-Assistant/main.py
index 1b20920936..7ac65d4135 100644
--- a/src/Active-Lane-Keeping-Assistant/main.py
+++ b/src/Active-Lane-Keeping-Assistant/main.py
@@ -62,7 +62,7 @@ def run(id:str, path:str, world:World, agent:Agent, steps:int = 100,
steer, throttle = agent.get_actions(detection_surface_area, error)
error, detection_surface_area, img, collision_detected = world.step(
steer=steer, throttle=throttle)
- agent.show_error()
+ #agent.show_error()
if save:
recorder.add_image(img=img)