diff --git a/examples/applications/parking_management.ipynb b/examples/applications/parking_management.ipynb new file mode 100644 index 0000000..350b9e9 --- /dev/null +++ b/examples/applications/parking_management.ipynb @@ -0,0 +1,308 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Degirum banner](https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/degirum_banner.png)\n", + "## Managing a Parking Lot in a Video Frame-by-Frame\n", + "This notebook demonstrates the management of a parking lot in a video.\n", + "\n", + "In each video frame, user-defined polygon zones are checked for occupancy. Vehicles are detected in each frame, and the counts of occupied and available zones are noted. This information is used to annotate a video.\n", + "\n", + "This script works with the following inference options:\n", + "\n", + "1. Run inference on the DeGirum Cloud Platform;\n", + "2. Run inference on a DeGirum AI Server deployed on the local host or on some computer in your LAN or VPN;\n", + "3. Run inference on a DeGirum ORCA accelerator directly installed on your computer.\n", + "\n", + "To try different options, you need to specify the appropriate `hw_location` option. \n", + "\n", + "When running this notebook locally, you need to specify your cloud API access token in the [env.ini](../../env.ini) file, located in the same directory as this notebook.\n", + "\n", + "When running this notebook in Google Colab, the cloud API access token should be stored in a user secret named `DEGIRUM_CLOUD_TOKEN`." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# make sure degirum and degirum-tools packages are installed\n", + "!pip show degirum-tools || pip install degirum-tools" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Note: this notebook requires a zones JSON file, which is provided for this example. If you would like to create your own zones JSON file, you can do so with the help of DeGirum's GUI annotation tool. This tool is demonstrated in the **'zone_annotation.ipynb'** notebook. Run the **'zone_annotation.ipynb'** notebook directly to generate a new zones JSON for this specific example." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Specify where you want to run your inferences, model zoo url, model name, path to zones JSON file, and video source" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "\n", + "# hw_location: where you want to run inference\n", + "# \"@cloud\" to use DeGirum cloud\n", + "# \"@local\" to run on local machine\n", + "# IP address for AI server inference\n", + "# vehicle_model_zoo_url: url/path for vehicle detection model zoo\n", + "# Use cloud_zoo_url for @cloud, @local, and AI server inference options.\n", + "# Use '' for an AI server serving models from a local folder.\n", + "# Use a path to a JSON file for a single model zoo in case of @local inference.\n", + "# vehicle_model_name: name of the model for person detection.\n", + "# zones_json_name: path to zone JSON file\n", + "# video_source: video source for inference\n", + "# camera index for local camera\n", + "# URL of RTSP stream\n", + "# URL of YouTube Video\n", + "# path to video file (mp4 etc)\n", + "# output_video: annotated video destination\n", + "# degirum_cloud_token: your token for accessing the DeGirum cloud platform\n", + "hw_location = \"@cloud\"\n", + "vehicle_model_zoo_url = \"https://cs.degirum.com/degirum/visdrone\"\n", + "vehicle_model_name = \"yolov8s_relu6_visdrone--640x640_float_openvino_cpu_1\"\n", + "zones_json_name = \"parking_zones.json\"\n", + "video_source = \"https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Parking.mp4\"\n", + "output_video = \"temp/Parking_annotated.mp4\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Specify arguments for the annotation process\n", + "Specify the occupied and vacant zone colors below, as well as the objects to be considered for detection.\n", + "\n", + "The ZoneCounter algorithm supports the following zone triggering conditions:\n", + " 1. User-defined point(s) on the object's bounding box is/are found inside the zone.\n", + " 2. The object bounding box's intersection over polygon area (IoPA) is greater than a user-defined threshold.\n", + "\n", + "To use the first approach, set the `use_triggers` flag below to True, and assign the list of triggering position(s) to `triggering_position_values`.\n", + "To use the second approach, set the `use_triggers` flag to False, and specify `iopa_threshold`.\n", + "\n", + "The algorithm can optionally resize the detected object bounding boxes, scaling the width and height\n", + "around the center point of the bounding box, before triggering the zones.\n", + "Specify `bounding_box_scale` to use this feature.\n", + "\n", + "In order to reduce fluctuations in the output of the algorithm, due to the event when an object is initially appears, disappears for a short period of time\n", + "in a video stream, and then reappears, information from the ObjectTracker algorithm can be used in the ZoneCounter if the `use_tracking` flag is set to True. For each detected object, the bounding box coordinates of that object in a user-defined number of prior frames is saved by the ObjectTracker algorithm, and the ZoneCounter can use this information to trigger a zone with an object that may not be necessarily present in the current frame, but was present a short while ago. The `track_buffer` variable can be assigned with the depth (in number of frames) of an object's position history.\n", + "\n", + "The output video can be optionally annotated with the detection model's results, if the `show_ai_overlay` flag is set to True." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import degirum_tools\n", + "\n", + "occupied_zone_color = (0, 0, 255)\n", + "available_zone_color = (0, 255, 0)\n", + "class_list = [\"car\", \"van\", \"truck\", \"bus\"]\n", + "\n", + "use_triggers = True\n", + "# if use_triggers is True, the trigger-point approach is used\n", + "triggering_position_values = [degirum_tools.AnchorPoint.CENTER]\n", + "triggering_positions = triggering_position_values if use_triggers else None\n", + "# else, the IoPA approach is used\n", + "iopa_threshold = 0.5\n", + "\n", + "bounding_box_scale = 1.0\n", + "\n", + "use_tracking = True\n", + "track_buffer = 30\n", + "\n", + "show_ai_overlay = False" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Load the model" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import degirum as dg\n", + "\n", + "# Load and configure model\n", + "model = dg.load_model(\n", + " vehicle_model_name,\n", + " hw_location,\n", + " vehicle_model_zoo_url,\n", + " degirum_tools.get_token(),\n", + " image_backend=\"opencv\",\n", + " overlay_show_labels=False,\n", + " overlay_line_width=2,\n", + " overlay_font_scale=2,\n", + " output_confidence_threshold=0.4,\n", + " overlay_show_probabilities=False,\n", + " output_max_detections=300,\n", + " output_max_detections_per_class=300,\n", + " input_letterbox_fill_color=(114, 114, 114),\n", + " output_class_set=set(class_list)\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Analyzers\n", + "In order to annotated a video, a class, called an Analyzer, is required. A child of this class, the ZoneCounter, is implemented in DeGirum tools, and keeps track of the number and type of objects in pre-defined polygon zones. A child class of the ZoneCounter needs to be created to determine the occupied and available zones. This class, called ZoneOccupancyCounter, is implemented below.\n", + "Additionally, to allow the ZoneCounter to use object tracking information, the ObjectTracker is instantiated." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import numpy as np\n", + "import json, cv2\n", + "\n", + "\n", + "class ZoneOccupancyCounter(degirum_tools.ZoneCounter):\n", + " def __init__(\n", + " self,\n", + " polygon_json,\n", + " triggering_positions,\n", + " bounding_box_scale,\n", + " iopa_threshold,\n", + " occupied_zone_color,\n", + " available_zone_color,\n", + " class_list,\n", + " use_tracking\n", + " ):\n", + " with open(polygon_json, \"r\") as poly_json:\n", + " self.polygon_json = json.load(poly_json)\n", + " self.polygon_json = [zone for zone in self.polygon_json[\"objects\"]]\n", + " self.occupied_zone_color = occupied_zone_color\n", + " self.available_zone_color = available_zone_color\n", + " super().__init__(\n", + " count_polygons=self.polygon_json,\n", + " class_list=class_list,\n", + " triggering_position=triggering_positions,\n", + " bounding_box_scale=bounding_box_scale,\n", + " iopa_threshold=iopa_threshold,\n", + " use_tracking=use_tracking,\n", + " window_name=\"Vehicle Management\"\n", + " )\n", + "\n", + " def annotate(self, result, image: np.ndarray) -> np.ndarray:\n", + " total_slots, filled_slots = len(self.polygon_json), 0\n", + " empty_slots = total_slots\n", + "\n", + " # draw annotations\n", + " for zi in range(len(self._polygons)):\n", + " region_occupied = result.zone_counts[zi].get('total', 0) > 0\n", + " line_color = self.occupied_zone_color if region_occupied else self.available_zone_color\n", + " cv2.polylines(\n", + " image, [self._polygons[zi]], True, line_color, result.overlay_line_width\n", + " )\n", + " if region_occupied:\n", + " filled_slots += 1\n", + " empty_slots -= 1\n", + "\n", + " label = \"Occupancy: {}\\nAvailable: {}\".format(filled_slots, empty_slots)\n", + " \n", + " back_color = (255, 255, 255)\n", + " font_color = degirum_tools.deduce_text_color(back_color)\n", + " degirum_tools.put_text(\n", + " image,\n", + " label,\n", + " (image.shape[1], 0),\n", + " corner_position=degirum_tools.CornerPosition.TOP_RIGHT,\n", + " bg_color=back_color,\n", + " font_color=font_color,\n", + " font_scale=result.overlay_font_scale,\n", + " font_thickness=4,\n", + " line_spacing=1.5\n", + " )\n", + " return image\n", + " \n", + "\n", + "# Instantiate a ZoneOccupancyCounter Analyzer\n", + "zone_occupancy_counter = ZoneOccupancyCounter(\n", + " zones_json_name,\n", + " triggering_positions,\n", + " bounding_box_scale,\n", + " iopa_threshold,\n", + " occupied_zone_color,\n", + " available_zone_color,\n", + " class_list,\n", + " use_tracking\n", + ")\n", + "\n", + "# Instantiate an ObjectTracker Analyzer\n", + "tracker = degirum_tools.ObjectTracker(\n", + " class_list=class_list,\n", + " track_buffer=track_buffer,\n", + " trail_depth=track_buffer,\n", + " show_overlay=False\n", + ")" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Annotate a video source with Analyzer(s)." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# annotate video\n", + "analyzers = [tracker, zone_occupancy_counter] if use_tracking else [zone_occupancy_counter]\n", + "degirum_tools.annotate_video(model, video_source, output_video, visual_display=True, show_ai_overlay=show_ai_overlay, analyzers=analyzers)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.0" + }, + "orig_nbformat": 4 + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/examples/applications/parking_zones.json b/examples/applications/parking_zones.json new file mode 100644 index 0000000..f3bd17f --- /dev/null +++ b/examples/applications/parking_zones.json @@ -0,0 +1,636 @@ +{ + "version": 1, + "type": "zone", + "objects": [ + [ + [ + 3498, + 673 + ], + [ + 3699, + 912 + ], + [ + 3513, + 912 + ], + [ + 3321, + 673 + ] + ], + [ + [ + 3321, + 673 + ], + [ + 3513, + 912 + ], + [ + 3318, + 913 + ], + [ + 3130, + 673 + ] + ], + [ + [ + 3130, + 673 + ], + [ + 3318, + 913 + ], + [ + 3115, + 913 + ], + [ + 2920, + 674 + ] + ], + [ + [ + 2920, + 674 + ], + [ + 3115, + 913 + ], + [ + 2911, + 913 + ], + [ + 2719, + 674 + ] + ], + [ + [ + 2719, + 674 + ], + [ + 2911, + 913 + ], + [ + 2710, + 914 + ], + [ + 2522, + 674 + ] + ], + [ + [ + 2522, + 674 + ], + [ + 2710, + 914 + ], + [ + 2510, + 914 + ], + [ + 2321, + 675 + ] + ], + [ + [ + 2321, + 675 + ], + [ + 2510, + 914 + ], + [ + 2306, + 914 + ], + [ + 2114, + 675 + ] + ], + [ + [ + 2114, + 675 + ], + [ + 2306, + 914 + ], + [ + 2102, + 915 + ], + [ + 1914, + 675 + ] + ], + [ + [ + 1914, + 675 + ], + [ + 2102, + 915 + ], + [ + 1893, + 915 + ], + [ + 1704, + 676 + ] + ], + [ + [ + 1704, + 676 + ], + [ + 1893, + 915 + ], + [ + 1695, + 915 + ], + [ + 1506, + 676 + ] + ], + [ + [ + 1506, + 676 + ], + [ + 1695, + 915 + ], + [ + 1491, + 916 + ], + [ + 1302, + 677 + ] + ], + [ + [ + 1302, + 677 + ], + [ + 1491, + 916 + ], + [ + 1290, + 916 + ], + [ + 1117, + 677 + ] + ], + [ + [ + 1117, + 677 + ], + [ + 1290, + 916 + ], + [ + 1093, + 917 + ], + [ + 913, + 677 + ] + ], + [ + [ + 913, + 677 + ], + [ + 1093, + 917 + ], + [ + 892, + 917 + ], + [ + 721, + 678 + ] + ], + [ + [ + 721, + 678 + ], + [ + 892, + 917 + ], + [ + 700, + 917 + ], + [ + 536, + 678 + ] + ], + [ + [ + 536, + 678 + ], + [ + 700, + 917 + ], + [ + 515, + 918 + ], + [ + 344, + 678 + ] + ], + [ + [ + 344, + 678 + ], + [ + 515, + 918 + ], + [ + 302, + 918 + ], + [ + 140, + 679 + ] + ], + [ + [ + 3085, + 1355 + ], + [ + 3357, + 1630 + ], + [ + 3184, + 1628 + ], + [ + 2890, + 1352 + ] + ], + [ + [ + 2890, + 1352 + ], + [ + 3184, + 1628 + ], + [ + 2968, + 1626 + ], + [ + 2689, + 1350 + ] + ], + [ + [ + 2689, + 1350 + ], + [ + 2968, + 1626 + ], + [ + 2779, + 1624 + ], + [ + 2501, + 1348 + ] + ], + [ + [ + 2501, + 1348 + ], + [ + 2779, + 1624 + ], + [ + 2575, + 1622 + ], + [ + 2306, + 1346 + ] + ], + [ + [ + 2306, + 1346 + ], + [ + 2575, + 1622 + ], + [ + 2381, + 1620 + ], + [ + 2105, + 1343 + ] + ], + [ + [ + 2105, + 1343 + ], + [ + 2381, + 1620 + ], + [ + 2177, + 1618 + ], + [ + 1911, + 1341 + ] + ], + [ + [ + 1911, + 1341 + ], + [ + 2177, + 1618 + ], + [ + 1985, + 1617 + ], + [ + 1716, + 1339 + ] + ], + [ + [ + 1716, + 1339 + ], + [ + 1985, + 1617 + ], + [ + 1785, + 1615 + ], + [ + 1524, + 1336 + ] + ], + [ + [ + 1524, + 1336 + ], + [ + 1785, + 1615 + ], + [ + 1599, + 1613 + ], + [ + 1332, + 1334 + ] + ], + [ + [ + 1332, + 1334 + ], + [ + 1599, + 1613 + ], + [ + 1401, + 1611 + ], + [ + 1132, + 1332 + ] + ], + [ + [ + 1132, + 1332 + ], + [ + 1401, + 1611 + ], + [ + 1207, + 1609 + ], + [ + 955, + 1330 + ] + ], + [ + [ + 955, + 1330 + ], + [ + 1207, + 1609 + ], + [ + 1009, + 1607 + ], + [ + 763, + 1327 + ] + ], + [ + [ + 763, + 1327 + ], + [ + 1009, + 1607 + ], + [ + 829, + 1605 + ], + [ + 587, + 1325 + ] + ], + [ + [ + 587, + 1325 + ], + [ + 829, + 1605 + ], + [ + 649, + 1604 + ], + [ + 407, + 1323 + ] + ], + [ + [ + 407, + 1323 + ], + [ + 649, + 1604 + ], + [ + 470, + 1602 + ], + [ + 230, + 1321 + ] + ], + [ + [ + 230, + 1321 + ], + [ + 470, + 1602 + ], + [ + 284, + 1600 + ], + [ + 38, + 1319 + ] + ], + [ + [ + 3291, + 1352 + ], + [ + 3447, + 1528 + ], + [ + 3360, + 1633 + ], + [ + 3091, + 1355 + ] + ], + [ + [ + 3714, + 685 + ], + [ + 3822, + 909 + ], + [ + 3708, + 915 + ], + [ + 3519, + 679 + ] + ] + ] +} \ No newline at end of file diff --git a/examples/applications/zone_annotation.ipynb b/examples/applications/zone_annotation.ipynb new file mode 100644 index 0000000..448bd10 --- /dev/null +++ b/examples/applications/zone_annotation.ipynb @@ -0,0 +1,124 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "![Degirum banner](https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/degirum_banner.png)\n", + "## Using DeGirum's GUI annotation tool to annotate zones in an image\n", + "This notebook demonstrates DeGirum's GUI annotation tool used to create a zones JSON file compatible with the ZoneOccupancyCounter analyzer used in 'parking_management.ipynb'." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# make sure degirum-tools packages are installed\n", + "!pip show degirum-tools || pip install degirum-tools" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Specify the arguments\n", + "Provide the name of the JSON file for the annotation utility, as well as the video source from which to obtain the image to annotate." + ] + }, + { + "cell_type": "code", + "execution_count": 21, + "metadata": {}, + "outputs": [], + "source": [ + "zones_json_name = \"parking_zones.json\"\n", + "video_source = \"https://raw.githubusercontent.com/DeGirum/PySDKExamples/main/images/Parking.mp4\"\n", + "image_name = \"Parking.jpg\"" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Obtain a frame from the video to annotate with zones" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "# Specify timestamp for a frame in video (in seconds)\n", + "timestamp = 0" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import cv2\n", + "import degirum_tools\n", + "\n", + "_, _, fps = degirum_tools.get_video_stream_properties(video_source)\n", + "frame_ind = int(timestamp * fps) + 1\n", + "i = 0\n", + "\n", + "# Obtain frame at specified timestamp\n", + "with degirum_tools.open_video_stream(video_source) as stream:\n", + " for frame in degirum_tools.video_source(stream):\n", + " i += 1\n", + " if i == frame_ind:\n", + " break\n", + "\n", + "# Save frame as an image file.\n", + "cv2.imwrite(image_name, frame)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "#### Annotate zones\n", + "Using the GUI annotation tool, define parking zones in an image.\n", + "\n", + "When the cell below is run, a window will appear. Left-click the corners of a 4-sided zone, clockwise or counter-clockwise, on the image. Add as many zones as needed, and then click File > Save to save the coordinates to a JSON file. A complete guide on using this tool is available by clicking the Help button in the GUI window. When the annotations are completed, close the GUI window." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": {}, + "outputs": [], + "source": [ + "!degirum_tools zone_annotator {image_name} --save-path {zones_json_name}" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "base", + "language": "python", + "name": "python3" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.10.0" + }, + "orig_nbformat": 4 + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/images/Parking.mp4 b/images/Parking.mp4 new file mode 100644 index 0000000..ade7f28 Binary files /dev/null and b/images/Parking.mp4 differ