Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,3 +110,7 @@ TypeDoc runs strictly at startup. If you modify comments in the TypeScript packa

1. Stop the server.
2. Run `yarn start` again to regenerate the Markdown files.

## Showcase recording sources

[showcase-sources.json](./showcase-sources.json) records the devices, capture and editing details, fixture sources, and licenses for the task showcases. It is contributor metadata and is kept outside `static/` so it is not published with the website.
46 changes: 4 additions & 42 deletions docs/docs/02-extensions/computer-vision/02-image-classification.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@ keywords:
]
---

import TaskShowcase from '@site/src/components/TaskShowcase';

# Image Classification

Image classification analyzes an input image and predicts the most likely visual
Expand All @@ -25,47 +27,7 @@ It is ideal for visual search, photo organization, quality inspection, and
accessibility tagging. Because inference runs entirely on-device, images never
leave the user's phone.

<table className="showcase-table">
<thead>
<tr>
<th>iOS</th>
<th>Android</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<div className="device-phone iphone-chassis">
<div className="device-screen iphone-screen">
<video
className="device-video"
src="/react-native-executorch/media/image-classification-ios.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
<td>
<div className="device-phone s24-chassis">
<div className="s24-camera-hole"></div>
<div className="device-screen s24-screen">
<video
className="device-video"
src="/react-native-executorch/media/image-classification-android.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<TaskShowcase task="image-classification" />

## Quick Start

Expand Down Expand Up @@ -97,7 +59,7 @@ function MyComponent() {
```

:::tip Full Interactive Example in Gallery App
See [`src/app/(screens)/image-classification.tsx`](<https://github.com/software-mansion-labs/react-native-executorch-gallery/blob/main/src/app/(screens)/image-classification.tsx>) in the [React Native ExecuTorch Gallery](https://github.com/software-mansion-labs/react-native-executorch-gallery) for a complete, runnable screen with photo picker, result overlays, and latency tracking.
See [`src/app/(screens)/image-classification.tsx`](<https://github.com/software-mansion-labs/react-native-executorch-gallery/blob/main/src/app/(screens)/image-classification.tsx>) in the [React Native ExecuTorch Gallery](https://github.com/software-mansion-labs/react-native-executorch-gallery) for a complete, runnable screen with a photo picker, the top prediction, and its confidence.
:::

## Output Format
Expand Down
44 changes: 3 additions & 41 deletions docs/docs/02-extensions/computer-vision/03-object-detection.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,53 +17,15 @@ keywords:
]
---

import TaskShowcase from '@site/src/components/TaskShowcase';

# Object Detection

Object detection locates and classifies multiple objects within an image. For every detected item, the model predicts its category label, confidence score, and exact bounding box coordinates scaled to the original image dimensions.

Unlike image classification (which predicts a single label for the entire scene), object detection tells you both **what** objects are present and **where** they are located. It is used for real-time camera tracking, retail item recognition, document scanning, robotics, and augmented reality.

<table className="showcase-table">
<thead>
<tr>
<th>iOS</th>
<th>Android</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<div className="device-phone iphone-chassis">
<div className="device-screen iphone-screen">
<video
className="device-video"
src="/react-native-executorch/media/object-detection-ios.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
<td>
<div className="device-phone s24-chassis">
<div className="s24-camera-hole"></div>
<div className="device-screen s24-screen">
<video
className="device-video"
src="/react-native-executorch/media/object-detection-android.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<TaskShowcase task="object-detection" />

## Quick Start

Expand Down
44 changes: 3 additions & 41 deletions docs/docs/02-extensions/computer-vision/04-pose-and-keypoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,53 +19,15 @@ keywords:
]
---

import TaskShowcase from '@site/src/components/TaskShowcase';

# Pose & Keypoints

Pose estimation and keypoint detection locate specific anatomical landmarks on detected subjects — such as human skeletal joints (eyes, shoulders, elbows, wrists, hips, knees, ankles) or facial landmarks (eyes, nose tip, mouth, ears). Each prediction outputs a subject bounding box, detection confidence, and landmark coordinates scaled to the input image with individual landmark confidence scores.

Unlike basic object detection (which only returns box boundaries), keypoint detection tracks body posture, movement, and facial alignment. Common use cases include fitness/workout tracking, gesture controls, motion analysis, face alignment, and AR filters.

<table className="showcase-table">
<thead>
<tr>
<th>iOS</th>
<th>Android</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<div className="device-phone iphone-chassis">
<div className="device-screen iphone-screen">
<video
className="device-video"
src="/react-native-executorch/media/pose-and-keypoints-ios.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
<td>
<div className="device-phone s24-chassis">
<div className="s24-camera-hole"></div>
<div className="device-screen s24-screen">
<video
className="device-video"
src="/react-native-executorch/media/pose-and-keypoints-android.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<TaskShowcase task="pose-and-keypoints" />

## Quick Start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,53 +14,17 @@ keywords:
]
---

<!-- cspell:ignore dewarping -->

import TaskShowcase from '@site/src/components/TaskShowcase';

# Optical Character Recognition (OCR)

Optical Character Recognition (OCR) detects and extracts text from images. The pipeline identifies text regions with oriented quadrilateral boundaries ([`Quad`](../../06-api-reference/react-native-executorch/namespaces/cv/type-aliases/Quad.md)) and transcribes their characters in reading order (top-to-bottom, left-to-right columns).

It is used for document digitizing, receipt scanning, license plate reading, sign translation, and invoice processing. Because inference runs entirely on-device with zero network latency, sensitive documents never leave the phone.

<table className="showcase-table">
<thead>
<tr>
<th>iOS</th>
<th>Android</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<div className="device-phone iphone-chassis">
<div className="device-screen iphone-screen">
<video
className="device-video"
src="/react-native-executorch/media/optical-character-recognition-ios.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
<td>
<div className="device-phone s24-chassis">
<div className="s24-camera-hole"></div>
<div className="device-screen s24-screen">
<video
className="device-video"
src="/react-native-executorch/media/optical-character-recognition-android.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<TaskShowcase task="optical-character-recognition" />

## Quick Start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,53 +17,15 @@ keywords:
]
---

import TaskShowcase from '@site/src/components/TaskShowcase';

# Semantic Segmentation

Semantic segmentation classifies every individual pixel of an input image into a designated category label (e.g. background, person, vehicle, dog). The pipeline produces a pixel-aligned segmentation mask matching the input dimensions.

Unlike object detection (which outputs rectangular bounding boxes), semantic segmentation delivers precise pixel boundaries. It powers photo portrait effects, background blur/replacement, scene parsing, medical imaging, and autonomous navigation.

<table className="showcase-table">
<thead>
<tr>
<th>iOS</th>
<th>Android</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<div className="device-phone iphone-chassis">
<div className="device-screen iphone-screen">
<video
className="device-video"
src="/react-native-executorch/media/semantic-segmentation-ios.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
<td>
<div className="device-phone s24-chassis">
<div className="s24-camera-hole"></div>
<div className="device-screen s24-screen">
<video
className="device-video"
src="/react-native-executorch/media/semantic-segmentation-android.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<TaskShowcase task="semantic-segmentation" />

## Quick Start

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,53 +17,15 @@ keywords:
]
---

import TaskShowcase from '@site/src/components/TaskShowcase';

# Instance Segmentation

Instance segmentation combines object detection and semantic segmentation. For every detected individual object in an image, the pipeline predicts its bounding box, category label, detection confidence, and a pixel-accurate binary mask cropped to the instance bounds.

Unlike semantic segmentation (which groups all pixels of the same category into a single collective mask), instance segmentation distinguishes between separate instances of the same class (e.g. `person #1`, `person #2`). It powers interactive photo cutouts, object isolation, background effects, AR occlusions, and automated video editing.

<table className="showcase-table">
<thead>
<tr>
<th>iOS</th>
<th>Android</th>
</tr>
</thead>
<tbody>
<tr>
<td>
<div className="device-phone iphone-chassis">
<div className="device-screen iphone-screen">
<video
className="device-video"
src="/react-native-executorch/media/instance-segmentation-ios.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
<td>
<div className="device-phone s24-chassis">
<div className="s24-camera-hole"></div>
<div className="device-screen s24-screen">
<video
className="device-video"
src="/react-native-executorch/media/instance-segmentation-android.mp4"
autoPlay
loop
muted
playsInline
/>
</div>
</div>
</td>
</tr>
</tbody>
</table>
<TaskShowcase task="instance-segmentation" />

## Quick Start

Expand Down
Loading
Loading