Skip to content
sima-neatPublic

About

Performance Visualizer and Interactive Testing Tool for NEAT Apps

Resources

Code of conduct

Contributing

Security policy

Stars

4 stars

Watchers

0 watching

Forks

neat-insight

Documentation: English | 한국어 | 日本語 | 繁體中文 | Українська

Vulcan CI License: Apache 2.0

neat-insight is a web app for inspecting NEAT apps and helping to setup test for vision ML apps.

It provides:

  • Multi-channel WebRTC video viewer with MetadataReceiver support
  • Frame-correlated auxiliary visualization panels, including BlazePose 3D
  • Codec-aware media source control with RTSP and HTTP MJPEG streaming
  • System and application metrics dashboard

User documentation for the software docs site is maintained in docs/index.md.

Install

Option 1: Install from PyPI (official release)

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install neat-insight

Option 2: Install from Neat artifacts

sima-cli neat install insight@main

This installs the platform-compatible neat-insight wheel into an isolated virtual environment.

Run

neat-insight uses mkcert to create a locally trusted HTTPS certificate at startup. If mkcert is missing, startup attempts to install it with a supported package manager: Homebrew on macOS, common Linux package managers, winget/Chocolatey/Scoop on Windows, or go install as a fallback.

source .venv/bin/activate
neat-insight --port 9900

Then open:

  • https://${NFS_SERVER_HOST_IP}:9900 when NFS_SERVER_HOST_IP is set
  • https://127.0.0.1:9900 otherwise

Notes:

  • When /sdk-cert/neat-sdk.pem exists, the app validates and uses that SDK-provided certificate before attempting mkcert generation. The private key may be embedded in that PEM or provided as /sdk-cert/neat-sdk-key.pem, /sdk-cert/neat-sdk.key, or /sdk-cert/key.pem.
  • The app runs mkcert -install and regenerates cert.pem/key.pem under the neat-insight data directory on startup.
  • Certificates include the configured host IP, 127.0.0.1, and localhost.
  • If automatic mkcert installation is unavailable, install mkcert manually and restart neat-insight.

Basic usage

  1. Open the Viewer tab to monitor active channels.
  2. Open Streaming to assign/start/stop input sources. To publish a webcam attached to your browser's computer as a live source, enable a camera under Media Sources → Local cameras and select it in a source slot.
  3. Open Stats to inspect live and recorded Sentinel telemetry from the selected board.

Viewer settings

Open the global viewer menu or a channel tile menu to configure overlays. Global settings apply to all channels; channel settings override the global values for that channel.

Available metadata settings:

  • Object Detection: confidence threshold and per-label box style.
  • Segmentation: confidence threshold, mask opacity, and per-label mask color and outline style.
  • Tracking: confidence threshold, track history visibility, trail length, and how long lost-track trails remain visible.
  • Other metadata types are rendered with defaults until type-specific settings are added.

ROI settings are split into filtering and display:

  • Apply ROI Filtering controls whether ROI polygons filter box-like metadata.
  • Show ROI Overlay controls only whether ROI polygons are drawn over the video.

General settings include Overlay Delay, which delays metadata selection so boxes and tracks can be aligned with the displayed WebRTC frame.

Frame-correlated data that should not cover the video can use an auxiliary visualization panel. See docs/auxiliary-visualizations.md for the generic contract and built-in BlazePose 3D renderer. Viewer Configuration controls the panel per channel, and the 3D skeleton stays frame-aligned with the 2D pose overlay.

Build from source

Use this when you need to modify functionality.

Prerequisites:

  • Python 3.10+
  • Node.js 20+ and npm
  • Go 1.24+
  • A C compiler for each Linux target: gcc for the host, plus gcc-aarch64-linux-gnu or gcc-x86-64-linux-gnu to cross-build the other Linux wheel. Linux wheels bundle a small LD_PRELOAD shim (tools/ffmpeg_nodelay.c) that keeps the RTSP publishers' TCP sockets free of Nagle delay; NEAT_INSIGHT_FFMPEG_PRELOAD overrides its path at runtime.
  • mkcert, or a supported package manager for automatic runtime installation

Build and install into your current virtualenv:

python3 -m venv .venv
source .venv/bin/activate
./build.sh --install

Run:

neat-insight --port 9900

Useful build options:

  • --target-platform <host|all|linux-aarch64|linux-amd64|macos-arm64|windows-amd64>
  • --skip-frontend
  • --skip-shim (Linux wheels then run ffmpeg with Nagle enabled)

Testing metadata

neat-insight now ships a metadata test sender for vf metadata ports (9100-9179).

Examples:

neat-insight-metadata-test --count 1 --types object-detection
neat-insight-metadata-test --count 4 --types object-detection,classification,pose-estimation,segmentation,tracking
neat_insight/tools/multisrc-harness.sh start --count 16

Notes:

  • Default destination is 127.0.0.1, starting at UDP port 9100.
  • Supported metadata types are object-detection, classification, pose-estimation, segmentation, and tracking.
  • Overlay colors follow the identity rules in docs/user-interface.md: class for detection, segmentation and classification, id for tracking and pose estimation.
  • An object-detection object may carry oriented_bbox ([cx, cy, width, height, angle], angle in radians, clockwise on screen) to draw a rotated box; bbox stays required as its upright extent. The test sender includes one rotated ship.
  • neat_insight/tools/metadata-test.py remains as a compatibility wrapper, but the packaged CLI is the preferred entry point.

About

Performance Visualizer and Interactive Testing Tool for NEAT Apps

Resources

Code of conduct

Contributing

Security policy

Stars

4 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages