Skip to content

ALLPROTO/satellite-data-toolkit

Repository files navigation

Satellite Data Toolkit

CI RustSec audit Release License: MIT

Desktop toolkit for NASA POWER data, EUMETSAT product access, NDVI calculation, and PV energy estimates.

This repository is a Tauri 2 application with a React/TypeScript UI and Rust backend/core. It is intended to replace the original Python/Tkinter zip with a one-click desktop app that does not require end users to install Python, Node, Rust, or libraries.

Reviewer Snapshot

  • Current package status: macOS and Windows packaging scripts are configured with checksums, and public-repository GitHub Actions workflows build CI checks, manual package artifacts, and tag-based GitHub releases.
  • Current repository status: Rust workspace crates declare MIT licensing, the root LICENSE carries the matching MIT license text for GitHub/release consumers, and SECURITY.md defines vulnerability reporting. npm run verify, npm run visual:smoke, ./scripts/build-macos.sh, and .\scripts\build-windows.ps1 back the local and hosted gates.
  • Current UI status: implemented desktop shell matching the requested dark toolkit structure with a production-oriented neutral palette, stable workflow tabs, request panels, response tables, logs, saved data, API slots, settings, and about screen.
  • Current backend status: NASA POWER live fetch/normalization, SQLite saved datasets, CSV/JSON export, local PV estimate, PVWatts/NLR command, API keychain slots, pure-Rust TIFF NDVI with common GeoTIFF metadata/nodata preservation, and checksum-gated EUMDAC sidecar execution are implemented.
  • Current release gaps: public macOS signing/notarization, Windows install/uninstall QA, signed/notarized EUMDAC sidecar release validation, live EUMETSAT/PVWatts verification with real credentials, and broader real-world GeoTIFF fixture QA for NDVI.

Feature Status

Area Status Notes
macOS app/DMG Partial release-ready Builds locally with ad-hoc signing and checksum output. Public distribution still needs Developer ID, hardened runtime, notarization, spctl, and stapler checks.
Windows MSI/NSIS Configured for native build Tauri config and PowerShell build script produce MSI/NSIS and checksums on Windows. Needs Windows 10/11 install/uninstall QA.
NASA POWER Implemented Uses JSON API, normalizes daily/hourly records, preserves units metadata, handles fill values. NASA POWER does not need an API key.
Saved Data Implemented Saves datasets to SQLite, supports preview, delete, and CSV/JSON export.
API Slots Implemented base Stores credentials in OS keychain under service Satellite Data Toolkit; no secrets are written to SQLite or logs.
PV Local Estimate Implemented approximate Uses normalized NASA POWER irradiance with explicit assumptions and missing-record accounting.
PVWatts/NLR Implemented, needs key QA Uses developer.nlr.gov endpoint and stored nlr_pvwatts_key; live validation requires a real key.
NDVI Production baseline Reads two TIFF rasters and writes Float32 NDVI TIFF. It preserves common GeoTIFF CRS/geotransform tags from the Red band, uses explicit or input GDAL_NODATA metadata, and has automated coverage for Deflate, LZW, PackBits, and multi-strip TIFF layouts.
EUMETSAT Partial production Sidecar discovery/search/download hooks exist. Credentials are read from OS keychain and synced to EUMDAC before CLI calls. Packaging scripts stage pinned EUMDAC 3.1.1 standalone binaries with checksum manifests; signed sidecar QA and live credential validation are still required.
Security posture Baseline+ CSP is enabled, Tauri env exposure is limited to VITE_, no shell/fs/http plugins are enabled, key slots are whitelisted, EUMDAC process errors redact stored secrets, and local RustSec/npm audit gates are documented. Signed sidecars are still required before public release.

Included Workflows

NASA POWER

The NASA POWER screen supports:

  • latitude/longitude/date range/community/time format input;
  • parameter chips such as ALLSKY_SFC_SW_DWN, T2M, and WS2M;
  • quick city examples;
  • live fetch through the NASA POWER API;
  • status cards, normalized table preview, units metadata, CSV/JSON export, and activity log.

Sample acceptance case:

New York
2024-05-01..2024-05-05
daily
ALLSKY_SFC_SW_DWN,T2M,WS2M

Expected result: 5 normalized records. Daily ALLSKY_SFC_SW_DWN is treated as kW-hr/m^2/day, not W/m^2.

PV Estimate

There are two modes:

  • Local quick estimate in Rust. This is approximate and intended for fast screening.
  • PVWatts/NLR mode. This calls the current NLR PVWatts V8 API when a nlr_pvwatts_key is stored.

Local estimate supports capacity, tilt, azimuth, losses, inverter efficiency, source parameter, used/missing record counts, and daily/hourly irradiance unit handling.

NDVI

The NDVI screen accepts:

  • red band TIFF path;
  • NIR band TIFF path;
  • output TIFF path;
  • red/NIR scale factors;
  • explicit nodata value.

Current NDVI output is a Float32 TIFF with NDVI values from (NIR - Red) / (NIR + Red). It handles zero denominators, nodata, scale factors, mismatched dimensions, and TIFF read/write tests across uncompressed, Deflate, LZW, PackBits, and multi-strip layouts. The Rust path preserves common GeoTIFF CRS/geotransform tags from the Red band, including model scale/tiepoint/transformation tags, GeoKey directory tags, Geo ASCII/double parameters, and GDAL_NODATA.

EUMETSAT

The EUMETSAT screen is wired for a bundled EUMDAC command-line sidecar:

  • check_eumdac_sidecar
  • get_eumdac_sidecar_status
  • fetch_eumetsat_products
  • download_eumetsat_product

The app currently looks for executable files named:

eumdac
eumdac.exe
eumdac-cli
eumdac-cli.exe

next to the packaged executable or in the Tauri resource directory. The backend computes the sidecar SHA256 and only trusts it when a matching entry exists in eumdac-sidecar-manifest.json or eumdac-sidecars.json next to the executable or in bundled resources. Packaging scripts run npm run eumdac:prepare, which downloads the pinned EUMDAC 3.1.1 standalone binary for the build platform, verifies archive and binary SHA256 values, stages the Tauri externalBin input, and writes the sidecar manifest. Public release signing/notarization must still cover the staged sidecar.

EUMDAC 3.x exposes set-credentials, search, and download commands. Before search/download, the app reads eumetsat_consumer_key and eumetsat_consumer_secret from the OS keychain and syncs them into an app-scoped EUMDAC config environment for the sidecar process. Search bounding boxes are sent as W,S,E,N coordinates in EPSG:4326 decimal degrees. Process errors are redacted before being returned to the UI.

Credentials

Credential slots are stored in the OS keychain with service name Satellite Data Toolkit.

Slot Purpose Required For
eumetsat_consumer_key EUMETSAT consumer key EUMETSAT sidecar workflows
eumetsat_consumer_secret EUMETSAT consumer secret EUMETSAT sidecar workflows
nlr_pvwatts_key PVWatts/NLR API key PVWatts mode

NASA POWER does not require an API key.

Important: EUMETSAT credential sync is implemented for the sidecar workflow, but final authentication/search/download behavior must be validated against the exact bundled EUMDAC distribution and real credentials. The API Slots test action reports EUMETSAT as ready only when both consumer key and consumer secret are stored and the EUMDAC sidecar is trusted or explicitly allowed for local development.

Local Storage

Runtime data is stored in the Tauri app data directory for identifier com.satellite.datatoolkit.

macOS expected location:

~/Library/Application Support/com.satellite.datatoolkit/

Files created there:

toolkit.sqlite
exports/

Limits:

  • maximum saved dataset records: 120,000;
  • maximum saved/exported dataset payload: 64 MiB;
  • maximum saved dataset name length: 160 bytes.

Development Prerequisites

These are build-time requirements only. End users of the packaged app do not need them.

  • Node.js: .node-version pins 24.13.0; package.json allows >=22.12.0.
  • npm: packageManager pins npm@11.6.2; package.json allows >=11.0.0.
  • Rust: rust-toolchain.toml pins 1.95.0 with clippy and rustfmt.
  • macOS packaging: Xcode Command Line Tools, codesign, hdiutil.
  • Windows packaging: Windows 10/11, MSVC Rust toolchain, Microsoft C++ Build Tools/Windows SDK, WebView2 installer policy, and signing certificate for release builds.

Install Dependencies

On this macOS workstation the dependencies were installed via Homebrew/rustup. A clean setup is:

brew install node rustup
rustup default 1.95.0
rustup component add clippy rustfmt
npm ci

Run From Source

Browser/demo UI only. This uses mocked Tauri responses and is useful for frontend inspection:

npm run dev

Native Tauri development. This uses real keychain, SQLite, exports, NASA POWER requests, and Rust commands:

npm run tauri:dev

Verify

Run all local checks:

./scripts/verify.sh

This currently runs:

  • npm run version:check
  • npm run security:tauri-surface
  • npm run typecheck
  • npm run test
  • npm run build
  • cargo fmt --all -- --check
  • cargo test --workspace --locked
  • cargo check --workspace --locked
  • cargo clippy --workspace --all-targets --locked -- -D warnings
  • npm run security:npm-prod
  • npm run security:npm-build-chain

Npm security policy is split by release risk: production dependencies must have zero known audit findings, while the full dependency tree including dev/build tooling blocks release on high or critical findings.

Build macOS

./scripts/build-macos.sh

Expected outputs:

target/release/bundle/macos/Satellite Data Toolkit.app
target/release/bundle/dmg/Satellite Data Toolkit_2.1.2_aarch64.dmg

Without Apple Developer ID secrets, the script performs a local ad-hoc signature, verifies the .app with codesign --verify --deep --strict, rebuilds the DMG with an /Applications symlink, verifies the DMG with hdiutil verify, and writes a .sha256 checksum next to the DMG. When APPLE_SIGNING_IDENTITY is configured, the script preserves the Tauri-signed output and can require notarization/stapling checks through SATELLITE_REQUIRE_MACOS_NOTARIZATION=1.

The Package artifacts workflow can run this build on GitHub-hosted macOS and upload the DMG/checksum as run artifacts. The Release workflow runs the same build for v* tags and uploads the DMG/checksum to the matching GitHub Release.

For public distribution, ad-hoc signing is not enough. Use Apple Developer ID signing, hardened runtime, notarization, stapling, and Gatekeeper verification.

Build Windows

Run on a Windows build machine:

.\scripts\build-windows.ps1

Expected outputs:

target\release\bundle\msi\
target\release\bundle\nsis\
target\release\bundle\SHA256SUMS.txt

Current status: MSI/NSIS packaging is configured for Windows runners, with checksum output and an MSI quiet install/uninstall smoke test. WINDOWS_SIGN_COMMAND can be set in a Windows environment or GitHub secret to Authenticode-sign Tauri Windows bundle targets through scripts/sign-windows.ps1; without it, signing is explicitly skipped. Before shipping broadly, run MSI and NSIS install/uninstall checks on real Windows 10/11 machines, verify WebView2 behavior, Credential Manager storage, first-run offline behavior, code signing, and SmartScreen reputation.

Local Gates And GitHub Automation

The repository is public, so hosted GitHub Actions are restored for CI and release automation:

  • CI: runs on main, pull requests, and manual dispatch; verifies Ubuntu, macOS, and Windows, then runs visual/browser smoke on Ubuntu.
  • RustSec audit: runs on main, pull requests, weekly, and manual dispatch.
  • Package artifacts: manual workflow that uploads macOS DMG/checksum and Windows MSI/NSIS/checksum artifacts without publishing a release.
  • Release: runs on v* tag pushes or manual dispatch for an existing tag, uploads run artifacts, and creates or updates the matching GitHub Release assets.

Local verification remains the fastest pre-push check:

npm run verify
npm run visual:smoke
./scripts/build-macos.sh

Run .\scripts\build-windows.ps1 on a Windows 10/11 build machine for an extra native Windows packaging gate when needed.

The local release asset set remains:

macOS DMG
Windows MSI
Windows NSIS installer
SHA256SUMS.txt

For release automation, create and push a tag matching package.json, for example v2.1.2. The release workflow validates the tag, builds macOS and Windows assets, uploads workflow artifacts, and publishes release assets plus SHA256SUMS.txt.

Dependabot config remains disabled to avoid noisy automated PRs. Run npm audit, npm outdated, cargo audit, and targeted dependency updates locally or add Dependabot back later if the notification volume is acceptable.

Security reports should follow SECURITY.md. Do not disclose exploit details or secrets in public issues; use GitHub private vulnerability reporting when it is enabled for the repository.

The main branch protection should require the restored hosted checks once the first green run completes. Keep force-pushes and branch deletion blocked on main.

Important repository state as of May 9, 2026: a separate rust-pro-v3.0.0 release exists from the codex/rust-pro-windows-exe branch and points to a portable Rust-only Windows EXE. The Tauri desktop app release line uses v* tags, and v2.1.2 is the current release tag for the restored hosted CI/CD path.

Review ZIP Contents

The handoff ZIP is intended to contain:

  • source code and lockfiles;
  • README.md;
  • docs/;
  • scripts/;
  • macOS Apple Silicon DMG under artifacts/macos/;
  • visual QA screenshots under artifacts/visual/;
  • SHA256SUMS.txt for included artifacts.

It intentionally excludes heavy/generated/local files:

  • node_modules/;
  • target/;
  • dist/;
  • .playwright-cli/;
  • old Python zip;
  • local screenshots;
  • previous zip files.

Known Missing Work Before Public Release

  • Windows install/uninstall QA on Windows 10/11.
  • Apple Developer ID signing, hardened runtime, notarization, stapling, and public Gatekeeper acceptance.
  • Bundled, signed, checksum-verified EUMDAC sidecar per platform.
  • Live EUMETSAT auth/search/download QA with real credentials and the exact bundled sidecar.
  • Live PVWatts/NLR QA with real API key.
  • NDVI QA with real-world tiled and multi-provider GeoTIFF fixtures.

Troubleshooting

If cargo is not found on macOS, make sure Homebrew rustup is on PATH:

export PATH="/opt/homebrew/opt/rustup/bin:$PATH"

If npm ci fails, verify Node and npm versions:

node --version
npm --version

If macOS blocks a local review app downloaded from a ZIP/DMG, remove quarantine only for private testing:

xattr -dr com.apple.quarantine "/Applications/Satellite Data Toolkit.app"

Do not use this as a substitute for Developer ID signing and notarization in a public release.

Reference Links

About

Desktop toolkit for NASA POWER, EUMETSAT, NDVI, and PV energy estimates

Topics

Resources

License

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Packages

 
 
 

Contributors