Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
node_modules
.git
.github
.vscode
.DS_Store

# Don't copy data files into the image — mount them at runtime
data

# SQLite files
*.sqlite
*.sqlite-journal
*.sqlite-wal
*.sqlite-shm

# Build output is generated in the Docker build stage
build
.output
.svelte-kit

# Environment files (use docker env vars instead)
.env
.env.*

# Tests and docs
src/**/*.test.ts
docs
vitest.config.ts
24 changes: 24 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# Shared secret for admin endpoints.
# Required to call /api/admin/* and /api/sync?force=true.
# Pass via Authorization: Bearer <token> header.
ADMIN_SECRET=change-me-to-a-random-string

# Language model for generating site assessment narratives.
# Options: ollama, openai, anthropic
LLM_PROVIDER=ollama

# Model name for the selected provider.
# Ollama default: llama3
# OpenAI default: gpt-4o-mini
# Anthropic default: claude-3-haiku-20240307
LLM_MODEL=llama3

# API keys (only needed for OpenAI and Anthropic)
OPENAI_API_KEY=
ANTHROPIC_API_KEY=

# Ollama base URL (only used when LLM_PROVIDER=ollama)
OLLAMA_BASE_URL=http://localhost:11434

# Interval in minutes between scheduled Bayesian assessments
ASSESS_INTERVAL_MINUTES=360
21 changes: 21 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: CI

on:
push:
branches: [main]
pull_request:
branches: [main]

jobs:
check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- run: npm ci
- run: npm run check
- run: npm run test
- run: npm run build
41 changes: 41 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
# Build stage — compiles native modules and builds the SvelteKit app
FROM node:22-slim AS builder

RUN apt-get update && apt-get install -y --no-install-recommends \
python3 make g++ && rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY package.json package-lock.json .npmrc ./

RUN npm ci

COPY tsconfig.json vite.config.ts vitest.config.ts ./
COPY src/ src/
COPY static/ static/

RUN npm run build

# Production stage — minimal runtime with native module support
FROM node:22-slim

RUN apt-get update && apt-get install -y --no-install-recommends \
python3 make g++ && rm -rf /var/lib/apt/lists/*

WORKDIR /app

COPY package.json package-lock.json ./
RUN npm ci --omit=dev

RUN apt-get remove -y python3 make g++ && apt-get autoremove -y && rm -rf /var/lib/apt/lists/*

COPY --from=builder /app/build build/

RUN mkdir -p /app/data/viirs /app/data/gan

ENV NODE_ENV=production
ENV PORT=3000

EXPOSE 3000

CMD ["node", "build"]
73 changes: 72 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,77 @@ The development server runs at `http://localhost:5173`.
| `npm run check` | Run Svelte type checking |
| `npm run test` | Run unit tests (Vitest) |

## Deployment

### Environment variables

| Variable | Required | Default | Description |
|---|---|---|---|
| `ADMIN_SECRET` | Yes | — | Shared token to protect admin endpoints. Pass via `Authorization: Bearer <token>` header. Without it, sync and ingest endpoints return 401. |
| `NODE_ENV` | No | `development` | Set to `production` to disable the TIFF file reader fallback. In production, VIIRS data must be pre-ingested. |
| `LLM_PROVIDER` | No | `ollama` | LLM provider for site narrative generation. Options: `ollama`, `openai`, `anthropic`. |
| `LLM_MODEL` | No | `llama3` | Model name for the selected provider. |
| `OPENAI_API_KEY` | No | — | Required only when `LLM_PROVIDER=openai`. |
| `ANTHROPIC_API_KEY` | No | — | Required only when `LLM_PROVIDER=anthropic`. |
| `OLLAMA_BASE_URL` | No | `http://localhost:11434` | Ollama server URL. |
| `ASSESS_INTERVAL_MINUTES` | No | `360` | Minutes between scheduled Bayesian assessments. |

### Docker (recommended)

The project includes a `Dockerfile` and `docker-compose.yml` for containerized deployment.

```bash
# Copy your dev database for production use (skip if starting fresh)
cp duskmeter-dev.sqlite duskmeter.sqlite

# Build and start
docker compose up --build -d

# Ingest VIIRS data
curl -X POST "http://localhost:3000/api/admin/ingest-viirs" \
-H "Authorization: Bearer your-secret"

# Run initial sync
curl -X POST "http://localhost:3000/api/sync?force=true" \
-H "Authorization: Bearer your-secret"
```

Volume mounts:
- `./data/viirs:/app/data/viirs` — VIIRS yearly GeoTIFF files
- `./data/gan:/app/data/gan` — Globe at Night cache
- `./duskmeter.sqlite:/app/duskmeter.sqlite` — Database

The container auto-restarts unless explicitly stopped. Set `ADMIN_SECRET` in your environment or a `.env` file before running.

### Standalone Node.js

```bash
npm install
npm run build

# Place your VIIRS yearly file
cp VNL_npp_2025_global_vcmslcfg_v2_c202604011200.average.dat.tif.gz data/viirs/

# Ingest and sync
curl -X POST "http://localhost:3000/api/admin/ingest-viirs" \
-H "Authorization: Bearer your-secret"

curl -X POST "http://localhost:3000/api/sync?force=true" \
-H "Authorization: Bearer your-secret"

# Start
NODE_ENV=production ADMIN_SECRET=your-secret node build
```

The production server listens on port 3000 by default. Set `PORT` to override.

### Health check

```bash
curl http://localhost:3000/api/health
# {"status":"ok","version":"0.1.0","uptime":120,"db":"connected","sites":12,"lastSync":"..."}
```

---

## Data sources
Expand All @@ -94,7 +165,7 @@ To use real satellite data:
```
VNL_npp_{year}_global_vcmslcfg_v2_c*.cf_cvg.dat.tif.gz
```
4. Run the ingestion endpoint: `curl -X POST http://localhost:5173/api/admin/ingest-viirs`
4. Run ingestion: `curl -X POST "http://localhost:3000/api/admin/ingest-viirs" -H "Authorization: Bearer your-secret"`
5. The ingestion pipeline decompresses the gzipped TIFF, reads pixel values at each site location, stores them in the database, and cleans up the temporary file.

A deterministic synthetic fallback is used for any year without a local file. The ingest endpoint takes roughly one to two minutes per yearly file.
Expand Down
20 changes: 20 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
services:
duskmeter:
build: .
ports:
- "3000:3000"
environment:
- NODE_ENV=production
- ADMIN_SECRET=${ADMIN_SECRET:-change-me}
- LLM_PROVIDER=${LLM_PROVIDER:-ollama}
- LLM_MODEL=${LLM_MODEL:-llama3}
- OLLAMA_BASE_URL=${OLLAMA_BASE_URL:-http://host.docker.internal:11434}
volumes:
# Mount your VIIRS files and Globe at Night cache
- ./data/viirs:/app/data/viirs
- ./data/gan:/app/data/gan
# Mount your existing database. If you have duskmeter-dev.sqlite from dev:
# cp duskmeter-dev.sqlite duskmeter.sqlite
# If starting fresh, this file will be created automatically.
- ./duskmeter.sqlite:/app/duskmeter.sqlite
restart: unless-stopped
Binary file modified duskmeter-dev.sqlite
Binary file not shown.
Binary file modified duskmeter-dev.sqlite-shm
Binary file not shown.
Binary file modified duskmeter-dev.sqlite-wal
Binary file not shown.
Binary file modified duskmeter.sqlite
Binary file not shown.
Loading
Loading