Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion .agents/skills/blacknode-workflow/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,25 @@ return here to integrate it into a validated workflow.
6. Use `PythonFn` only for workflow-local adapters. Use
`blacknode-development` for reusable nodes or packages.

## Outcome-First Workflow Design

- Begin with the requested outcome and build the shortest coherent graph that
produces it.
- A tracked template must perform a useful end-to-end task. Do not create
templates whose main result is proving a node works, showcasing wiring,
smoke testing, or confirming that a dependency is ready.
- Every visible node must materially create, transform, route, persist,
deploy, or operate something needed by the final result.
- Do not add checker, test, report, echo, or confirmation nodes as graph
padding. The node that owns an operation should validate inputs, preflight
dependencies, report progress, and return actionable errors when practical.
- Keep implementation validation in automated tests or untracked local
developer workflows rather than turning it into a product template.
- Preserve required physical-motion safety, authorization, cost consent, and
destructive-action confirmation. Integrate these controls into the owning
action or managed service when possible; keep a separate node when the
safety contract must be explicit and reusable.

## Available Surfaces

Preferred MCP stdio command:
Expand Down Expand Up @@ -187,7 +206,7 @@ then return a concise graph plan with node ids, node types, key params, edges,
entrypoint, and expected result.

Build loop:
1. Understand the user goal and choose the smallest runnable graph.
1. Understand the user goal and choose the smallest outcome-producing graph.
2. Inspect list_nodes or get_node_schema before using unfamiliar nodes.
3. Create or load a workflow.
4. Add nodes with stable, descriptive ids.
Expand Down Expand Up @@ -221,6 +240,8 @@ Use `list_nodes` for the live catalog. Current core groups:

## Graph Reliability Rules

- Make every node contribute directly to the requested result; never add a
node merely to prove, echo, check, or confirm another node.
- Treat Blacknode workflows as DAGs. Do not create cycles or back-edges.
- Always connect from `outputs` to `inputs`; never invent port names.
- Respect types: `Any` accepts everything, exact type matches are valid, and
Expand Down
20 changes: 20 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,26 @@ under `packages/` are separate Git repositories and carry their own `AGENTS.md`.
- Keep physical motion disarmed by default. Retain stale-data, joint-limit, and
shutdown safeguards in every transport path.

## Outcome-first workflows and templates

- Ship tracked templates only when they perform a real user or operator task
and produce the intended artifact, service, deployment, model, dataset,
action, or decision.
- Do not add tracked templates whose primary purpose is proving that a node
works, showcasing wiring, smoke testing, or confirming readiness. Put those
checks in automated tests or untracked local developer workflows.
- Keep graphs as short and direct as the outcome permits. Every visible node
must materially create, transform, route, persist, deploy, or operate
something required by the result.
- Do not pad workflows with separate checker, test, report, echo, or
confirmation nodes when the node that owns the operation can validate its
inputs, preflight dependencies, report progress, and return actionable
errors.
- Required physical-motion safety, authorization, cost consent, and
destructive-action confirmation remain mandatory. Prefer integrating these
controls into the owning action or managed service; use a separate node only
when the safety contract must remain explicit and reusable.

## Managed Runtime release decision

Decide whether a managed-device Runtime release is required before completing
Expand Down
13 changes: 13 additions & 0 deletions docs/agent-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -359,6 +359,19 @@ Use these instead:

## Template Workflow

Shared templates are outcome-producing product workflows, not test fixtures or
proof-of-concept graphs. A template should perform a real task and produce the
artifact, service, deployment, model, dataset, action, or decision its name
promises. Keep the graph short and direct, and include only nodes that
materially contribute to that result.

Do not add checker, test, echo, report, or confirmation nodes merely to prove
that another node works. Put implementation checks in automated tests or local
developer workflows. Operation-owning nodes should handle practical preflight
validation, progress, and actionable failures. Required motion safety,
authorization, cost consent, and destructive-action confirmation still apply
and should be integrated into the owning action when the contract allows it.

To make a new shared template:

1. Build and save the workflow in the editor.
Expand Down
4 changes: 2 additions & 2 deletions docs/packages.md
Original file line number Diff line number Diff line change
Expand Up @@ -169,15 +169,15 @@ development:
| `blacknode-agent` | Persistent task memory and executive planning, mission execution, skill selection, confirmation, and review. |
| `blacknode-motion` | Arm and base planning, trajectories, execution, learned policies, arbitration, and motion safety. |
| `blacknode-cuda` | CUDA capability, image-processing, tensor-operation, and optional benchmark components backed by internal kernels. |
| `blacknode-dataset` | Default recording, replay, and validation with optional evaluation, export, and repository publishing. |
| `blacknode-dataset` | Default recording, replay, and validation with optional BlacknodeDataset adapters, evaluation, export, and repository publishing. |
| `blacknode-drivers` | Selectively enabled concrete physical drivers; the `feetech` component provides inert bus configuration, read-only probing, and torque-safe bus primitives. |
| `blacknode-isaac` | Direct closed-loop ACT and compatible PPO evaluation using Isaac Sim articulation state, semantic observations, safety-gated targets, and runtime replay logs. |
| `blacknode-perception` | Camera, tracking, VLM, and spatial-perception components, organized as selectable components. |
| `blacknode-robot` | Robot contracts, profiles, calibration, connected-device discovery and lifecycle, normalized telemetry, driver descriptors, and driver process launch. |
| `blacknode-ros2` | Native DDS graph, topic, service, and diagnostic integration with optional rosbridge and managed processes. |
| `blacknode-runtime` | Authenticated remote deployment, target manifests, process supervision, logs, and rollback on Raspberry Pi, Jetson, and Linux targets. |
| `blacknode-skills` | Task-level follow, pick-place, delivery, docking, and inspection behavior over stable capabilities. |
| `blacknode-training` | Optional dataset checks, managed jobs, checkpoints, policy previews, and deployable policy artifacts for training workloads. |
| `blacknode-training` | Optional managed policy, reinforcement-learning, and OpenPI π0.5 VLA training with deployable model artifacts. |

Keep the layers separate: `blacknode-robot` owns profiles, calibration,
connected devices, normalized telemetry, and the generic robot contract;
Expand Down
6 changes: 4 additions & 2 deletions docs/project-artifacts.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ of its files and native manifest.
| Training run | `blacknode-training` | run ID, output path, phase, progress, losses |
| Checkpoint | `blacknode-training` | checkpoint path, run ID, step |
| Policy | `blacknode-training` | policy path, type, source checkpoint, dimensions |
| VLA model | `blacknode-training` | model ID, architecture, provider, base revision, dataset revision, checkpoint digest |
| Replay evaluation | `blacknode-training` | episode, frames, aggregate errors |
| Simulation run | `blacknode-isaac` | run ID, log path, phase, inference counters |

Expand Down Expand Up @@ -56,7 +57,7 @@ Each indexed reference has this provider-neutral shape:
```

`artifact_type` is one of `dataset`, `training_run`, `checkpoint`, `policy`,
`evaluation`, or `simulation_run`. `status` is `available`, `running`,
`model`, `evaluation`, or `simulation_run`. `status` is `available`, `running`,
`completed`, or `failed`.

The ID is deterministic from provider, artifact type, and locator. Repeated
Expand All @@ -78,6 +79,7 @@ The v1 importer understands:
- `blacknode.training-run`
- `blacknode.action-chunking-checkpoint`
- `blacknode.policy-artifact`
- `blacknode.vla-model`
- `blacknode.policy-replay-metrics`
- `blacknode.policy-runtime` from an Isaac node

Expand All @@ -103,7 +105,7 @@ successful node cook into a failed cook.
| Stage | Complete evidence |
|---|---|
| Collect | A linked dataset reports one or more saved episodes |
| Train | A linked policy artifact exists |
| Train | A linked policy or VLA model artifact exists |
| Simulate | A linked simulation run or evaluation is completed |

A created empty dataset, running training job, checkpoint, or running
Expand Down
51 changes: 51 additions & 0 deletions docs/vla-training.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# VLA Training

Blacknode Cloud VLA Training turns a versioned robotics dataset into a persisted
Blacknode model. V0 supports OpenPI π0.5 LoRA training through its native JAX
stack. Warp remains available for simulation and synthetic-data workloads; it
is not part of the supervised π0.5 training path.

## V0 workflow

Use the **OpenPI π0.5 Fine-Tune** template. It contains two outcome-producing
nodes:

1. `LeRobotDataset` resolves a local LeRobot v3 dataset or an immutable Hugging
Face dataset revision as a `blacknode.dataset-source`.
2. `OpenPIFineTune` adapts the source to the OpenPI-pinned LeRobot format,
computes normalization statistics, runs π0.5 JAX LoRA training, and exports
a `blacknode.vla-model`.

The workflow entrypoint is the trained `model` output. It has no confirmation,
checker, or pass-through output nodes. Required dataset validation and artifact
integrity checks run inside the nodes that own those responsibilities.

For a remote source, replace `PIN_DATASET_COMMIT` with the dataset repository's
immutable commit SHA before submitting the workflow. The Cloud executor uses an
NVIDIA L40S profile for this V0 workload.

## BlacknodeDataset boundary

`BlacknodeDataset` is the model-independent data boundary. Its lazy adapter
exposes episode metadata, timestamps, observations, actions, task language,
robot identity, and camera-frame references without loading a complete dataset
into memory. V0 provides native and LeRobot v3 adapters. Future ROS bag,
simulation, and robot-recorder sources can implement the same adapter contract.

The OpenPI provider performs model-specific conversion. This keeps OpenPI,
JAX, checkpoint layout, action transforms, and normalization details outside
the dataset core.

## Model artifact

A completed run writes a `blacknode.vla-model` manifest alongside:

- the LoRA checkpoint archive;
- training configuration and pinned OpenPI revision;
- normalization statistics;
- structured metrics and logs;
- dataset URI and immutable revision;
- inference compatibility metadata.

The artifact remains disarmed: `physical_motion_authorized` is always `false`.
Deployment and real-robot inference are separate, guarded workflows.
37 changes: 36 additions & 1 deletion editor-server/artifact_store.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,8 @@ def _path_locator(value: Any) -> str:
clean = _clean_text(value, maximum=2000)
if not clean:
return ""
if clean.startswith(("blacknode://", "blacknode-cloud://")):
return clean
return str(Path(clean).expanduser().resolve())


Expand Down Expand Up @@ -192,6 +194,38 @@ def _artifact_candidate(
"metrics",
),
)
elif kind == "blacknode.vla-model":
artifact_type = "model"
provider = "blacknode-training"
locator = _path_locator(payload.get("path"))
model_id = _clean_text(payload.get("model_id"), maximum=160)
name = model_id or (Path(locator).name if locator else "VLA model")
status = "completed"
metadata = _safe_metadata(
payload,
(
"model_id",
"owner",
"provider",
"architecture",
"backend",
"base_model",
"base_model_revision",
"dataset",
"training_method",
"step",
"seed",
"action_horizon",
"action_mode",
"checkpoint",
"checkpoint_sha256",
"normalization",
"metrics",
"inference",
"physical_motion_authorized",
"job_id",
),
)
elif kind == "blacknode.policy-replay-metrics":
artifact_type = "evaluation"
provider = "blacknode-training"
Expand Down Expand Up @@ -432,6 +466,7 @@ def inspect_path(
if payload.get("kind") in {
"blacknode.episode-dataset",
"blacknode.policy-artifact",
"blacknode.vla-model",
}:
payload["path"] = str(candidate.parent)
elif not payload.get("path"):
Expand Down Expand Up @@ -503,6 +538,6 @@ def _save(self, records: dict[str, dict[str, Any]]) -> None:
def _hydrate(record: dict[str, Any]) -> dict[str, Any]:
locator = str(record.get("locator") or "")
exists = True
if locator and not locator.startswith("blacknode://"):
if locator and not locator.startswith(("blacknode://", "blacknode-cloud://")):
exists = Path(locator).exists()
return {**record, "exists": exists}
44 changes: 43 additions & 1 deletion editor-server/cloud_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
import urllib.request
from collections.abc import Iterator
from dataclasses import dataclass
from typing import Any
from typing import Any, BinaryIO


@dataclass(frozen=True)
Expand Down Expand Up @@ -92,6 +92,48 @@ def chunks() -> Iterator[bytes]:
return chunks(), media_type, disposition


def upload(
path: str,
stream: BinaryIO,
*,
size: int,
headers: dict[str, str],
authorization: str,
timeout: float = 86_400.0,
) -> dict[str, Any]:
config = configuration()
if not config.available:
raise CloudClientError(503, "Configure BLACKNODE_CLOUD_URL on the editor server.")
request = urllib.request.Request(
f"{config.base_url}{path}",
data=stream,
headers={
"Accept": "application/json",
"Authorization": f"Bearer {authorization}",
"Content-Length": str(size),
"Content-Type": "application/gzip",
**headers,
},
method="PUT",
)
try:
response = urllib.request.urlopen(request, timeout=timeout)
except urllib.error.HTTPError as exc:
message = _error_message(exc)
exc.close()
raise CloudClientError(exc.code, message) from exc
except (OSError, urllib.error.URLError) as exc:
raise CloudClientError(502, "Blacknode Cloud is unreachable.") from exc
with response:
try:
value = json.loads(response.read().decode("utf-8"))
except (UnicodeDecodeError, json.JSONDecodeError) as exc:
raise CloudClientError(502, "Blacknode Cloud returned invalid JSON.") from exc
if not isinstance(value, dict):
raise CloudClientError(502, "Blacknode Cloud returned an invalid response.")
return value


def _open(
method: str,
path: str,
Expand Down
8 changes: 8 additions & 0 deletions editor-server/device_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -632,6 +632,14 @@ def set_deployment_motion_armed(
timeout=15.0,
)

def save_deployment_map(self, deployment_id: str) -> dict[str, Any]:
return self._request(
"POST",
f"{self._deployment_endpoint(deployment_id)}/control",
payload={"command": "save-map"},
timeout=150.0,
)

def ros2_diagnostics(self) -> dict[str, Any]:
return self._request("GET", "/diagnostics/ros2", timeout=90.0)

Expand Down
Loading