Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions docs/user-guide/cli-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ Commands:
sample-frames Sample PNG frames from a JABS project filtered by a behavior label.
sample-pose-intervals Sample contiguous intervals from a batch of JABS pose and video files.
update-pose Update a JABS project to use updated pose files while remapping labels.
update-labels Replace a JABS project's labels with labels imported from another project.
```

For full documentation of the `convert-to-nwb` command, including output modes, subjects
Expand Down Expand Up @@ -208,6 +209,41 @@ After a successful live pose update, features are regenerated automatically only
jabs-cli update-pose /path/to/project /path/to/updated_pose_dir --min-iou-thresh 0.5
```

If instead you want to import labels from another JABS project while keeping the target's pose untouched, see [`jabs-cli update-labels`](#jabs-cli-update-labels).

## jabs-cli update-labels

The `jabs-cli update-labels` command is the inverse of [`update-pose`](#jabs-cli-update-pose): instead of keeping the target's labels and replacing its pose, it keeps the target's pose and replaces its labels with labels imported from another JABS project. The source project provides both the labels and the pose used for IoU-based identity matching.

**Usage:**

```bash
jabs-cli update-labels <project_dir> <source_project_dir> [--min-iou-thresh <FLOAT>] [--verbose] [--annotate-failures] [--drop-timeline-annotations]
```

- `<project_dir>`: Path to the target JABS project whose labels will be replaced in place. The target's pose is unchanged. If `<project_dir>` is a directory of videos + pose files with no `jabs/` subdirectory, a minimal JABS project is scaffolded automatically — features are not generated, so you may want to run `jabs-init` separately afterwards.
- `<source_project_dir>`: Path to a JABS project providing the replacement labels and the pose used for IoU matching. Must already be a valid JABS project; every source-labeled video must also exist in the target.
- `--min-iou-thresh <FLOAT>`: Minimum acceptable median IoU for a label remap match. Blocks below this threshold are skipped. Default: `0.5`.
- `--verbose`: Print successful label remap assignments in addition to warnings.
- `--annotate-failures`: Add timeline annotations to the target for blocks whose label remap fails. Annotations use the same `behavior-remap-failed` / `not-behavior-remap-failed` tags as `update-pose`; the description text distinguishes the originating operation.
- `--drop-timeline-annotations`: Discard source timeline annotations instead of copying or remapping them.

Before modifying the project, the command validates both inputs, runs the label remap in disposable staging projects, and creates a timestamped backup zip under `<project_dir>/.backup` covering `jabs/project.json`, annotations, and predictions (pose files are not touched). Labels are processed block by block, matched by median bbox IoU between the source pose and the target's existing pose, and written to the staged destination label track. By default, source timeline annotations are also carried forward and remapped the same way as label blocks; use `--drop-timeline-annotations` to discard them.

Existing target labels for videos that the source does not cover are left untouched (per-video replace). Behaviors named in the source's `project.json` but not present in the target are merged into the target's `project.json` so the imported labels are usable in the GUI; behaviors already configured in the target keep their existing settings.

The target's pose is unchanged, so the feature cache stays valid and is **not** regenerated. Predictions are cleared because they are stale relative to the new labels; classifiers, the performance cache, and feature files are all left in place. If you need to retrain after a label import, run training from the GUI or via `jabs-classify`.

If a failure occurs after the live apply begins, the command prints the backup path plus cleanup and manual restore instructions instead of restoring automatically.

**Example:**

```bash
jabs-cli update-labels /path/to/target_project /path/to/source_project --min-iou-thresh 0.5
```

If instead you want to replace the target's pose while keeping its labels, see [`jabs-cli update-pose`](#jabs-cli-update-pose).

## jabs-cli sample-frames

The `jabs-cli sample-frames` command extracts individual video frames as PNG images
Expand Down
2 changes: 2 additions & 0 deletions packages/jabs-core/src/jabs/core/utils/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

from .update_checker import check_for_update, is_pypi_install
from .utilities import (
copy_file_atomic,
get_bool_env_var,
hash_file,
hide_stderr,
Expand All @@ -11,6 +12,7 @@

__all__ = [
"check_for_update",
"copy_file_atomic",
"get_bool_env_var",
"hash_file",
"hide_stderr",
Expand Down
29 changes: 29 additions & 0 deletions packages/jabs-core/src/jabs/core/utils/utilities.py
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import hashlib
import os
import re
import shutil
import sys
from collections.abc import Generator
from contextlib import contextmanager
Expand Down Expand Up @@ -86,6 +87,34 @@ def pose_file_stem(path: str | Path) -> str:
return _POSE_SUFFIX_RE.sub("", Path(path).stem)


def copy_file_atomic(source: Path, destination: Path) -> None:
"""Copy a file to ``destination`` so the replacement is atomic.

The file is copied (via :func:`shutil.copy2`) into a sibling temporary file
in ``destination``'s parent directory, then renamed into place with
:meth:`pathlib.Path.replace`. ``destination``'s parent directory is created
if it does not exist.

Because the temporary file is created in the same directory as
``destination``, the final rename is on the same filesystem and is
therefore atomic on POSIX and Windows. ``source`` may live on a different
filesystem (e.g. a ``tempfile.TemporaryDirectory()`` on ``tmpfs``); the
intermediate copy is what makes cross-filesystem sources safe.

Readers of ``destination`` never observe a partially written file: they
see either the previous contents or the new contents.

Args:
source: Path to the file whose contents should be installed at
``destination``. Metadata is preserved via :func:`shutil.copy2`.
destination: Final path. Any existing file at this path is replaced.
"""
destination.parent.mkdir(parents=True, exist_ok=True)
tmp_path = destination.with_suffix(destination.suffix + ".tmp")
shutil.copy2(source, tmp_path)
tmp_path.replace(destination)


def to_safe_name(behavior: str) -> str:
"""Create a version of the given behavior name that is safe to use in filenames.

Expand Down
63 changes: 62 additions & 1 deletion packages/jabs-core/tests/test_utilities.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
"""Unit tests for jabs.core.utils.utilities module."""

import os
from pathlib import Path

import pytest

from jabs.core.utils import pose_file_stem
from jabs.core.utils import copy_file_atomic, pose_file_stem


@pytest.mark.parametrize(
Expand Down Expand Up @@ -50,3 +51,63 @@ def test_pose_file_stem_video_and_pose_match() -> None:
video file (jabs-init, GUI).
"""
assert pose_file_stem("video.mp4") == pose_file_stem("video_pose_est_v6.h5")


def test_copy_file_atomic_replaces_existing_file(tmp_path: Path) -> None:
"""The destination's contents should be overwritten by the source's contents."""
source = tmp_path / "source.txt"
destination = tmp_path / "destination.txt"
source.write_text("new contents")
destination.write_text("old contents")

copy_file_atomic(source, destination)

assert destination.read_text() == "new contents"


def test_copy_file_atomic_creates_missing_parent_dirs(tmp_path: Path) -> None:
"""Missing parent directories of the destination should be created."""
source = tmp_path / "source.txt"
destination = tmp_path / "nested" / "dir" / "destination.txt"
source.write_text("payload")

copy_file_atomic(source, destination)

assert destination.read_text() == "payload"


def test_copy_file_atomic_does_not_leave_temp_file_on_success(tmp_path: Path) -> None:
"""A successful copy must leave no ``.tmp`` sibling behind."""
source = tmp_path / "source.txt"
destination = tmp_path / "destination.json"
source.write_text("payload")

copy_file_atomic(source, destination)

assert not (tmp_path / "destination.json.tmp").exists()
assert set(tmp_path.iterdir()) == {source, destination}


def test_copy_file_atomic_preserves_mtime(tmp_path: Path) -> None:
"""``shutil.copy2`` semantics: file metadata such as mtime should be preserved."""
source = tmp_path / "source.txt"
destination = tmp_path / "destination.txt"
source.write_text("payload")
original_mtime = 1_700_000_000.0
os.utime(source, (original_mtime, original_mtime))

copy_file_atomic(source, destination)

assert destination.stat().st_mtime == pytest.approx(original_mtime, abs=1.0)


def test_copy_file_atomic_handles_file_without_extension(tmp_path: Path) -> None:
"""A destination with no suffix should still receive an atomic replacement."""
source = tmp_path / "source"
destination = tmp_path / "destination"
source.write_text("payload")

copy_file_atomic(source, destination)

assert destination.read_text() == "payload"
assert not (tmp_path / "destination.tmp").exists()
36 changes: 36 additions & 0 deletions src/jabs/resources/docs/user_guide/cli-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ Commands:
sample-frames Sample PNG frames from a JABS project filtered by a behavior label.
sample-pose-intervals Sample contiguous intervals from a batch of JABS pose and video files.
update-pose Update a JABS project to use updated pose files while remapping labels.
update-labels Replace a JABS project's labels with labels imported from another project.
```

For full documentation of the `convert-to-nwb` command, including output modes, subjects
Expand Down Expand Up @@ -208,6 +209,41 @@ After a successful live pose update, features are regenerated automatically only
jabs-cli update-pose /path/to/project /path/to/updated_pose_dir --min-iou-thresh 0.5
```

If instead you want to import labels from another JABS project while keeping the target's pose untouched, see [`jabs-cli update-labels`](#jabs-cli-update-labels).

## jabs-cli update-labels

The `jabs-cli update-labels` command is the inverse of [`update-pose`](#jabs-cli-update-pose): instead of keeping the target's labels and replacing its pose, it keeps the target's pose and replaces its labels with labels imported from another JABS project. The source project provides both the labels and the pose used for IoU-based identity matching.

**Usage:**

```bash
jabs-cli update-labels <project_dir> <source_project_dir> [--min-iou-thresh <FLOAT>] [--verbose] [--annotate-failures] [--drop-timeline-annotations]
```

- `<project_dir>`: Path to the target JABS project whose labels will be replaced in place. The target's pose is unchanged. If `<project_dir>` is a directory of videos + pose files with no `jabs/` subdirectory, a minimal JABS project is scaffolded automatically — features are not generated, so you may want to run `jabs-init` separately afterwards.
- `<source_project_dir>`: Path to a JABS project providing the replacement labels and the pose used for IoU matching. Must already be a valid JABS project; every source-labeled video must also exist in the target.
- `--min-iou-thresh <FLOAT>`: Minimum acceptable median IoU for a label remap match. Blocks below this threshold are skipped. Default: `0.5`.
- `--verbose`: Print successful label remap assignments in addition to warnings.
- `--annotate-failures`: Add timeline annotations to the target for blocks whose label remap fails. Annotations use the same `behavior-remap-failed` / `not-behavior-remap-failed` tags as `update-pose`; the description text distinguishes the originating operation.
- `--drop-timeline-annotations`: Discard source timeline annotations instead of copying or remapping them.

Before modifying the project, the command validates both inputs, runs the label remap in disposable staging projects, and creates a timestamped backup zip under `<project_dir>/.backup` covering `jabs/project.json`, annotations, and predictions (pose files are not touched). Labels are processed block by block, matched by median bbox IoU between the source pose and the target's existing pose, and written to the staged destination label track. By default, source timeline annotations are also carried forward and remapped the same way as label blocks; use `--drop-timeline-annotations` to discard them.

Existing target labels for videos that the source does not cover are left untouched (per-video replace). Behaviors named in the source's `project.json` but not present in the target are merged into the target's `project.json` so the imported labels are usable in the GUI; behaviors already configured in the target keep their existing settings.

The target's pose is unchanged, so the feature cache stays valid and is **not** regenerated. Predictions are cleared because they are stale relative to the new labels; classifiers, the performance cache, and feature files are all left in place. If you need to retrain after a label import, run training from the GUI or via `jabs-classify`.

If a failure occurs after the live apply begins, the command prints the backup path plus cleanup and manual restore instructions instead of restoring automatically.

**Example:**

```bash
jabs-cli update-labels /path/to/target_project /path/to/source_project --min-iou-thresh 0.5
```

If instead you want to replace the target's pose while keeping its labels, see [`jabs-cli update-pose`](#jabs-cli-update-pose).

## jabs-cli sample-frames

The `jabs-cli sample-frames` command extracts individual video frames as PNG images
Expand Down
2 changes: 2 additions & 0 deletions src/jabs/scripts/cli/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@
from .postprocessing import apply_postprocessing_command
from .sample_frames import sample_frames_command
from .sample_pose_intervals import sample_pose_intervals_command
from .update_labels import update_labels_command
from .update_pose import update_pose_command

# find out which classifiers are supported in this environment
Expand All @@ -48,6 +49,7 @@ def cli(ctx: click.Context, verbose):
cli.add_command(compute_features_command)
cli.add_command(convert_parquet_command)
cli.add_command(update_pose_command)
cli.add_command(update_labels_command)
cli.add_command(sample_pose_intervals_command)
cli.add_command(sample_frames_command)

Expand Down
Loading
Loading