Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions .github/workflows/sdk.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,15 +65,16 @@ jobs:
- run: pip install build twine
- run: python -m build
- run: twine check dist/*
- name: the 5 design JSONs and the demo CSVs must actually be inside the wheel
- name: the design JSONs and the demo CSVs must actually be inside the wheel
run: |
python - <<'PY'
import glob, zipfile
whl = glob.glob("dist/*.whl")[0]
names = zipfile.ZipFile(whl).namelist()
jsons = [n for n in names
if n.startswith("sfm_analysis/report/designs/") and n.endswith(".json")]
assert len(jsons) == 5, (whl, jsons)
assert len(jsons) == 6, (whl, jsons)
assert any(n.endswith("actogram_takes.json") for n in jsons), jsons
demo_csvs = [n for n in names
if n.startswith("sfm_analysis/report/demo/") and n.endswith(".csv")]
assert len(demo_csvs) == 2, (whl, demo_csvs)
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Report generation itself is cross-platform — it doesn't need a Raspberry Pi or

## About

VFM is a firmware library for the Spatial Foraging Platform node. It provides non-blocking, service-oriented stepper-driven pellet dispensing, CAN bus communication with a base-station command/event/heartbeat protocol.
VFM is a firmware library for the Spatial Foraging Platform node. It provides non-blocking, service-oriented stepper-driven pellet dispensing, and talks to the base station over **CAN** (Controller Area Network) — the shared communication bus every node is wired onto. A **CAN event** is a message a node posted on that bus (Loaded, Pellet Taken, Fault, …).

The Arduino library is the `firmware/` folder (not the repo root). Copy or symlink `firmware/` into `Arduino/libraries/VFM`, or zip that folder and add it via *Sketch → Include Library → Add .ZIP Library…*.

Expand Down
4 changes: 3 additions & 1 deletion firmware/docs/DISPENSE_CYCLE.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,9 @@ still applies, since it is about motion, not the pellet.

## Events

Node → base on CAN ID `0x300 + nodeId`. Byte 0 is the event code.
A **CAN event** is a message the node posts on the CAN bus (the shared
communication wire every node is connected to). Node → base on CAN ID
`0x300 + nodeId`. Byte 0 is the event code.


| Code | Event | Extra payload | Meaning |
Expand Down
11 changes: 10 additions & 1 deletion packages/dev_gui/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# SFM Developer GUI

Python desktop application (DearPyGui) for the **Spatial Foraging Module (SFM)** —
a base station plus multiple **VFM** nodes — over the CAN bus on a Raspberry Pi 5.
a base station plus multiple **VFM** nodes on a Raspberry Pi 5. They share one
**CAN** bus (Controller Area Network): the communication wire all nodes are
connected to. A **CAN event** is a frame a node posted on that bus (Loaded,
Pellet Taken, Fault, sensor edge, …), as opposed to an experiment-engine
(`EXP`) row the GUI invented.

![SFM Developer GUI](docs/GUI.png)

Expand Down Expand Up @@ -377,6 +381,11 @@ The base station keeps a dictionary of discovered modules in

## CAN frame reference

**CAN** is the shared communication bus every feeder node is connected to.
Commands go base → node; a **CAN event** is the opposite direction — a node
telling the bus (and therefore the base station) that something just happened
on that module.

| Direction | CAN ID | Content |
|---------------|------------------|-------------------------------|
| base → node | `0x100 + nodeId` | Command (Dispense, Recover, …) |
Expand Down
8 changes: 5 additions & 3 deletions packages/dev_gui/base_station/experiment/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
# Writing Custom Experiment Templates (Python API)

This is the guide for authoring your own experiments for the SFM dev GUI. An
**experiment** automates the pellet-dispensing nodes over CAN: it decides *when*
to dispense, *which* node(s), and *how* to react to what the nodes and the BNC
sync inputs do.
**experiment** automates the pellet-dispensing nodes over **CAN** (Controller
Area Network) — the shared communication bus every node is connected to. It
decides *when* to dispense, *which* node(s), and *how* to react to **CAN
events** (messages a node posted on that bus: Loaded, Pellet Taken, Fault, …)
and to the BNC sync inputs.

You write experiments in Python. There is **no base class to inherit** — a
template is just a module with a `build(...)` factory that returns a configured
Expand Down
1 change: 0 additions & 1 deletion packages/dev_gui/docs/BASE_STATION_HARDCODED_VALUES.md
Original file line number Diff line number Diff line change
Expand Up @@ -191,7 +191,6 @@ The report generator lives in a separate package,
| 0.25 s | `dedup_window_s` | `report/metrics.py` `dome_bouts` | Collapse `DomeOpened` milestone + InputChanged edge for the same physical open |
| 600 s floor, adaptive above | `window_s` / `min_window_s` | `report/sections/timeline.py` `session_raster_section` | Raster detail-panel width. Adaptive by default: `max(min_window_s, duration/12)`, so a run caps at ~12 panels instead of growing without bound; `min_window_s` raises the short-run floor (free_feeding: 900 s), `window_s` pins a fixed width regardless of duration |
| 2 (days) | (threshold, not a named constant) | `report/sections/timeline.py` `actogram_section` | Minimum distinct calendar days of activity before the actogram renders at all |
| 6:00 / 18:00 | `lights_on` / `lights_off` | `report/designs/*.json` `timeline.actogram` options | Default 12:12 light cycle assumed for actogram night-phase shading |
| 5 / 15 | `pre` / `post` | `report/designs/two_armed_bandit.json` | Reversal-curve trials before/after a block flip |
| 5 | `rolling_window` | `report/designs/two_armed_bandit.json` | Block-curve smoothing window (trials) |
| 8 | `_MIN_TRIALS_FOR_CURVE` | `report/sections/bandit.py` | Skip choice/reversal charts below this many analyzed trials |
Expand Down
96 changes: 90 additions & 6 deletions packages/sfm-analysis/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ sfm-report --list
# One session, opened in your browser when done:
sfm-report EXP-Test-02 --open

# Same report, but the actogram ticks pellet takes instead of presence:
sfm-report EXP-Test-02 --design actogram_takes --open

# Every session for a cohort, combined into one comparative report:
sfm-report "cohortA_*" --combine -o /tmp/cohortA.html
```
Expand All @@ -60,7 +63,7 @@ sfm-report --help
--demo Render the bundled demo session (no rig or log dir needed)
--since, --until Filter by date (YYYY-MM-DD)
--run Only this run_id (a file can hold several)
--design Force a report design instead of resolving by experiment
--design Force a report design: bundled name, or path to a .json file
--align Combined-report time alignment: relative | wall | trial | event:<name>
--out, -o Output file (single report) or directory (multiple)
--no-explorer Skip the embedded interactive timeline (guaranteed script-free, leaner output)
Expand Down Expand Up @@ -172,11 +175,91 @@ panels go print-only: hidden on screen since the embedded explorer
covers the same ground with real zoom, still present in the printed PDF.

`timeline.actogram` — one row per calendar day, time-of-day on the
x-axis, night-phase shaded (`lights_on`/`lights_off` options, default
06:00/18:00) — renders automatically whenever a run spans 2 or more
distinct days; it's the figure that matters for a multi-day experiment,
where session_raster's panels (even adaptively sized) stop being the
right tool. Absent entirely for shorter runs.
x-axis — renders automatically whenever a run spans 2 or more distinct
days; it's the figure that matters for a multi-day experiment, where
session_raster's panels (even adaptively sized) stop being the right
tool. Absent entirely for shorter runs.

It deliberately draws **no light/dark shading**. The rig doesn't record
the facility's light schedule, so any shading would be a fixed
clock-time assumption rendered as though it were measured data. Time of
day is on the axis; apply your own light cycle to it. (For zeitgeber
time in your own analysis, `report.timezones.zeitgeber_time(row,
lights_on=...)` takes the schedule explicitly, where it's your stated
input rather than a silent report-wide default.)

The section heading and figure caption both name the plotted event
(`Actogram — MousePresence Detected` by default) so a printed page is
unambiguous about what each tick is. Ticks default to presence onsets.
You can remap them **without editing the installed package** — see
[Customize the actogram](#customize-the-actogram) below.

## Customize the actogram

After `pip install sfm-analysis` on a laptop, two knobs change which CAN
events become actogram ticks. Names must match `event_name` on
`frame_type == "EVENT"` rows — the same strings as the GUI log
(`Pellet Taken`, `Dome Opened`, `Loaded`, `Fault: Jam`, …). Several
names overlay as **one** series (union of timestamps), not separate
colours. Nothing in `site-packages` needs to be edited.

### HTML report (no file copy)

`actogram_takes` ships in the wheel — presence stays the default; this
name is opt-in only (`sfm-report --list-designs`):

```bash
sfm-report MySession --design actogram_takes --open
```

### Python (any session CSV)

```python
from sfm_analysis.analysis import load_session
from sfm_analysis.report.metrics import activity_by_day

s = load_session("MySession") # name, glob, or path to the CSV
days = activity_by_day(s.run(), event_names=("Pellet Taken",))
for day in days:
print(day.date, len(day.times), day.times[:3])
```

Compare presence vs takes vs dome+takes against the bundled demo, or
pass your own CSV. This is the same recipe after a pip install (no git
checkout):

```bash
python -m sfm_analysis.examples.actogram_by_event
python -m sfm_analysis.examples.actogram_by_event /path/to/MySession.csv
```

### A different event, or several

Copy the shipped design next to your logs and edit `event_names`, then
pass the **path** (so you still do not patch the install):

```json
{ "ref": "timeline.actogram",
"options": { "event_names": ["Dome Opened", "Pellet Taken"] } }
```

```bash
sfm-report MySession --design ./my_actogram.json --open
```

A starting file lives at
[`examples/report_design/designs/actogram_takes.json`](examples/report_design/designs/actogram_takes.json)
(identical to the bundled design). Or from Python:

```python
from pathlib import Path
from sfm_analysis.report import build_session_report

build_session_report(
Path("MySession.csv"),
design="actogram_takes", # bundled name, or a path to your .json
)
```

## Python API

Expand Down Expand Up @@ -315,6 +398,7 @@ scratch:
| [`retrieval_latency_by_node.py`](examples/analysis/retrieval_latency_by_node.py) | Per-node summary stats, stdlib-only and pandas paths side by side |
| [`takes_after_fault.py`](examples/analysis/takes_after_fault.py) | Pellet takes within a window after each fault interval started |
| [`exp_events_by_name.py`](examples/analysis/exp_events_by_name.py) | Inventory a session's experiment-engine (`source=EXP`) events — the starting point for analysing a custom template's own log rows |
| [`actogram_by_event.py`](examples/analysis/actogram_by_event.py) | Remap actogram ticks (presence vs pellet takes vs dome+takes). After pip install: `python -m sfm_analysis.examples.actogram_by_event` |

Every one of them runs standalone against the bundled demo session, no
rig or `--log-dir` needed:
Expand Down
38 changes: 36 additions & 2 deletions packages/sfm-analysis/docs/ANALYSIS_GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,7 @@ doesn't bloat the main log.
| `session` | the session name | Groups rows into files; combined with `run_id` for run-scoping |
| `run_id` | increments each time a session is reopened | A single CSV can hold several runs — see trap #1 |
| `trial` | current trial number, `0` outside a trial | Convenience column; the authoritative trial boundary is the `trial` EXPERIMENT event |
| `source` | `CAN` \| `EXP` \| `BNC` \| `SYS` | `CAN` = node hardware events, `EXP` = experiment-engine events, `BNC` = photogate/beam-break, `SYS` = base-station lifecycle |
| `source` | `CAN` \| `EXP` \| `BNC` \| `SYS` | `CAN` = a **CAN event**: a message on the communication bus all nodes share (node hardware). `EXP` = experiment-engine. `BNC` = base-station photogate/beam-break. `SYS` = base-station lifecycle |
| `direction` | `TX` \| `RX` \| `SYS` \| `LOCAL` | Bus direction for CAN frames; not meaningful for EXP rows |
| `node_id` | which node (`0` = broadcast / session-scope, not a real node) | |
| `frame_type` | `EVENT` \| `COMMAND` \| `HEARTBEAT` \| `PELLET_AUDIT` \| ... | What kind of frame this is, independent of `event_name` |
Expand All @@ -77,9 +77,14 @@ doesn't bloat the main log.

## 3. Event vocabulary

**CAN** (Controller Area Network) is the shared communication bus every feeder
node is wired onto. A **CAN event** is a frame a node posted on that bus —
`Loaded`, `Pellet Taken`, `Fault`, a sensor edge — as opposed to an
experiment-engine (`EXP`) row the base station invented.

### CAN events (`source == "CAN"`, `protocol.CanEvent`)

One node-hardware event per row. `event_name` is the *display* name
One node-hardware event per row, received on the CAN bus. `event_name` is the *display* name
(`protocol.CAN_EVENT_DISPLAY_NAME`), not the enum member name — use
`LogRow.can_event` when you need the underlying enum back (see the dome
trap below).
Expand Down Expand Up @@ -283,6 +288,35 @@ event, in order. Feed straight to a step chart.
midnight, sorted) — see [§7](#7-time-timezone-and-time-of-day) for why this
needs no UTC offset.

`activity_by_day(run, event_names=...)` selects which CAN EVENT display
names count as ticks (default: `("MousePresence Detected",)`). The
printed actogram (`timeline.actogram`) passes the same knob through from
design JSON `options.event_names`. Multiple names are pooled into one
series. Days with zero matching events are omitted, not drawn as a blank
row. The section title names those events (`Actogram — Pellet Taken`),
as does the figure's own SVG `<desc>`.

The actogram draws no light/dark shading: the rig doesn't record the
facility's light schedule, so shading it would render a fixed
clock-time assumption as though it were measured data. Time of day is
on the axis — apply your own light cycle to it, or use
`zeitgeber_time(row, lights_on=...)` ([§7](#7-time-timezone-and-time-of-day)),
where the schedule is an explicit input you supply.

A pip-installed user does not edit the package:

```bash
python -m sfm_analysis.examples.actogram_by_event
python -m sfm_analysis.examples.actogram_by_event /path/to/MySession.csv
sfm-report MySession --design actogram_takes --open
```

`actogram_takes` ships in the wheel and is opt-in only (it is not
auto-selected). To plot a different event, copy
[`examples/report_design/designs/actogram_takes.json`](../examples/report_design/designs/actogram_takes.json)
next to your logs, edit `event_names`, and pass the path to
`--design`. See the README section *Customize the actogram*.

## 6. Tidy-table columns

`sfm_analysis.analysis.tables` — every function returns `list[dict]`, one
Expand Down
17 changes: 17 additions & 0 deletions packages/sfm-analysis/examples/analysis/actogram_by_event.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/usr/bin/env python3
"""actogram_by_event.py — map which CAN events count as actogram ticks.

After ``pip install sfm-analysis`` (no source tree)::

python -m sfm_analysis.examples.actogram_by_event
python -m sfm_analysis.examples.actogram_by_event /path/to/MySession.csv

From a git checkout this file is the same recipe::

python examples/analysis/actogram_by_event.py
"""

from sfm_analysis.examples.actogram_by_event import main

if __name__ == "__main__":
main()
16 changes: 16 additions & 0 deletions packages/sfm-analysis/examples/report_design/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,22 @@ cp examples/report_design/sections/alternation.py src/sfm_analysis/report/sectio
`experiment` field is `"alternation"` (the `name` of the experiment
template). Force it with `--design alternation`.

### Actogram event mapping (no package edit)

After `pip install sfm-analysis`, pellet-take ticks are a bundled
design — no file copy:

```bash
sfm-report MySession --design actogram_takes --open
python -m sfm_analysis.examples.actogram_by_event
python -m sfm_analysis.examples.actogram_by_event /path/to/MySession.csv
```

[`designs/actogram_takes.json`](designs/actogram_takes.json) is the same
file as the one in the wheel. Copy it next to your logs only when you
want a *different* event (or several): edit `options.event_names` and
pass the path. Nothing goes into `site-packages`.

Without a matching design, the generic `default.json` still renders —
you do not have to ship a design on day one.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
{
"name": "actogram_takes",
"label": "Generic Behavior Report — actogram from Pellet Taken",
"description": "Same sections as default.json, but the actogram ticks pellet takes instead of presence onsets. Opt-in only: pass --design actogram_takes (ships with pip install sfm-analysis). To plot a different CAN EVENT, copy this file next to your logs, edit event_names, and pass the path. Several names become one tick series.",
"matches": [],
"sections": [
{ "ref": "timeline.explorer" },
{ "ref": "timeline.session_raster" },
{ "ref": "timeline.actogram",
"options": {
"event_names": ["Pellet Taken"]
}
},
{ "ref": "generic.provenance" },
{ "ref": "generic.data_quality" },
{ "ref": "generic.pellet_accounting" },
{ "ref": "generic.retrieval_latency" },
{ "ref": "generic.presence" },
{ "ref": "generic.interaction_funnel" },
{ "ref": "generic.throughput" },
{ "ref": "generic.faults" },
{ "ref": "generic.apparatus_health" }
],
"combined_sections": [
{ "ref": "compare.cohort_table" },
{ "ref": "compare.learning_curve", "options": { "metric": "take_rate" } },
{ "ref": "compare.subject_spread", "options": { "metric": "take_rate" } },
{ "ref": "compare.node_preference" },
{ "ref": "compare.cumulative_overlay" },
{ "ref": "compare.quality_matrix" },
{ "ref": "generic.retrieval_latency" },
{ "ref": "generic.faults" }
]
}
2 changes: 1 addition & 1 deletion packages/sfm-analysis/pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "sfm-analysis"
version = "0.1.0"
version = "0.1.1"
description = "Analysis and printable HTML behavior reports for SFM/VFM feeder session logs"
readme = "README.md"
requires-python = ">=3.9"
Expand Down
2 changes: 1 addition & 1 deletion packages/sfm-analysis/src/sfm_analysis/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,6 @@

from __future__ import annotations

__version__ = "0.1.0"
__version__ = "0.1.1"

__all__ = ["__version__"]
Loading
Loading