Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
83 changes: 43 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
# Akiki
# Akiki Python

Akiki provides local voice activity detection (VAD) and decision models.
VAD returns a speech probability, a speech/silence decision, and timestamps for
each 16 ms frame. Decision models classify text with saved specialists and
return label probabilities through [`DecisionModel`](#decision-models).
Training uses an optional companion package. An experimental teacher API can
request example proposals from a server on your own computer or infrastructure.
Akiki is a Python package for local audio and vision inference and decision models.

Decision models use a frozen base checkpoint and specialist adapters.
The Akiki engine currently supports MiniLM/BERT exports for decision inference.
Support for other model families requires additional inference implementations.
- **Audio** detects speech in 16 kHz mono audio. Get a probability, speech
decision, and timestamps for each 16 ms frame. Choose the bundled Akiki-VAD
model or use local TEN-VAD ONNX weights. [Voice activity detection](#voice-activity-detection-vad)
- **Vision** detects objects in RGB images with Akiki Cam Seen. Each detection
gives the object family, score, and bounding box. [Object detection](#object-detection)
- **Decision models** classify text using a local MiniLM/BERT checkpoint and
saved specialist adapters. [Decision models](#decision-models)

The VAD model is included in the package. VAD runs locally, without a model
download, network connection, or external service. The Akiki engine uses INT8
model inference by default, with an FP32 FFT.
Inference runs on your computer. The package includes the Akiki-VAD model.
Supply the local model files and shared libraries for TEN-VAD and Seen.
Training uses an optional companion package. The experimental teacher API can
request example proposals from a server on your own infrastructure.

## Install

Expand All @@ -32,14 +32,15 @@ Or add it to a uv project:
uv add akiki
```

The base installation uses the Akiki engine on supported hosts.
ONNX and PyTorch provide compatibility fallbacks for VAD.
Install `akiki[onnx]` or `akiki[pytorch]` when you need a compatibility engine.
Native wheels include the Akiki-VAD and MiniLM/BERT libraries.
Install `akiki[onnx]` or `akiki[pytorch]` to use an Akiki-VAD compatibility
engine. See [TEN-VAD setup](#select-the-vad-model) and [Seen setup](#object-detection)
for their local libraries and model files.

### Windows and macOS

Native wheels support Windows x64, macOS Intel, and macOS Apple Silicon.
Each wheel includes the Akiki engine for VAD and decision models.
Each wheel includes the Akiki engine libraries for Akiki-VAD and MiniLM/BERT.
Pip selects the wheel for your Python installation. No engine build is required.
Windows ARM64 wheels are not available.
The macOS wheels target macOS 11.0 or later.
Expand Down Expand Up @@ -90,7 +91,7 @@ To use an existing local MiniLM/BERT classifier, run:
```

Replace `./models/my-classifier` with your local export directory.
The Python VAD and decision-model APIs below are the same on all supported systems.
The Python API examples apply on all supported systems with the required libraries.

#### VAD compatibility fallback

Expand All @@ -110,8 +111,8 @@ On macOS, run:
.venv/bin/python -m akiki vad "./recording.wav" --engine onnx
```

This fallback uses the bundled VAD model. It does not enable decision-model inference.
PyTorch is the second VAD compatibility fallback.
This fallback uses the bundled Akiki-VAD model.
PyTorch is the second Akiki-VAD compatibility fallback.

## Python usage

Expand Down Expand Up @@ -151,23 +152,23 @@ modules remain accessible but are omitted from that list.
The decision-model module is named `akiki.decision`. Existing imports from
`akiki.minilm` remain available for compatibility and use the same classes
and precision enum. Named top-level compatibility imports remain available.
`akiki.vad` and `akiki.VAD` are deprecated and will be removed in version 0.3.0.
The legacy `akiki.vad` and `akiki.VAD` imports remain available in 0.3.0.
Both emit a deprecation warning and are scheduled for removal in 0.4.0.
Use `from akiki.audio import vad` and call `vad.VAD()`.
Wildcard imports expose only `VAD`, `MiniLM`, `audio`, `decision`, `vision`, and `__version__`.

## Object detection

Use `akiki.vision.object_detection` to find objects in an RGB image.
This interface supports `Akiki-AI/akiki-cam-seen`.
It loads the Seen r4c files from the local Hugging Face cache.
Use the Seen r4c model from `Akiki-AI/akiki-cam-seen`.
Set `AKIKI_SEEN_LIBRARY` to the path of your local Akiki Seen shared library,
or pass `library_path` when you create the detector.

Set `HF_HUB_OFFLINE=1` when you use the Hugging Face cache.
Use the Hugging Face extra to locate the cached snapshot.
You can also pass an existing snapshot folder as `model_path`.
Set `AKIKI_SEEN_LIBRARY` to the path of your local Akiki Seen library, or
pass `library_path` when you create the detector.
Pass the directory of your existing model snapshot as `model_path`.
To use a snapshot already in the Hugging Face cache, install
`akiki[huggingface]`, set `HF_HUB_OFFLINE=1`, and omit `model_path`.

This example uses Pillow to read an image file.
Install Pillow to read the image file in this example.

```python
import numpy as np
Expand All @@ -178,7 +179,7 @@ from akiki.vision import object_detection
with Image.open("frame.jpg") as frame:
image = np.asarray(frame.convert("RGB"))

with object_detection.ObjectDetector() as detector:
with object_detection.ObjectDetector(model_path="path/to/seen-snapshot") as detector:
for result in detector.detect(image):
print(result.word, result.score, (result.x0, result.y0, result.x1, result.y1))
```
Expand Down Expand Up @@ -214,7 +215,8 @@ The public API includes docstrings for editor hints and interactive help:
```python
import akiki

help(akiki) # Overview of audio, decision, training, and teacher
help(akiki)
help(akiki.vision.object_detection) # Object detection and result types
help(akiki.audio.vad) # Speech detection, configuration, and result types
help(akiki.decision) # Decision models, specialists, and result types
help(akiki.decision.DecisionModel) # Base-model selection and specialist inference
Expand Down Expand Up @@ -279,9 +281,9 @@ It does not manage decision-model specialists.

### Inference engines

The Akiki engine is the primary engine for VAD and decision-model inference.
Its current decision-model implementation supports MiniLM/BERT exports.
ONNX and PyTorch are VAD compatibility fallbacks.
The Akiki engine is the primary engine for audio, vision, and decision-model
inference. Akiki-VAD supports ONNX and PyTorch compatibility fallbacks.
TEN-VAD supports the Akiki engine and ONNX Runtime.

Linux x86_64 wheels include the Akiki engine for systems with glibc 2.34 or later.
Pip and uv select a compatible wheel automatically. No engine build is required.
Expand All @@ -298,7 +300,7 @@ Version **0.1.4** also includes the Akiki MiniLM engine in the Linux ARM64 wheel
**ARM64 MiniLM support is experimental**, including inference with local Ariadne
adapters. It uses the existing MiniLM API and requires glibc 2.28 or later.

`VAD(engine=Engine.AUTO)` tries the Akiki engine first.
For Akiki-VAD, `VAD(engine=Engine.AUTO)` tries the Akiki engine first.
If it is unavailable, selection tries ONNX, then PyTorch.
The compatibility engines are optional. The base package requires neither runtime.

Expand All @@ -320,16 +322,15 @@ The PyTorch extra does not install ONNX Runtime. Install `akiki[onnx,pytorch]`
if you want both alternatives. If no engine is available, Akiki raises an error
with installation instructions.

Import `Engine` from `akiki` and choose an engine with `VAD(engine=Engine.AKIKI)`,
Import `VAD` and `Engine` from `akiki.audio.vad`. Choose an engine with `VAD(engine=Engine.AKIKI)`,
`VAD(engine=Engine.ONNX)`, or `VAD(engine=Engine.PYTORCH)`. The default is
`Engine.AUTO`; the Python API requires enum members. On the command line, use
`akiki vad recording.wav --engine akiki`, `--engine onnx`, or `--engine pytorch`.
An explicitly selected engine raises an error if it is unavailable. Check
`vad.engine` to see the active `Engine` member. `vad.engine_errors` maps
`Engine` members to the reasons they were unavailable during automatic selection.

All VAD engines use the same bundled model and streaming API.
The compatibility fallbacks apply to VAD.
All Akiki-VAD engines use the same bundled model and streaming API.
Decision-model inference currently requires the Akiki engine and a supported export.

Automatic library discovery checks only the package's `native` directory.
Expand All @@ -339,12 +340,14 @@ on the command line. You can also set an engine-specific environment variable:

| Library | Environment variable |
| --- | --- |
| VAD | `AKIKI_VAD_LIBRARY` |
| Akiki-VAD | `AKIKI_VAD_LIBRARY` |
| TEN-VAD | `AKIKI_TEN_VAD_LIBRARY` |
| Seen object detection | `AKIKI_SEEN_LIBRARY` |
| Decision models (current MiniLM/BERT runtime) | `AKIKI_MINILM_LIBRARY` |

An explicit path takes priority over the environment variable and bundled library.
If the selected library is missing or incompatible, no other library is tried.
VAD with `Engine.AUTO` can still use the ONNX, then PyTorch compatibility fallbacks.
Akiki-VAD with `Engine.AUTO` can use the ONNX, then PyTorch compatibility fallbacks.

### Model precision

Expand Down
15 changes: 9 additions & 6 deletions docs/releasing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,13 @@
The public PyPI description comes from `README.md`. Keep it focused on package
features and usage. Build and publishing instructions belong here.

Akiki provides VAD and decision models.
The primary classes are `akiki.audio.vad.VAD` and `akiki.decision.DecisionModel`.
Current native decision inference supports MiniLM/BERT exports.
The library filenames below retain the name of that implementation.
Akiki provides audio and vision inference and decision models.
Use `akiki.audio.vad.VAD` for voice activity detection.
Use `akiki.vision.object_detection.ObjectDetector` for Seen object detection.
Use `akiki.decision.DecisionModel` for MiniLM/BERT specialist inference.

Native wheels ship the Akiki-VAD and MiniLM/BERT libraries.
Use separate local libraries and model files for TEN-VAD and Seen.

## Branches and release preparation

Expand Down Expand Up @@ -42,8 +45,8 @@ Publishing the GitHub release starts PyPI publication.

Update the version in `pyproject.toml`, `src/akiki/__init__.py`, the version
assertion in `tests/test_vad.py`, and the package version in
`src/akiki/models/release.json` and `src/akiki/models/MODEL_CARD.md`. Preserve the model identity
and weight checksums unless deliberately releasing a different model.
`src/akiki/models/release.json` and `src/akiki/models/MODEL_CARD.md`.
Preserve the model identity and weight checksums unless releasing a different model.

PyPI takes a release's description from its first upload. A description change
requires a new version; an additional wheel does not update an existing page.
Expand Down
10 changes: 6 additions & 4 deletions docs/style.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,12 +14,14 @@ Apply this standard when you revise existing text.

Name the primary inference engine the Akiki engine.
Use `Engine.AKIKI` in Python examples and `--engine akiki` in CLI examples.
ONNX and PyTorch are VAD compatibility fallbacks.
`Engine.AUTO` tries Akiki first, then ONNX, then PyTorch.
For Akiki-VAD, `Engine.AUTO` tries Akiki first, then ONNX, then PyTorch.
TEN-VAD uses the Akiki engine by default. With `Engine.ONNX`, the Akiki
frontend supplies features to ONNX Runtime.
Do not use the implementation language as the engine name.

Describe the two primary interfaces as VAD and decision models.
Use `akiki.audio.vad.VAD` and `akiki.decision.DecisionModel` in API overviews.
Describe the primary interfaces as audio, vision, and decision models.
Use `akiki.audio.vad.VAD`, `akiki.vision.object_detection.ObjectDetector`,
and `akiki.decision.DecisionModel` in API overviews.
Describe MiniLM/BERT as the family supported by the current decision-model runtime.
Use `MiniLM` for direct access to that family.
Keep model discovery, training support, and inference support separate.
Expand Down
6 changes: 3 additions & 3 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ build-backend = "setuptools.build_meta"

[project]
name = "akiki"
version = "0.2.2"
description = "Local voice activity detection, object detection, and decision models"
version = "0.3.0"
description = "Local audio and vision inference and decision models"
readme = "README.md"
requires-python = ">=3.10"
license = "Apache-2.0"
license-files = ["LICENSE", "THIRD_PARTY_NOTICES.txt"]
authors = [{name = "Akiki AI"}]
keywords = ["vad", "audio", "speech", "voice-activity-detection", "decision-models", "intent-classification", "minilm", "sentence-embeddings", "text-classification"]
keywords = ["vad", "audio", "speech", "voice-activity-detection", "vision", "object-detection", "decision-models", "intent-classification", "minilm", "sentence-embeddings", "text-classification"]
classifiers = [
"Development Status :: 3 - Alpha",
"Programming Language :: Python :: 3",
Expand Down
15 changes: 7 additions & 8 deletions src/akiki/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Local voice activity detection, object detection, and decision models.
"""Local audio and vision inference and decision models.

The primary interfaces are ``akiki.audio.vad.VAD``,
``akiki.vision.object_detection.ObjectDetector``, and ``akiki.decision.DecisionModel``.
Expand Down Expand Up @@ -59,15 +59,14 @@
>>> vad.ModelPrecision.INT8 is decision.ModelPrecision.INT8
False

Use ``help(akiki.audio.vad)`` and ``help(akiki.decision)`` to explore each API, or
``help(akiki.audio.vad.VAD)`` and ``help(akiki.decision.DecisionModel)`` for the
primary classes. Use ``help(akiki.decision.MiniLM)`` for direct access to the
currently supported decision-model family.
Use ``help(akiki.audio.vad)``, ``help(akiki.vision.object_detection)``, and
``help(akiki.decision)`` to explore these APIs. Use ``help(akiki.decision.MiniLM)``
for direct access to the current decision-model family.
Explicit model retrieval utilities live in ``akiki.hub``.

Compatibility
-------------
``akiki.vad`` and ``akiki.VAD`` will be removed in version 0.3.0.
``akiki.vad`` and ``akiki.VAD`` will be removed in version 0.4.0.
Use ``from akiki.audio import vad`` and call ``vad.VAD()``.
``akiki.minilm`` remains a compatibility module. It re-exports the decision
API's MiniLM class, result types, and current precision enum.
Expand Down Expand Up @@ -101,7 +100,7 @@
if TYPE_CHECKING:
from .audio.vad import VAD as VAD

__version__ = "0.2.2"
__version__ = "0.3.0"
__all__ = [
"MiniLM",
"VAD",
Expand All @@ -115,7 +114,7 @@
def __getattr__(name: str):
if name == "VAD":
warnings.warn(
"akiki.VAD is deprecated and will be removed in version 0.3.0. "
"akiki.VAD is deprecated and will be removed in version 0.4.0. "
"Use akiki.audio.vad.VAD instead.",
DeprecationWarning,
stacklevel=2,
Expand Down
2 changes: 1 addition & 1 deletion src/akiki/models/MODEL_CARD.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Akiki VAD — release model, seed 1730

Release ID: `teacher-only623-v2-s1730`. Bundled by `akiki==0.2.2`.
Release ID: `teacher-only623-v2-s1730`. Bundled by `akiki==0.3.0`.

CNN + GRU128, **210,561 parameters**, full-waveform FP32 ONNX with explicit
recurrent state. Finite mono float32 audio at 16,000 Hz; **256 new samples per
Expand Down
2 changes: 1 addition & 1 deletion src/akiki/models/release.json
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@
},
"model_id": "teacher-only623-v2-s1730",
"output_hop_ms": 16,
"package_version": "0.2.2",
"package_version": "0.3.0",
"parameters": 210561,
"sample_rate": 16000,
"schema_version": 1,
Expand Down
2 changes: 1 addition & 1 deletion src/akiki/vad.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ def _deprecation_stacklevel() -> int:


warnings.warn(
"akiki.vad is deprecated and will be removed in version 0.3.0. "
"akiki.vad is deprecated and will be removed in version 0.4.0. "
"Use 'from akiki.audio import vad' instead.",
DeprecationWarning,
stacklevel=_deprecation_stacklevel(),
Expand Down
4 changes: 2 additions & 2 deletions tests/test_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ def test_existing_top_level_imports_keep_their_identity(name, expected):
@pytest.mark.parametrize("access", ["from akiki import VAD", "VAD = akiki.VAD"])
def test_legacy_top_level_vad_warns_and_keeps_its_identity(access):
namespace = {"akiki": akiki}
with pytest.warns(DeprecationWarning, match=r"akiki\.VAD .* removed in version 0\.3\.0"):
with pytest.warns(DeprecationWarning, match=r"akiki\.VAD .* removed in version 0\.4\.0"):
exec(access, namespace)
assert namespace["VAD"] is vad.VAD

Expand Down Expand Up @@ -70,7 +70,7 @@ def test_legacy_vad_module_warns_and_keeps_its_exports(access):
[warning] = recorded
assert warning.filename == "<string>", warning.filename
assert warning.category is DeprecationWarning
assert "akiki.vad is deprecated and will be removed in version 0.3.0" in str(warning.message)
assert "akiki.vad is deprecated and will be removed in version 0.4.0" in str(warning.message)
assert legacy is importlib.import_module("akiki.vad")
for name in vad.__all__:
assert getattr(legacy, name) is getattr(vad, name)
Expand Down
2 changes: 1 addition & 1 deletion tests/test_vad.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ def test_default_is_selected_release_checkpoint():

model_dir = Path(akiki.__file__).parent / "models"
metadata = json.loads((model_dir / "akiki-vad.json").read_text())
assert akiki.__version__ == "0.2.2"
assert akiki.__version__ == "0.3.0"
with VAD() as default, VAD(model=Model.AKIKI_VAD) as named:
assert default.model_id == named.model_id == "teacher-only623-v2-s1730"
assert default.push([0.0] * 256) == named.push([0.0] * 256)
Expand Down