Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .cspell-wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ RNET
libexecutorch
libxnnpack
libvulkan
verbalis
xcframework
metallib
ASPP
Expand Down
9 changes: 8 additions & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ jobs:
- name: Check out the submodules the tests build
run: |
git submodule update --init --depth 1 third-party/googletest
GIT_LFS_SKIP_SMUDGE=1 git submodule update --init --depth 1 \
GIT_LFS_SKIP_SMUDGE=1 git submodule update --init --recursive --depth 1 \
packages/react-native-executorch/third-party/common/phonemis
working-directory: ${{ github.workspace }}

Expand Down Expand Up @@ -287,6 +287,13 @@ jobs:
- name: Checkout
uses: actions/checkout@v6

# The podspec suites evaluate the podspec, which requires phonemis'
# phonemis_sources.rb. Nothing reads phonemis' Git LFS `data/`.
- name: Check out phonemis
run: |
GIT_LFS_SKIP_SMUDGE=1 git submodule update --init --depth 1 \
packages/react-native-executorch/third-party/common/phonemis

- name: Setup
uses: ./.github/actions/setup

Expand Down
14 changes: 9 additions & 5 deletions .github/workflows/clang-tidy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,15 @@ jobs:
steps:
- name: Checkout
uses: actions/checkout@v6
with:
# The phonemis submodule carries headers that cpp/extensions/speech
# includes; without it those sources fail to parse and clang-tidy
# reports misleading findings on the broken AST.
submodules: true

# cpp/extensions/speech includes phonemis' headers, and through them
# Verbalis'; without them those sources fail to parse and clang-tidy
# reports misleading findings on the broken AST. phonemis' `data/` is
# Git LFS that nothing here reads, so the checkout skips it.
- name: Check out phonemis
run: |
GIT_LFS_SKIP_SMUDGE=1 git submodule update --init --recursive --depth 1 \
packages/react-native-executorch/third-party/common/phonemis

- name: Setup
uses: ./.github/actions/setup
Expand Down
2 changes: 1 addition & 1 deletion .gitmodules
Original file line number Diff line number Diff line change
Expand Up @@ -3,5 +3,5 @@
url = https://github.com/google/googletest.git
[submodule "packages/react-native-executorch/third-party/common/phonemis"]
path = packages/react-native-executorch/third-party/common/phonemis
url = https://github.com/IgorSwat/Phonemis
url = https://github.com/software-mansion-labs/Phonemis
branch = main
31 changes: 30 additions & 1 deletion LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,17 @@ its own license, reproduced in full further down this file.
OpenCV Apache-2.0 vision pipelines and multimodal LLMs
KleidiCV Apache-2.0 OpenCV Arm HAL, Android only
phonemis MIT grapheme-to-phoneme for text-to-speech
Verbalis MIT text preprocessing for phonemis
MLX MIT MLX backend, iOS only

Which of these end up in an application binary depends on the backends and
libraries it enables; see the Native Libraries documentation. ExecuTorch in
turn bundles further components, among them XNNPACK, pthreadpool, cpuinfo,
the tokenizers library and PCRE2, each under its own permissive license and
covered by the notices shipped with the ExecuTorch distribution.
covered by the notices shipped with the ExecuTorch distribution. Likewise,
Verbalis bundles a port of cn2an (MIT) for Chinese numbers, and phonemis and
Verbalis list their own components in the THIRD_PARTY_NOTICES.md files under
third-party/common/phonemis.


===============================================================================
Expand Down Expand Up @@ -107,6 +111,31 @@ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


===============================================================================
Verbalis, MIT
===============================================================================

Copyright (c) 2026 Software Mansion <swmansion.com>

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.


===============================================================================
MLX, MIT
===============================================================================
Expand Down
1 change: 1 addition & 0 deletions packages/react-native-executorch/compile_flags.txt
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
-isystem../../node_modules/react-native/ReactCommon/jsi
-isystemthird-party/include
-isystemthird-party/common/phonemis/src
-isystemthird-party/common/phonemis/third-party/verbalis/cpp/include
-isystemthird-party/include/executorch/extension/llm/tokenizers/include
-isystemthird-party/include/executorch/extension/llm/tokenizers/third-party/json/include
-isystemthird-party/include/executorch/extension/llm/tokenizers/third-party/re2
Expand Down
44 changes: 30 additions & 14 deletions packages/react-native-executorch/cpp/tests/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -227,18 +227,6 @@ endif()
if(RNE_TESTS_ENABLE_PHONEMIS)
add_subdirectory("${PHONEMIS_DIR}" "${CMAKE_BINARY_DIR}/phonemis" EXCLUDE_FROM_ALL)
list(APPEND RNE_SOURCES ${PHONEMIS_SOURCES})

# Two phonemis headers use std::optional and std::u32string without including
# <optional> / <string>. libc++ pulls both in transitively, so the Android
# (NDK) and iOS builds never notice; libstdc++ does not, and the member
# declarations then fail to parse, leaving a pile of "no declaration matches"
# errors on their definitions. Force-include the two headers rather than patch
# a pinned submodule; drop this once upstream adds the includes.
# SHELL: keeps the two -include flags paired; without it CMake de-duplicates
# the repeated -include and the second header becomes an input file.
if(CMAKE_CXX_COMPILER_ID MATCHES "GNU|Clang")
target_compile_options(phonemis PRIVATE "SHELL:-include optional" "SHELL:-include string")
endif()
endif()

add_library(rne_under_test STATIC ${RNE_SOURCES})
Expand Down Expand Up @@ -296,7 +284,7 @@ if(RNE_TESTS_ENABLE_OPENCV)
endif()

# --- Test support -----------------------------------------------------------
add_library(rne_test_support STATIC support/JsiTestEnv.cpp)
add_library(rne_test_support STATIC support/JsiTestEnv.cpp support/Snapshots.cpp)
target_link_libraries(rne_test_support PUBLIC rne_under_test gtest gmock)
target_include_directories(rne_test_support PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}")

Expand All @@ -306,6 +294,11 @@ target_include_directories(rne_test_support PUBLIC "${CMAKE_CURRENT_SOURCE_DIR}"
target_compile_definitions(rne_test_support PUBLIC
RNE_ERROR_TS_SOURCE="${PACKAGE_DIR}/src/core/error.ts")

# Snapshots of what third-party code produces, committed and updated with
# RNE_UPDATE_SNAPSHOTS=1 (support/Snapshots.h).
target_compile_definitions(rne_test_support PUBLIC
RNE_SNAPSHOT_DIR="${CMAKE_CURRENT_SOURCE_DIR}/snapshots")

# --- Model fixture ----------------------------------------------------------
# Suites that read ExecuTorch MethodMeta need a real .pte. It is downloaded
# rather than committed (scripts/fetch-test-fixtures.sh), so the suites that
Expand Down Expand Up @@ -337,6 +330,25 @@ else()
" scripts/fetch-test-fixtures.sh")
endif()

# With Kokoro's tagger, segmenter and lexicons, phonemis produces real
# phonemes, so what it outputs can be pinned, not only how it is called. Same
# story again.
set(PHONEMIZER_FIXTURE_DIR "${CMAKE_CURRENT_SOURCE_DIR}/fixtures/phonemizer")
set(RNE_HAVE_PHONEMIZER_FIXTURE ${RNE_TESTS_ENABLE_PHONEMIS})
foreach(file en-us/tags.json en-us/lexicon.json en-gb/lexicon.json zh/segmenter.json zh/lexicon.json)
if(NOT EXISTS "${PHONEMIZER_FIXTURE_DIR}/${file}")
set(RNE_HAVE_PHONEMIZER_FIXTURE OFF)
endif()
endforeach()
if(RNE_HAVE_PHONEMIZER_FIXTURE)
target_compile_definitions(rne_test_support PUBLIC
RNE_PHONEMIZER_FIXTURE_DIR="${PHONEMIZER_FIXTURE_DIR}")
elseif(RNE_TESTS_ENABLE_PHONEMIS)
message(WARNING
"Phonemizer fixtures missing, skipping the PhonemizerOutputTest suite. Fetch them with:\n"
" scripts/fetch-test-fixtures.sh")
endif()

# --- Test binaries ----------------------------------------------------------
enable_testing()
include(GoogleTest)
Expand All @@ -359,7 +371,7 @@ if(RNE_TESTS_ENABLE_OPENCV)
endif()

if(RNE_TESTS_ENABLE_PHONEMIS)
list(APPEND TEST_SUITES extensions/PhonemizerTest.cpp)
list(APPEND TEST_SUITES extensions/PhonemizerTest.cpp extensions/PreprocessingTest.cpp)
endif()

if(RNE_HAVE_MODEL_FIXTURE)
Expand All @@ -370,6 +382,10 @@ if(RNE_HAVE_TOKENIZER_FIXTURE)
list(APPEND TEST_SUITES extensions/TokenizerTest.cpp)
endif()

if(RNE_HAVE_PHONEMIZER_FIXTURE)
list(APPEND TEST_SUITES extensions/PhonemizerOutputTest.cpp)
endif()

foreach(suite_path IN LISTS TEST_SUITES)
get_filename_component(suite_name "${suite_path}" NAME_WE)
add_executable(${suite_name} "${suite_path}")
Expand Down
27 changes: 24 additions & 3 deletions packages/react-native-executorch/cpp/tests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,8 @@ rather than a lookalike.
| `support/JsiTestEnv.*` | Fixture owning a Hermes runtime with the module installed, plus `eval*` helpers and the `isCodedError` / `throwsCoded` assertions |
| `core/` | `dtype`, `error`, `conversions`, `tensor`, `schema`, `model`, `utils` |
| `extensions/` | `math`, `speech`, `nlp` (tokenizer), `llm`, and — behind OpenCV — `cv` and `ocr` ops; `phonemizer` behind phonemis |
| `fixtures/` | Downloaded `.pte` program and `tokenizer.json` (gitignored) |
| `fixtures/` | Downloaded `.pte` program, `tokenizer.json` and phonemizer data (gitignored) |
| `snapshots/` | Committed outputs of phonemis, see below |

One binary per suite, so a crash in one area cannot take the run down with it
and `ctest -R MathOpsTest` targets a single suite.
Expand All @@ -62,7 +63,7 @@ RNET_HEADERS_ONLY=1 node scripts/download-libs.js
# 2. The submodules the tests compile. phonemis' `data/` is Git LFS and nothing
# here reads it, so the checkout skips it.
git submodule update --init --depth 1 ../../third-party/googletest
GIT_LFS_SKIP_SMUDGE=1 git submodule update --init --depth 1 third-party/common/phonemis
GIT_LFS_SKIP_SMUDGE=1 git submodule update --init --recursive --depth 1 third-party/common/phonemis

# 3. Hermes + a minimal ExecuTorch host build (several minutes, cached afterwards)
scripts/build-native-test-deps.sh
Expand Down Expand Up @@ -113,14 +114,34 @@ CMake package when present and otherwise locates the two libraries directly.

## The fixtures

`scripts/fetch-test-fixtures.sh` downloads two, both pinned to an exact Hugging
`scripts/fetch-test-fixtures.sh` downloads them, all pinned to an exact Hugging
Face revision and checksum-verified. `run-native-tests.sh` fetches them
automatically; they land in `fixtures/` and are gitignored rather than committed.

- A **.pte program** — selfie-segmentation, ~486 KB, the smallest the org
publishes. Anything reading ExecuTorch `MethodMeta` needs a real program.
- A **tokenizer.json** — Whisper tiny.en's, ~2.4 MB. A plain BPE vocabulary, so
the nlp extension's encode/decode path runs for real.
- Kokoro's **phonemizer data** for en-us, en-gb and zh, the tagger, segmenter
and lexicons the app ships, ~26 MB together. `PhonemizerOutputTest` runs
phonemis on them. The other languages need a neural model, so
`PreprocessingTest` covers only the text phonemis feeds it.

## Snapshots

What phonemis reads a text as changes with every phonemis or Verbalis bump, so
those readings are not hard-coded in the tests. `snapshots/*.txt` lists the
inputs and their committed outputs; a bump that changes one fails
`MatchesTheSnapshot` with the old and new value. Accept the change and review
the diff in the PR:

```bash
RNE_UPDATE_SNAPSHOTS=1 scripts/run-native-tests.sh -R 'PhonemizerOutput|Preprocessing'
```

To add a case, add its `<lang> | <input>` line and run the same command. The
`LeavesNoDigitsOrSymbols` and `LocatesEveryWordOfTheInput` checks hold across
versions and have no snapshot: a failure there is a real regression.

The useful part is that this needs **no XNNPACK delegate**, even though the
fixture is XNNPACK-delegated. `ModelHostObject`'s constructor only calls
Expand Down
Loading
Loading