Skip to content

fix(llm): fall back to sampling defaults for options passed as undefined - #1611

Merged
msluszniak merged 1 commit into
software-mansion:mainfrom
RaphaelFakhri:fix/sampler-undefined-options
Nov 5, 2026
Merged

msluszniak merged 1 commit into
software-mansion:mainfrom
RaphaelFakhri:fix/sampler-undefined-options

Conversation

@RaphaelFakhri

Copy link
Copy Markdown
Contributor

Description

sample() merged the caller's config over the defaults with an object spread, so an option passed as undefined (for example { temperature: settings.temperature } where the setting is unset) replaced its default with undefined. The native sampler reads these options as required numbers and throws INVALID_ARGUMENT, which fails generate and chat sessions on every LLM.

This change resolves each default with ??, for the top-level options and for the dry and xtc sub-configs, so an undefined field behaves like an omitted one. A fake-runtime test covers the case.

Introduces a breaking change?

  • Yes
  • No

Type of change

  • Bug fix (change which fixes an issue)
  • New feature (change which adds functionality)
  • Documentation update (improves or adds clarity to existing documentation)
  • Other (chores, tests, code style improvements etc.)

Tested on

  • iOS
  • Android

TypeScript-only change, verified with the fake-runtime API tests. Not run on a device.

Testing instructions

  1. yarn workspace react-native-executorch test __tests__/extensions/sampler.test.ts
  2. The new test falls back to the defaults for options passed as undefined passes. Without the change to sampler.ts, it fails because temperature, topK, topP, minP, repetitionPenalty, penaltyLastN, and the dry/xtc defaults reach the native call as undefined.

Screenshots

Not applicable.

Related issues

Fixes #1610

Checklist

  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have updated the documentation accordingly
  • My changes generate no new warnings

Additional notes

The native side reads these options with getRequiredProperty in cpp/extensions/llm/sampler.cpp, which accepts a missing key only through the TypeScript defaults. Documentation needs no update, since the documented defaults are unchanged.

@msluszniak msluszniak self-assigned this Nov 5, 2026
@msluszniak msluszniak added the bug fix PRs that are fixing bugs label Nov 5, 2026

@msluszniak msluszniak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. @barhanc could you also check? I'm doing the analogical PR for other places that are affected by this problem.

@barhanc barhanc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@msluszniak
msluszniak merged commit d97aeee into software-mansion:main Nov 5, 2026
7 checks passed
msluszniak added a commit that referenced this pull request Nov 5, 2026
## Description

Same bug as #1610 in three more places: an option passed as `undefined`
replaced its default.

- `normalize`: native throws `INVALID_ARGUMENT`.
- VAD `detectVoice`: thresholds become `NaN`, silently returns no
segments.
- Chat session per-turn `genConfig`: drops the session default (e.g.
`maxNewTokens` becomes unlimited).

### Introduces a breaking change?

- [ ] Yes
- [x] No

### Type of change

- [x] Bug fix (change which fixes an issue)
- [ ] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)

### Tested on

- [ ] iOS
- [ ] Android

### Testing instructions

- `yarn workspace react-native-executorch test
__tests__/extensions/normalize.test.ts
__tests__/tasks/remainingTasks.test.ts
__tests__/tasks/llmChatSession.test.ts`
- The three new tests fail without the `src/` changes.

### Screenshots

### Related issues

#1610, sampler part fixed in #1611.

### Checklist

- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [x] My changes generate no new warnings

### Additional notes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix PRs that are fixing bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

LLM sampling options passed as undefined override the defaults and make generation throw

3 participants