Repository navigation
Commit d97aeee
authored
fix(llm): fall back to sampling defaults for options passed as undefined (#1611)
## Description
`sample()` merged the caller's config over the defaults with an object
spread, so an option passed as `undefined` (for example `{ temperature:
settings.temperature }` where the setting is unset) replaced its default
with `undefined`. The native sampler reads these options as required
numbers and throws `INVALID_ARGUMENT`, which fails `generate` and chat
sessions on every LLM.
This change resolves each default with `??`, for the top-level options
and for the `dry` and `xtc` sub-configs, so an `undefined` field behaves
like an omitted one. A fake-runtime test covers the case.
### Introduces a breaking change?
- [ ] Yes
- [x] No
### Type of change
- [x] Bug fix (change which fixes an issue)
- [ ] New feature (change which adds functionality)
- [ ] Documentation update (improves or adds clarity to existing
documentation)
- [ ] Other (chores, tests, code style improvements etc.)
### Tested on
- [ ] iOS
- [ ] Android
TypeScript-only change, verified with the fake-runtime API tests. Not
run on a device.
### Testing instructions
1. `yarn workspace react-native-executorch test
__tests__/extensions/sampler.test.ts`
2. The new test `falls back to the defaults for options passed as
undefined` passes. Without the change to `sampler.ts`, it fails because
`temperature`, `topK`, `topP`, `minP`, `repetitionPenalty`,
`penaltyLastN`, and the `dry`/`xtc` defaults reach the native call as
`undefined`.
### Screenshots
Not applicable.
### Related issues
Fixes #1610
### Checklist
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have updated the documentation accordingly
- [x] My changes generate no new warnings
### Additional notes
The native side reads these options with `getRequiredProperty` in
`cpp/extensions/llm/sampler.cpp`, which accepts a missing key only
through the TypeScript defaults. Documentation needs no update, since
the documented defaults are unchanged.1 parent d35ad55 commit d97aeee
2 files changed
Lines changed: 56 additions & 3 deletions
File tree
- packages/react-native-executorch
- __tests__/extensions
- src/extensions/llm
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
67 | 67 | | |
68 | 68 | | |
69 | 69 | | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
70 | 109 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
171 | 171 | | |
172 | 172 | | |
173 | 173 | | |
| 174 | + | |
| 175 | + | |
| 176 | + | |
174 | 177 | | |
175 | | - | |
176 | 178 | | |
177 | | - | |
178 | | - | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
| 189 | + | |
| 190 | + | |
| 191 | + | |
| 192 | + | |
179 | 193 | | |
180 | 194 | | |
181 | 195 | | |
0 commit comments