form_v2#686
Open
rubenarslan wants to merge 147 commits into
Open
Conversation
Introduces the plumbing for the form_v2 engine described in plan_form_v2.md.
No runtime behavior change: the new Form unit extends Survey and delegates to
the v1 rendering pipeline unchanged. Phase 0 exists so later phases can branch
renderers on rendering_mode without retrofitting existing Survey units.
- New RunUnit type "Form" (application/Model/RunUnit/Form.php), registered
unconditionally in RunUnitFactory::SupportedUnits so existing Form units
load regardless of the feature flag.
- Admin "Add Form" button gated by $settings['form_v2_enabled'] (default
false); when enabled, appears next to Add Survey with the fa-wpforms icon.
- New admin editor templates/admin/run/units/form.php with a beta banner and
type=Form save URL.
- Form::create() stamps rendering_mode='v2' on the linked SurveyStudy so
future renderers can branch.
- SQL patch 47 adds survey_studies.rendering_mode ENUM('v1','v2') NOT NULL
DEFAULT 'v1'.
Captures the dev-loop learnings from Phase 0 UI verification: config mount path, stale snapshot refs, Xdebug-in-AJAX, mariadb vs mysql client, FK order on run cleanup. Future sessions shouldn't have to rediscover these.
The .playwright-mcp/ directory is written by the Playwright MCP server during UI test sessions; it is a local artifact, not source.
The Form RunUnit now delegates to a new FormRenderer when the linked study
is rendering_mode='v2'. All items render server-side inside <section
data-fmr-page> wrappers in one document; a new client bundle (Alpine + BS5 +
FA6 + Tom-select, scoped under .fmr-form-v2) handles page navigation and
per-page AJAX submission. Legacy Survey units are unaffected.
Server
- New FormRenderer extends SpreadsheetRenderer (OpenCPU-evaluated
showif/value still runs exactly as in v1). Emits v2 markup + header/nav.
- Form::getUnitSessionOutput branches on SurveyStudy.rendering_mode and
returns use_form_v2=true for the controller to pick up.
- Form::create no longer re-points survey_run_units.unit_id at the study's
id (that's a Survey quirk); it stores the link in survey_units.form_study_id
instead, so the RunUnit factory keeps instantiating a Form at request time.
- Form::getStudy / load read the link from form_study_id.
- SurveyStudy declares $rendering_mode so assignProperties actually loads
the column (without this, the property was silently dropped).
- Run::exec and RunSession::executeUnitSession pass use_form_v2 through.
- New endpoint POST /{runName}/form-page-submit
(RunController::formPageSubmitAction) accepts JSON
{page, data, item_views}, validates+saves via
UnitSession::updateSurveyStudyRecord, returns JSON.
Client
- New Webpack entry 'form' → webroot/assets/form/js/main.js, produces
form.bundle.js. Imports scoped BS5 (via npm alias 'bootstrap5' to keep
admin BS3 untouched), FA6, Alpine.js 3, and custom form.scss.
- main.js: native page navigation, MySQL-format timestamps for item_views,
ConstraintValidation-driven validation, fetch() to page-submit with
idempotent redirect-on-success.
- New templates/run/form_index.php loads only the form bundle (not the v1
frontend bundle), selected by RunController when use_form_v2 is set.
DB
- Patch 048 adds survey_units.form_study_id (INT UNSIGNED NULL).
Docs
- CLAUDE.md: inventory of documentation/example_surveys/*.xlsx and
documentation/run_components/*.json fixtures for UI testing.
Verified end-to-end on the dev instance with the enter_email Google Sheet:
participant fills email, clicks Submit, AJAX saves to the results table,
run advances to the next unit.
FormRenderer now groups rendered items by survey_items_display.page
(populated by UnitSession::createSurveyStudyRecord when walking submit-
delimited chunks of the item list), so surveys with multiple submit
buttons render as multiple <section data-fmr-page="N"> wrappers in one
document.
formPageSubmitAction checks the highest page number on the current
unit session and returns {status:"ok", next_page: N+1} while earlier
pages remain; on the final page it still returns {redirect: run_url}
so Run::exec can advance to the next unit.
Client already wired this shape (history.pushState for ?page=N,
in-place section swap); no JS change required.
Per-phase checkboxes after the TL;DR so the plan doubles as a running status doc. No change to the design content itself.
Three Phase 1.5 fixes surfaced by the all_widgets smoke test.
FormRenderer::processItems: v1's SpreadsheetRenderer stops after the
first submit-delimited chunk (v1 renders one page at a time, so that's
intentional). v2 needs every chunk in one document, so override
processItems to fetch all unanswered items via a new
getAllUnansweredItems() query and run showif/value/label OpenCPU calls
in one batch. Multi-submit surveys (e.g. all_widgets with a mid-sheet
'page1' submit) now render 2 sections correctly.
Geopoint: port v1's navigator.geolocation wiring into the form bundle
without webshim/jQuery. The .geolocator button is unhidden on init and
click triggers getCurrentPosition; on success the hidden JSON input and
the visible lat/long input are filled; on failure the visible input
becomes editable for manual entry.
Error display: when the server returns {status:'errors', errors:{name:msg}},
add .is-invalid + .invalid-feedback sibling nodes per offending input,
scroll the first one into view, and fall back to a top-of-form banner
for errors whose target input we can't locate. Array-named inputs
(geoloc[]) are found via a secondary lookup.
Verified end-to-end: 2-page all_widgets form walks page 1 -> server
returns {next_page:2} -> client transitions in-place (URL becomes
?page=2) -> page 2 submits -> run advances.
…erver
Three more Phase 1.5 items off the list.
TomSelect auto-wired on every named <select> inside .fmr-form-v2:
large dropdowns (>20 options or .select2zone) get a search box; small
ones become a styled click-to-open menu. remove_button plugin for
multi-select. Phase 2 may still need per-item tweaks but this gets
partner[], time_period, and the 400-entry timezone select looking right
out of the box.
form.scss BS3->BS5 compat now covers:
- .form-group spacing + dividers between siblings
- .control-label (with nested h1-h6 and p)
- .label-{danger,info,success,warning,default} -> BS5 text-bg equivalents
- .input-group-btn (including the .hidden guard v1 uses before JS is up)
- .has-error: red border + label colour
- radio/checkbox group layout under .controls
- .square/.square80/.blank_button (rating_button, mc_button) as card-style
selectable tiles with :has(input:checked) selected state
- .is-invalid + .fmr-invalid-feedback + .fmr-error-banner for
server-returned error display
- .hidden_debug_message stays hidden
IntersectionObserver stamps .item_shown / .item_shown_relative on each
item only once it actually enters the viewport (threshold 0.25). Each
item self-unobserves after the first hit. Fallback for browsers without
IntersectionObserver: stamp everything immediately when a page becomes
visible (matches the previous behaviour).
Previous collectPayload accumulated every input into a hash, with a last-wins scalar OR a growing array. When a Check item emits a hidden+checkbox pair sharing a name, that turned into an array ["0", "1"], which Check_Item::validateInput reasonably crashed on (h() on an array -> TypeError in htmlspecialchars). Match PHP form parsing explicitly: inputs whose name ends with [] are array fields (mc_multiple, select_multiple, geopoint display); all other names are scalar, last-wins. Also recognize <select multiple> as always-array. Unchecked checkboxes/radios skip, as PHP does.
Reactive showif without server round-trip, using v1's existing regex R->JS transpile (Item::$js_showif) as the expression source. FormRenderer now forces $item->data_showif=true for every item with a showif, so the server emits data-showif="<js expr>" on every such wrapper — not only when v1's setVisibility hid it server-side. Client: on init and on every input/change event, walk the cached list of [data-showif] items, evaluate the expression (via new Function() with-scope over a live answers object), and toggle .hidden class + data-fmr-hidden + style.display + input.disabled on the wrapper. The .hidden class toggle is load-bearing because Bootstrap's .hidden uses display:none !important; inline style alone can't override it. Verified on all_widgets: mc_god (showif 'mc_polytheism == 2') is hidden on load and becomes visible once mc_polytheism is set to 2 by the participant, without any network round-trip. Items whose showif survives v1's regex transpile (most simple comparisons, &&, ||, %contains%, is.na, tail(), current()) now work client-side. Items using R-only constructs still need Phase 3's r() wrapper + server proxy, not yet implemented.
Phase 1.5 most gaps closed during the all_widgets audit (Tom-select, IntersectionObserver, BS5 inline errors, BS3->5 compat, multi-chunk processing, geopoint, PHP $_POST-semantics fix). Remaining: per-form Previous-button opt-in, history.pushState back/forward verification, file/audio/video gating. Phase 3 core shipped (client-side showif reactivity using v1's existing R->JS transpile plus a FormRenderer change that emits data-showif on every showif-bearing item). Remaining: standard library helpers, dedicated showif_js column, r() opt-in, transpiler hardening, Alpine x-show integration.
plan_form_v2.md §13 — architectural and design-level findings from Phase 0 -> 1 -> 1.5 -> 3: form_study_id rationale, Model property declaration requirement, FormRenderer multi-chunk override, page grouping via survey_items_display.page, three-layer use_form_v2 passthrough, Bootstrap coexistence via npm alias, showif reactivity using v1 transpile, v1 .hidden CSS specificity gotcha, client payload $_POST semantics, readonly+required :invalid blindspot, et al. CLAUDE.md — operational subset of the same for future dev sessions: where error logs go with error_to_stderr=1, routing convention (dash->camelCase vs underscore-literal), Form unit identity rules, Tom-select reactivity, all_widgets vs all_widgets_with_values fixture choice.
- .claude/agents/ui-playwright-tester.md is a project-scoped subagent with the full Playwright MCP tool surface, the dev-instance URL, and knowledge of this repo's operational gotchas. Committing so any fresh clone gets it registered automatically. - CLAUDE.md: when to delegate to it (template/view/RunUnit changes needing real-browser E2E) and when not to (single-click/single-snapshot debugging where direct MCP calls from the main agent are faster). - .gitignore: split .claude/ by convention — share agents + team settings.json, ignore .local overrides and agent-memory cache.
Adds an allowlisted server-side R path for showifs the regex transpiler
can't translate. Admin wraps in r(...); the server records the inner
expression in survey_r_calls at first render and emits data-fmr-r-call;
the client POSTs {call_id, answers} to /{run}/form-r-call on input
change. No R source reaches the client.
- sql/patches/049_survey_r_calls.sql: per-study allowlist keyed by
(study_id, expr_hash, slot) so the same expression dedups across items.
- Spreadsheet/RAllowlistExtractor.php: top-level r(...) unwrap (handles
nested parens, string literals, trailing semicolon) + hash upsert via
LAST_INSERT_ID(id).
- Spreadsheet/FormRenderer.php: populates survey_r_calls lazily in
processItems, replaces item showif with unwrapped inner R (so OpenCPU
doesn't call a non-existent r()), clears js_showif (would transpile to
garbage), sets parent_attributes['data-fmr-r-call']. renderV2Header
emits data-rcall-url alongside data-submit-url.
- Controller/RunController::formRCallAction: POST endpoint, verifies
call_id belongs to the session's study, overlays client-provided
answers on tail(survey_name, 1), evaluates via OpenCPU with tryCatch,
returns {result: bool}. Rate-limit deferred (RateLimitService is
email-specific; needs a generic bucket API — Phase 4).
- webroot/assets/form/js/main.js: debounced (300ms) r-call resolver with
seq-guarded stale-response protection and args-dedup; applies
result.result via the same class+display+input.disabled triple the JS
showif path uses.
Smoke test: uploaded documentation/example_surveys/rcall_smoke.xlsx with
a plain showif (trigger == "yes") and an r()-gated one
(r(nchar(trigger) > 2)). Typing "yes" shows both; "no" hides both;
"yo!" correctly keeps plain hidden (not == "yes") but shows gated
(nchar 3 > 2), proving JS and R paths diverge as expected.
plan_form_v2.md §0.5 updated; §13 gains learnings 13.19–13.21
(render-time vs import-time allowlist, mandatory r() unwrap before
OpenCPU, DB_Select::fetch() vs DB::findRow()).
Default JSON page-submit can't carry file bytes, so the client now
auto-switches to multipart FormData when the current page has a file
input with a selected file; server branches on Content-Type.
- webroot/assets/form/js/main.js::submitPage: detect non-empty file
inputs, build FormData with `data[name]`, `data[name][]`,
`item_views[bucket][id]` flat keys paralleling the JSON shape, plus
`files[name]` for blobs. data[name] is deleted for file items so
File_Item reads $_FILES, not $_POST.
- Controller/RunController::formPageSubmitAction: Content-Type-sniff
multipart/form-data; when multipart, read $_POST + $_FILES and
re-project file entries into the flat {name,type,tmp_name,error,size}
shape File_Item::validateInput expects. JSON path unchanged.
Smoke test (documentation/example_surveys/file_smoke.xlsx): uploaded a
33-byte text file via a Form unit; resulting row lands in
s<id>_file_smoke with `attach` pointing at the stored file URL, and
user_uploaded_files has the path+original_filename record.
plan_form_v2.md §0.5 (Phase 2) updated: geolocation and submit handling
were already done; file upload now too. §13 gains learnings 13.22–13.24
(multipart $_FILES['files'] namespace, Playwright MCP upload path
restriction, study subdomain DNS + session-reset notes).
Corrects the now-stale claim that R-only showifs have no client-side path (they do now: r(...) opt-in via /form-r-call), and adds five operational notes collected during this session's smoke tests: - r() mechanics: unwrap before OpenCPU, allowlist at render time (not import), LAST_INSERT_ID(id) dedup, debounced+seq-guarded client. - DB::select()->fetch() vs DB::findRow() — fetchRow doesn't exist on either class; linter won't catch, only end-to-end smoke will. - FormData auto-switch for file inputs, with files[<name>] namespace kept outside data[] so File_Item::validateInput sees the canonical $_FILES shape. - Playwright MCP browser_file_upload path restriction (repo root or .playwright-mcp only). - Participant URL form on this dev (study.researchmixtape.com/<run>/ not <run>.researchmixtape.com/) and session-reset recipe when a run without a Stop unit dangles. plan_form_v2.md §13.19–13.24 already captured the same in the earlier two Phase 3 / Phase 2 commits.
Inject isNA/answered/contains/containsWord/startsWith/endsWith/last helpers into every showif eval context (answer keys shadow helpers by ordering). Also rewrite v1's `(typeof(X) === 'undefined')` regex- transpile output to `isNA(X)` client-side: collectAnswers normalizes empty/unchecked inputs to null, not undefined, so v1's emitted check silently never fires on the v2 client.
FormRenderer detects r(...) wrap on the `value` column, unwraps via
RAllowlistExtractor with slot='value', clears $item->value so
needsDynamicValue() returns false and the OpenCPU batch skips the item
(r() isn't an R function; passing the wrapped string torches the whole
batch). The item wrapper emits data-fmr-fill-id; the client POSTs
{call_id, answers} to /{run}/form-fill once on load and sets the
first named input/textarea/select value (only if empty — don't clobber
user back-nav state), firing input+change so showifs re-evaluate. On
OpenCPU error the wrapper flips to .fmr-fill-error with inline feedback.
/form-fill and /form-r-call now share evaluateAllowlistedRCall() which
enforces slot match, session/study ownership, and runs the same R
overlay + tryCatch evaluator. Slot enforcement blocks cross-reuse of a
showif call_id as a fill.
Not yet in scope: survey_r_call_results cache with TTL, per-session
rate-limit (blocked on generic RateLimitService bucket API), embedded
Rmd in labels/page_body. Smoke-tested end-to-end via fill_smoke.csv:
r("hello") fills correctly; r(trigger) with empty trigger resolves to
"" (NA → empty string) as expected for a one-shot deferred fill.
Client: IndexedDB store `formrQueue` with one object store `queue`
(keyPath uuid, index on client_ts). When /form-page-submit fetches
rejects or returns 5xx on a JSON-path page, the payload is persisted
with a client-generated RFC 4122 UUID; the participant continues to
the next page locally and sees a queued banner. On `online` event and
initial page load the queue drains in client_ts order via POST to
/{run}/form-sync. A successful drain deletes the entry; if the queue
empties and the final response carries `redirect`, follow it so the
run advances. drop_entry from the server stops the retry loop; 4xx
validation errors surface the error banner.
Server: patch 050 adds `survey_form_submissions` (uuid unique, FK
CASCADE to unit_session). RunController::formSyncAction accepts one
entry, dedups via pre-check + UNIQUE constraint backstop, applies via
the same UnitSession::updateSurveyStudyRecord path as /form-page-
submit. Regex-validates uuid shape, records client_ts (MySQL DATETIME
format, not ISO-8601), and returns the same status/redirect/next_page
shape as the direct page-submit endpoint.
Not in this slice: service-worker interception, Background Sync, file
(Blob) queueing, opt-out `offline_mode` flag on survey_studies, iOS
Safari pass. Multipart/file pages still alert "offline" without
queueing. Smoke-tested end-to-end by patching window.fetch to reject
for /form-page-submit, submitting, restoring fetch via a (persistent)
iframe, firing `online`, verifying the DB got the participant's
answer and the ledger has exactly one row for the UUID; a duplicate
POST with the same uuid returned 200 without re-applying.
v1's ButtonGroup.js leaned on jQuery + webshim.addShadowDom to mirror
visible buttons against hidden radios/checkboxes. v2's form bundle has
neither dep. Replace with vanilla initButtonGroups() that wires
.btn[data-for] clicks to toggle the paired input (radio: clear
siblings; checkbox/check: toggle independently) and fires `change` so
showifs re-evaluate. Validation: hidden inputs inside .mc-table.js_hidden
still fire native `invalid` events, but display:none kills the browser
tooltip anchor — surface validationMessage as an inline .fmr-btn-feedback
next to the visible .btn-group instead. Clears on next change.
Also re-assert .js_hidden { display:none !important } in form.scss.
v1's frontend bundle ships that rule globally; v2's form.scss doesn't
import it, so without this line every mc_button item renders both the
raw radio list AND the button UI side by side. Ditto .no_js .js_shown
for the fallback.
Covers mc_button, mc_multiple_button, check_button (and their
rating/scale button variants — same DOM). Smoke-tested via
button_mc.xlsx uploaded through a curl cookie jar: radio switch,
checkbox toggle, and required-group invalid feedback all behave as
expected.
Drops the custom `applyShowifs` / `compileShowif` / manual input+change
listener pile (~80 lines) in favor of Alpine's own reactivity:
- `Alpine.data('fmrForm', …)` exposes one top-level reactive field per
form input name plus helper methods (isNA, answered, contains,
containsWord, startsWith, endsWith, last). Vue 3 Proxy tracks new keys
assigned in `init()`, so showif expressions like `trigger == 'yes'`
resolve against `$data` without any scope wrangling.
- `Alpine.directive('showif', …)` runs expressions through `evaluateLater`
+ `effect()`. Dep-tracking + re-run come free. Toggles `.hidden` class
+ `style.display` + `input.disabled` (Alpine's `x-show` only touches
`display`, which can't override Bootstrap's `.hidden !important`).
- Server emission stays as-is: `Item::render` still emits `data-showif`.
The bundle promotes `data-showif` → `x-showif` and adds
`x-data="fmrForm"` on the form on init, before `Alpine.start()`. No
Item.php / FormRenderer changes needed.
Expression robustness added alongside the refactor:
- Strip `//` and `/* */` comments before wrapping in parens. v1's
`//js_only` marker otherwise commented out our closing paren and
produced a SyntaxError at `new AsyncFunction()` time, silently leaving
affected items in their server-rendered visibility forever.
- Wrap runtime eval in `(()=>{try{…}catch(e){return undefined}})()` so
references to names not in $data (run-level vars like `ran_group`,
items from future pages like `puppy`) fall back to undefined (→
visible) rather than spamming ReferenceError every keystroke.
- On compile failure, force visible so a bad expression never hides an
item permanently.
Smoke-tested via Playwright against rcall_smoke (single showif-gated
item, `trigger == "yes"`) and all_widgets (8+ showif expressions
including `mc_polytheism == 2`, `monotheist == 1 & kitten == 1`,
`(the_needy + suffering_animals) > 100`, and the `//js_only` block that
was the SyntaxError trigger). Reactivity verified in both directions,
zero console errors, r-call path unchanged and still working alongside.
Plan §0.5 Phase 3 Alpine box now ticked; §13.33 captures the refactor
rationale and the two robustness gotchas. CLAUDE.md's "v2 showif
reactivity" note updated to reflect Alpine-driven path.
`bin/form_v2_compat_scan.php <study_id|study_name>` classifies every non-empty showif/value in a study as one of: - empty - r-wrapped (server-evaluated via /form-r-call or /form-fill) - JS-transpile OK (client-evaluated after Item.php's regex pass) - needs r(...) wrap (residual R tokens the client evaluator won't handle) The heuristic scans the post-transpile expression for R-only tokens — ifelse/c/tail/paste/is.na/%in%/%%/NA/`<-`/`$`-access — with string literals stripped so tokens inside labels don't false-positive. Value columns don't get the Item.php transpile (v1 OpenCPU-evaluates them), so the scan checks the raw source for value. Exits 0 if clean, 2 if anything was flagged, so it's usable as a CI gate. Informational only — doesn't mutate `survey_items.showif`. Flagged rows print the source, the transpiled JS (when different), and a suggested `r(...)` wrapping. An opt-in `--auto-wrap` mode that rewrites rows in place would be a reasonable follow-up but has migration implications (SurveyStudy version bump? allowlist auto-populate?) — deferred. Smoke-tested against all_widgets (surfaces the one `ifelse(...)` value in the `monotheist` helper as needing r-wrapping; the 17 showifs including `mc_polytheism == 2`, `(the_needy + suffering_animals) > 100`, and `//js_only` annotated ones all classify as JS-OK), and rcall_smoke (1 r-wrapped / 1 JS-OK / 2 empty).
Adds three new gotchas surfaced during the Alpine-driven showif refactor and compat-scan CLI work: - plan §13.34 — Webpack production mode silently skips the bundle write when output is byte-identical with the previous build. mtime lies; confirm rebuilds with `grep <distinctive-string> bundle.js` instead. - plan §13.35 — Programmatic `radio.checked = true` does NOT trigger sibling uncheck; two radios in the same group can both be `:checked` simultaneously under Playwright/`browser_evaluate`, and Alpine's `_syncInput` ends up reading the wrong value. Fix: explicit uncheck loop, or click the visible `.btn[data-for]` button. CLAUDE.md form_v2 section picks up the same two items plus: - A bullet summarizing the Alpine-driven showif path (replaces the stale "`applyShowifs()`" reference). - A pointer to `bin/form_v2_compat_scan.php` as the upgrade-readiness tool for showif/value classification.
Plan: - Rewrite plan_form_v2.md to match on-branch reality (793 → ~420 lines): status-first, shared mermaid flow, single phase checklist, §13 learnings collapsed (architecture-shaping ones promoted into §2-§5 prose, tactical ones point to CLAUDE.md). Stale references killed: FormController, /form/page-submit paths, showif_js/_r_call_id/_r_call_id columns, SW interception of page-submit, "24 months" sunset, etc. Honest references (form-page-submit, RAllowlistExtractor, patches 047-050, Form extends Survey) present. CHANGELOG: - Catch up [Unreleased] with entries for Phases 2-5 (multipart file upload; button groups without webshim/jQuery; Alpine-driven reactive showif; r(...) opt-in server path via survey_r_calls; compat_scan CLI; deferred fill for r()-wrapped value; page-lifetime offline queue). Previous release notes only covered Phases 0-1. Per-study v2 flags (patch 051): - survey_studies.offline_mode (default 1): when 0, the v2 client skips the IndexedDB queue on network failure and surfaces a hard error. Wired as data-offline-mode on the form root; main.js treats syncUrl as empty when off so both the submit-path enqueue and the drain-on-load no-op. - survey_studies.allow_previous (default 0): when 1, FormRenderer emits the "Previous" page-nav button on non-first pages. - Admin UI: new "Form_v2 settings" section on /admin/survey/<name>, visible only when rendering_mode='v2'. changeSettings() accepts the toggles only for v2 studies so v1-era settings can't silently mutate columns the v1 renderer doesn't read. SurveyStudy::toArray() allowlist fix: - toArray() now includes rendering_mode, offline_mode, allow_previous. Without it, $study->update($settings) quietly dropped the new fields because Model::save() writes only what toArray() returns. CLAUDE.md gains a new gotcha documenting the three touch-points (patch file, public property, toArray entry). Unverified-types notice: - FormRenderer::renderUnverifiedTypesNotice emits an alert-warning above the form header when the study contains audio or video items. Soft notice, not a hard gate — items still render and submit through the File_Item multipart path. Not admin-gated: the participant subdomain is a separate origin and can't see the admin cookie. Verified end-to-end on the dev instance via Playwright MCP: - Admin settings page shows the new section for v2 studies only. - Toggling offline_mode=0 + allow_previous=1 persists via the form POST. - Participant form root carries data-offline-mode="off" / data-allow-previous="on". - With fetch patched to reject /form-page-submit: client alerts the "could not send" message and does NOT POST to /form-sync, confirming the offline-mode gate. - Flipping a test item's type to audio surfaces the notice; reverted.
…g hide
Matches v1's participant-facing aesthetic while keeping the v2 bundle
jQuery-free. Driven by side-by-side screenshots of /widgetsv1 vs
/widgetstest on the dev instance.
Layout (form.scss):
- `.form-group.form-row` becomes flex row: label column (260px right-
aligned, muted-heading + body copy) + controls column (flex-grow).
Stacks vertically under 768px.
- item-note / item-block / item-note_iframe / item-submit collapse to
full-width single column since they have no input side.
- Range and range_ticks: controls-inner is flex with nowrap so v1's
left-label + slider + right-label stays on one row (was wrapping).
- Rating/mc/check/mc_multiple button groups: same flex treatment for
the `.keep-label + .btn-group + .keep-label` pole pattern.
- Progress bar: thinner track, green fill (matches v1's teal-ish), right-
aligned "Page N of M" label; kept sticky under the viewport top.
- Input widths capped to v1 ranges (~360px default, 280px for
email/date/tel/url/color, 560px for textarea) — stops wide inputs
looking like search fields.
- Ditched the always-on `input:invalid` red-halo CSS — required-but-empty
inputs match :invalid immediately which painted radios and text fields
red on page load. Validation is server-authoritative now (same as v1);
`.is-invalid` from applyErrors and the button-group `invalid` handler
still provide the live feedback.
Page-level (unscoped) rules:
- `.hidden_debug_message { display: none !important }` + `.hidden` +
`.js_hidden` moved OUT of the `.fmr-form-v2` scope because the
OpenCPU debug panels render into `.render-alerts` which is a sibling
of the form wrapper. The admin-pasted debug cyan boxes no longer
bleed into the participant view.
- Monkey bar styled (fixed bottom-right pill, BS5 colors, tooltip-free
but native-title'd buttons). BS3 modals the template ships with are
force-hidden since the v2 bundle doesn't run BS3 modal JS.
Icons:
- Added `@fortawesome/fontawesome-free/css/v4-shims.min.css` to the form
bundle so FA4.7 class names in the monkey bar template + Item render
(fa-check-square-o / fa-lightbulb-o / fa-user-md / fa-trash-o / …)
resolve to FA6 glyphs. Without the shim, icons rendered as blank
squares.
Tom-select wiring for select_or_add_one / _multiple:
- v1 used select2 on `input.select2add`; v2 now binds tom-select directly
on the `<input>`, seeds options from `data-select2add`, respects
`data-select2multiple` for multi-item, and opts into free-text entry
UNLESS the wrapper has `.network_select`, `.ratgeber_class`, or
`.cant_add_choice` (lockdown classes for network/referral studies).
maxItems comes from `data-select2maximum-selection-size`; maxLength
from `data-select2maximum-input-length`.
Monkey bar wiring:
- `show_hidden_items`: un-hides showif-hidden `.form-group.hidden`.
- `show_hidden_debugging_messages`: toggles `.hidden` off on OpenCPU
debug panels (overrides the new `!important` page-level rule).
- `monkey` (auto-fill): vanilla port of FormMonkey's default table.
Picks first radio/first checkbox/first select option, plausible
defaults for text/email/url/date/tel/color/number, midpoint for
ranges. Skips notes/blocks/submits and already-hidden items.
Dev experience:
- `templates/run/form_index.php` prefers `webroot/assets/dev-build/`
when the dev bundle exists so `npm run webpack:watch` loops work
without editing the template each time. Falls back to `build/` in
prod (where dev-build/ doesn't exist).
Unverified-types notice broadened:
- FormRenderer::$unverifiedTypes now includes `add_to_home_screen`,
`push_notification`, `request_cookie`, `request_phone` — these
render through Item's HTML but rely on PWAInstaller.js which v2
doesn't import. Admins now get a heads-up banner when they author
a form with any of these.
…polish
Phase 5 + Phase 6 push-through. Verified end-to-end against rcallsmoke
and widgetstest via Playwright MCP on the dev instance.
Service-worker interception + Background Sync:
- Extended webroot/assets/common/js/service-worker.js with the form_v2
drain handler — fmrOpenIDB / fmrQueueGetAll / fmrQueueDelete helpers
matching the page-side store, plus a fmrDrainBackground() that POSTs
each entry to the captured sync URL (multipart when .files present,
JSON otherwise) and a 'sync' listener bound to tag 'form-v2-drain'.
- The v2 bundle registers the SW unconditionally on page load at
/{runName}/service-worker with scope /{runName}/, re-using
RunController::serviceWorkerAction so Service-Worker-Allowed is set.
Sync URL is handed over via postMessage (FMR_REGISTER_SYNC_URL).
- Pre-cache failures (missing PWA manifest etc.) no longer discard the
SW — v2 forms that aren't installable PWAs still need the drain path.
File-blob queueing:
- submitPage now stashes the File object into IDB alongside the JSON
payload when multipart submits fail transiently; drainQueue detects
entries with .files and rebuilds FormData (fields mirror
formPageSubmitAction's multipart branch). formSyncAction Content-Type-
sniffs multipart and re-projects $_FILES['files'][...] the same way.
- 10 MB per-file cap — over-cap attempts surface a hard error rather
than filling IDB quota and blocking the queue.
r-call rate limit:
- Generic token bucket in $_SESSION keyed on the run-session id. 30
calls / 60 s window; overflow returns HTTP 429. Prevents a buggy /
hostile client from hammering OpenCPU through the reactive-showif
proxy. RateLimitService is still email-specific; the bucket is local
to evaluateAllowlistedRCall.
Admin compat-scan UI:
- New action AdminSurveyController::formV2CompatScanAction routes
/admin/survey/<name>/form_v2_compat_scan. Scanner logic extracted to
application/Spreadsheet/FormV2CompatScanner.php so the admin view and
bin/form_v2_compat_scan.php share it; CLI now calls the class.
- Surfaced from the v2 study settings page as a "Run v2 compatibility
scan" button.
UI polish:
- Two-column label-left layout (260 px right-aligned label column +
flex-grow controls column; stacks under 768 px). Progress bar in
BS5 success green, slim and sticky. Page nav below the fold.
- Imported webroot/assets/common/css/custom_item_classes.css into the
form bundle + added 'form-horizontal' to the form wrapper so v1's
admin-choosable modifiers (mc_width*, rotate_label*, mc_vertical,
rating_button_label_width*, hide_label, …) light up.
- Imported fontawesome-free v4-shims so FA4.7 class names in
monkey_bar.php + Item::render resolve to FA6 glyphs.
- .hidden_debug_message hidden unscoped — the OpenCPU debug cyan-box
panels live in .render-alerts (sibling of the form wrapper) and so
weren't caught by the .fmr-form-v2-scoped rule. v2 participants no
longer see admin-pasted debug HTML.
- Monkey bar: all three buttons wired (show-hidden items, show-debug,
auto-fill). Vanilla port of FormMonkey.doMonkey's defaults table.
Fixed bottom-right BS5 pill styling.
- Dropped the always-on :invalid border / box-shadow — required-but-
empty inputs matched :invalid immediately and painted radios and
text boxes red on page load. Validation is server-authoritative as
in v1; .is-invalid + .fmr-btn-feedback still carry live feedback.
Item-type coverage:
- Tom-select on input.select2add (select_or_add_one / _multiple). Seeds
from data-select2add, respects data-select2multiple / -maximum-
selection-size / -maximum-input-length, opts into free-text entry
unless the wrapper has .network_select / .ratgeber_class /
.cant_add_choice.
- Vanilla ports of RequestCookie and RequestPhone wiring (happy path;
QR-code + browser-switch guidance still lives in PWAInstaller.js).
- Range / range_ticks + rating/mc/check/mc_multiple button-group pole
labels stay on one row (flex-nowrap on .controls-inner).
- mc_heading + rotate_label45: item padding-top bump so the -25px
label rotation doesn't clip above the container.
Deep-link + pushState:
- Initial ?page=N lands by matching data-fmr-page rather than array
index, so back-navigation and link-sharing work when the server only
renders the participant's remaining pages.
Dev ergonomics:
- templates/run/form_index.php prefers webroot/assets/dev-build/js/
form.bundle.js when it exists so `npm run webpack:watch` iteration
doesn't need template edits. Falls back to build/ in prod.
Memoize OpenCPU-evaluated r(...) results per (call_id, args_hash) so the
reactive-showif cadence (debounced 300ms, one-per-keystroke) hammering
the same expression + same answers skips OpenCPU. Big practical win:
dev smoke shows 512 ms cold → 29 ms warm for /form-r-call on rcallsmoke
(~18× on cache hits).
Schema:
- patch 052_survey_r_call_results.sql: (call_id, args_hash) UNIQUE,
result_json, created_at DEFAULT CURRENT_TIMESTAMP, FK CASCADE to
survey_r_calls.
Read path (evaluateAllowlistedRCall):
- TTL by slot: 30s showif / 5min value. showif wants short so admins'
iterative edits don't serve stale cache; value is one-shot per load.
- Hash normalizes answers via ksort + JSON, so {a:1,b:2} and {b:2,a:1}
hit the same row.
- Cache hit returns the stored result_json['result'] immediately.
Write path:
- REPLACE on success so a stale row bumps to current timestamp (acts
as LRU-ish refresh without a separate touch). Best-effort: PDO errors
log and continue.
Plan + CHANGELOG:
- Phase 4 checklist: cache + rate-limit now checked; embedded Rmd
deferred-fill remains (requires Rmd-aware evaluator path; cache
softens the cost already).
- §8 P2 list updated to surface the remaining items honestly.
…work
Drops the four PWA-button items from the unverified-types notice. v2
participants now get the same install + push UX as v1, without the
jQuery/webshim/PWAInstaller.js stack.
AddToHomeScreen wiring (form bundle):
- Capture `beforeinstallprompt` at module load (before the participant
reaches the page that hosts the button). Re-fire on click in response
to the user gesture; mark hidden input via the validateInput allowlist
(added / not_added / not_prompted / already_added / ios_not_prompted).
- `display-mode: standalone` short-circuits to `already_added` so the
participant doesn't see the install button when the PWA is already
running from the home screen.
- iOS Safari has no programmatic install API; show inline "tap Share →
Add to Home Screen" guidance and stamp `ios_not_prompted` so the
required-flag still surfaces validation if the admin enforces it.
PushNotification wiring (form bundle):
- Subscribe via the SW's pushManager + `window.vapidPublicKey` (with a
small `urlBase64ToUint8Array` helper). On success, POST the
subscription JSON to `/{run}/ajax_save_push_subscription` and store
the JSON in the hidden input — required items validate against the
endpoint+keys.p256dh+keys.auth shape.
- Optional item path stamps `not_supported` / `permission_denied` /
`not_requested`. Already-subscribed sessions short-circuit on init.
- iOS-specific catch path explains the "install first, then reopen
from home screen (iOS 16.4+)" requirement.
PWA infra in form_index.php:
- Mirrors templates/public/head.php: emits the run's manifest link,
apple-touch-icon set (152/167/192), mobile-web-app-capable +
apple-mobile-web-app-* metas, and `window.vapidPublicKey`. Only when
the run actually has these assets configured (`getManifestJSONPath()`
/ `getVapidPublicKey()` / `getPwaIconPath()` non-empty); otherwise
the v2 page is unchanged.
Verified end-to-end on the dev instance against the user-provided
appstinence-v2 run (after copying the original Appstinence run's PWA
config across so v2 had something to register against): manifest link
present, vapid key on window, SW registered + activated with scope
/appstinence-v2/, AddToHomeScreen + PushNotification buttons render with
their FA icons. Push subscribe path attempts the OS-level permission
prompt; Playwright headless Chrome blocks it (so the catch branch fires
the "install first / check browser settings" message — same path real
users hit when they deny permission).
unverified-types notice now only flags `audio` and `video` (which still
need a real-browser getUserMedia + multipart smoke; documented as P0
remaining work).
Items with a leading/trailing icon emit Bootstrap-3 input-group markup (<span class=input-group-addon><i class=fa></i></span> + a span-wrapped field). The v2 bundle is Bootstrap 5, which lays the .input-group out as a flex row but doesn't style .input-group-addon (it expects .input-group-text) — so the icon rendered as a bare top-aligned glyph, unsized and gapless. Style the addon as a boxed icon: vertically centred, align-self:stretch to match the field height, bordered + tinted, with a 0.4rem gap; the span-wrapped field flexes to fill. Scoped via :has(> .input-group-addon) so button input-groups are untouched. Verified live: addon 48x58 matches the input height, glyph centred, 5px gap.
…ge/audio/video, …) Items whose value lives outside a natively-validated input — a hidden carrier (bot_check token, VAS slider, geopoint coords, audio/video/file recordings) or a readonly field (geopoint) — slipped past the client validator: a required-but- empty one returned valid, the page POSTed, the server rejected it, and the participant landed back on the same page with NO error (the 'can't finish without uploading files, no message' bug). Replace the per-type bot_check/VAS special-cases with one generic required-but-unanswered gate: any required, visible, non-showif-hidden group that isn't 'answered' (a checked option, a chosen file, a non-empty non-readonly named field, or a non-empty value-carrier hidden input) is flagged with a per-type inline message. Scope handles both default (page section) and solo (single seated group). submitPageInner already gates on the validator, so this blocks before the POST instead of bouncing off the server. Verified live: all 30 input-bearing types now surface a required error when empty (geopoint/file/image/audio/video were the gaps); optional items still proceed.
…lightweight lane) Reuses the all_widgets v2 fixture and drives the real bundled validator (window.fmrValidatePage) per item: required+empty must block with an inline error; optional+empty must proceed. Forces each group visible so it works in any layout without toggling the study (no DB mutation, CI-safe). Asserts the previously-broken hidden-value types (geopoint, file, image, audio, video, VAS, bot_check) now gate. Excludes NEVER_EMPTY types (range/range_ticks/color always carry a value; check/check_button submit "0" which the server accepts) — their non-blocking is correct, not a gap. A true end-to-end empty-submit lane on a dedicated all-required fixture follows.
…step after a note) A text step is min-height:100vh with justify-content:center. On iOS the layout viewport doesn't shrink when the soft keyboard opens (only the visual viewport does), so the centred field lands in the lower half BEHIND the keyboard — and since the step already fits 100vh there's no overflow to scroll it up. Most visible on the first text input after a tall note. On visualViewport resize the controller now toggles html.fmr-solo-kbd-open when the keyboard is up; CSS then top-aligns the seated step (justify-content: flex-start) so the label+field sit in the area above the keyboard. Also release the fit-lock and nudge the focused field into view (as before). Reverts to centred + re-locked when the keyboard closes. Needs on-device confirmation (automation can't raise the iOS keyboard).
…) for Brave/mobile
Reported broken in Brave on mobile. v1 and v2 are logically identical (click ->
getCurrentPosition with no timeout and a silent error callback), so this isn't a
v2 logic regression — it's that Brave/mobile increasingly blocks or stalls
geolocation, and our no-timeout + silent no-op left the participant with no
feedback and no obvious way forward.
geopoint.js now: passes options with a 10s timeout (+ maximumAge, no high-
accuracy) so the lookup can't hang; shows a transient 'Locating…' state; and on
denial/timeout/unavailable surfaces a clear hint ('Location access was blocked —
type your location instead'), keeps the field editable and focuses it for manual
entry. Success path unchanged; on success it also clears any required-gating
error. Needs confirmation on real Brave mobile (can't reproduce Brave here).
…d fixture Provisions/uses e2e_all_widgets_req_v2 (all_widgets, every item required, default layout) → run e2e-aw-req-v2 (added to runs.json). Two real-submit tests: - empty submit on page 1 is BLOCKED client-side (no network request) with visible inline errors, incl. the readonly geopoint (the primary bug target) flagged is-invalid, and stays on page 1; - answering the fillable required items clears their errors (gate is per-item responsive; filling never adds errors). Full-advance isn't asserted — page 1 has a required file/image/audio/video + geopoint that can't be satisfied without real uploads/recordings/geolocation. (Fixture provisioned by the ui-playwright-tester agent; its throwaway webroot/e2e_clone_study.php helper was removed.)
…terrupted) Parked work-in-progress so it isn't lost. NOT functional/verified yet: - composer: AltchaOrg\Altcha PHP lib; package.json: altcha widget - BotCheckChallenge rewritten on Altcha (Argon2id when ext-sodium, SHA-256 fallback), session-bound, lazy challenge - BotCheck_Item renders <altcha-widget> with choices->strings - RunController form-bot-challenge endpoint, webpack.config.js worker copy, template wiring, bot-check.js Resume from here; do not merge until verified.
… + webpack noParse Still INCOMPLETE. Finding: altcha's browser build (dist/external) is ESM whose plugin loader uses a dynamic require() webpack can't statically resolve, so bundling via `import 'altcha/external'` throws 'require is not defined' at init (breaking the whole form bundle). noParse doesn't help (invalid on ESM modules). Next: load altcha's standalone build via a <script> tag (CopyPlugin the file + template script + wait for window.$altcha) instead of importing it.
…ipt loading)
Resolves the bundling blocker: altcha's ESM browser build has a plugin loader
that calls a dynamic require() webpack can't resolve ('require is not defined'
at bundle init, breaking the whole form bundle); noParse is invalid on ESM. So
ship altcha's prebuilt dist/external standalone verbatim (webpack CopyPlugin →
js/altcha/altcha.min.js, self-hosted next to the bundle, no CDN) and inject it
as a <script type=module> on demand from bot-check.js, only when a bot_check is
present. The template exposes its URL (altchaScriptUrl) alongside the Argon2id
worker URL; bot-check.js sets the lazy challenge URL, loads the script, then
registers the Argon2id worker.
Verified live on e2e-botcheck-v2: widget registers, fetches /form-bot-challenge,
solves the memory-hard Argon2id PoW (worker), and submit returns {status:ok} —
the server (BotCheckChallenge::verify) accepts the token. Argon2id confirmed
available in the container (ext-sodium).
…the client gate
- feedback.js: bot_check is gated by Altcha's own required checkbox, whose
native message ("Please check this box…") is generic. Special-case
item-bot_check in the offenders loop to show the same domain message as the
server (BotCheckChallenge::verify) so client and server agree.
- bot-check-v2.spec.js: rewrite for the Altcha widget — render/lazy-challenge,
required+unverified blocks with the domain message, trusted-click solves
Argon2id + writes the base64 payload, solved submit stores "verified", forged
payload (client gate bypassed, checkbox ticked) rejected server-side. 5/5.
- bot_check_smoke.php: rewrite as Altcha mint→solve→verify + negatives
(empty/garbage/forged-sig/wrong-counter/re-bound). 7/7 with Argon2id.
- docs: documentation/agent_doc/bot_check_altcha.md — protocol, threat model,
config knobs, standalone-script loading, and the vs-custom comparison.
…over the keyboard
Two coupled mobile bugs on the solo (one-item-per-screen) layout, both rooted in
the visualViewport keyboard handler:
- The handler did `activeElement.scrollIntoView({block:'center'})` on every
visualViewport resize AND scroll while the keyboard was open. block:center
targets the layout viewport (which iOS doesn't shrink), so it scrolled the
field toward the middle — i.e. behind the keyboard — and the scroll it caused
fired another visualViewport `scroll`, re-entering the handler in a loop. The
drifting offsetTop shrank the computed keyboard overlap, so the lifted Back/OK
nav crept back DOWN behind the keyboard. Net: an "ultra slow scroll that ends
with OK hidden behind the keyboard." Fix: drop the programmatic field scroll
entirely (CSS flex-start on kbd-open + the browser's native focus-scroll place
the field) and toggle the fit-lock only on the open/close transition, not every
tick. The nav lift is still recomputed each event so it tracks the keyboard.
- Tapping OK/Back from a focused field blurred it, which started closing the
keyboard and dropped the lifted nav out from under the finger mid-tap, so the
tap was lost and OK "didn't fire." Fix: preventDefault on the buttons'
pointerdown so they don't steal focus — the field stays focused, the keyboard
and nav stay put, the click still fires, and a text→text advance hands the
keyboard over without a flicker. seat() now blurs the field itself when the
next step has no text input, so the keyboard closes exactly when it should.
Verified on desktop: real OK clicks still validate + advance, text→text keeps
the keyboard, text→non-text closes it, change still fires. iOS keyboard behaviour
needs on-device confirmation.
The previous addon styling left a 0.4rem gap between the icon box and the field and rounded all four corners of each, so they read as two separate boxes. Make them one continuous pill instead: - Icon addons (email/tel/url/color/…): drop the gap; round only the OUTER corners (icon's left, field's right) at a shared radius; drop the icon's inner border and square the field's inner corner so the field's border is the single rectangular divider line. Handles leading, trailing, and both-sides addons via :first-child/:last-child/:has. Border width matched to the field's 1px. - Geopoint: same pill, but its trailing element is a real button (.geolocator fetches the location), so style it as a pushable blue action — flush against the readonly field, rounded only on its outer (right) edge, with an inner "lip" shadow that flattens on :active. The button's blue left border is the divider; the field rounds its left. Kept the full-pill look when the button is hidden (no geolocation) via :not(.hidden). Verified on the dev instance (email + geopoint screenshots, grey and invalid states).
…tor-scoped Record the gotcha surfaced while styling the icon addons: editing webroot/assets/form/** (the v2 form bundle, e.g. form.scss) cannot affect v1 because v1 never loads that bundle — v2 = FormRenderer + form_index.php + form.bundle.css; v1 = SpreadsheetRenderer + public/head.php's print_stylesheets() + assets/site/**. The .fmr-form-v2 scoping is belt-and-suspenders, not the isolation mechanism.
…olo) The reported blockers plus the deeper completion-counting issues that kept the all_widgets v2 survey from ever finishing in either layout. Reported items: - mc_heading: excluded from the v2 unanswered query + solo non-navigable types, so it's never a blank required step. - gods/kittens showif reveal: an item whose server-side showif is NA (dependency unanswered) is now rendered (hidden=null) instead of pruned, so Alpine reveals it instantly — no server round-trip (FormRenderer/SpreadsheetRenderer naShowifIsClientResolvable hook). Server-only-var showifs (random ran_group) still resolve server-side, so random assignment stays correct. - block recurs / can't finish: `block` + `blank` display items render but never count toward completion; and the client now reports its CURRENT showif visibility (_item_views[hidden] from data-fmr-hidden) so the server marks conditional/random-gated required items hidden (RunController:: markClientHiddenItems) — they stop blocking studyCompleted(). - empty optional file/image/audio/video: post '' so they get a saved row (File_Item tolerates the empty scalar). - select_or_add_multiple: opens its dropdown on focus (openOnFocus + focus the tom-select instance in solo). - year: validates instantly (4-digit pattern + min/max). - VAS: ~44px touch target (transparent finger-wide thumb, painted pencil mark). - solo submit item: renders as a big centred button; footer OK hidden. Also a real correctness fix in collectPayload: skip DISABLED inputs up front, so a checkbox/radio left checked before a showif hid it no longer posts stale state (this was surfacing as a spurious "required" error on a hidden conditional item). Tests: tests/e2e/v2-finish.spec.js (+ helpers/v2Complete.js, fixtures/upload.png) drives the whole all_widgets survey to completion in BOTH solo and non-solo layout (geopoint, select_or_add, give-100 block pair, altcha bot_check, empty optional files), toggling survey_studies.layout around each run. required-gating sweep updated to skip showif-conditional items (v2 renders showif=FALSE items Alpine-hidden rather than server-pruning them).
…nish - Bump per-test timeouts when RUNNING_ON_BS (a real-device walk pays per-step network latency): solo 720s, non-solo 300s. - On a client-blocked submit, report the visible required items left unanswered (no-arg evaluate, BS-bridge-safe) to make iOS-Safari failures diagnosable.
v1's client (survey.js showIf) defaults an unevaluable showif to HIDDEN (`_hide = true`), revealing an item only when its condition is true. v2's Alpine directive did the opposite — `result === undefined ? true` forced NA/unevaluable showifs VISIBLE — and the server rendered NA items without `.hidden`. So an item whose showif references a name the client can't resolve (a server-only run var like `ran_group`) or that hadn't evaluated yet flashed visible and, on slow iOS Safari, blocked the page submit before Alpine ran. Align with v1: - Server renders an NA client-resolvable showif item `.hidden` by default (Item::hideByDefaultPendingClient: CSS .hidden + disabled input, but $this->hidden stays null so the client still owns/reveals it). - Alpine x-showif: on an unevaluable expression (undefined), PRESERVE the server's render decision instead of forcing visible. The server already resolves server-only vars (ran_group) to shown/pruned, so preserving is correct; client-evaluable showifs (mc_polytheism == 2, the give-100 block) still react normally. Local v2-finish solo + non-solo stay green.
…e driver The non-solo driver detected page advancement via submitV2's waitForResponse, which under the BrowserStack service worker (serviceWorkers: 'allow') never fires — the form-page-submit POST is intercepted — so it falsely reported "blocked" even though the page had advanced. Detect advancement by the visible page number changing (or navigating away) instead, and settle long enough for the final-page redirect before declaring a failure. BS-aware submit timeout (60s) for the slow real-device OpenCPU round-trip. Both solo and non-solo now complete on iPhone 15 Pro Max / iOS 17 Safari, and on local Chromium. Also updates the CLAUDE.md form_v2 showif notes to the corrected behaviour: NA/unevaluable showif is hidden by default (v1 parity), not force-visible.
… form The external Altcha build (dist/external/altcha.min.js) we ship does not inject its own CSS at runtime (the default build does), and we only ever loaded the JS — so <altcha-widget> rendered with no border/padding/background: a bare checkbox + label. The branch's form.scss still carried the dead hand-rolled .fmr-botcheck-box rules, which target markup the widget no longer emits, so nothing themed the actual widget. - bot-check.js: import 'altcha/altcha.css' so webpack extracts Altcha's base stylesheet into form.bundle.css. Survives the production minifier (oklch / light-dark / @layer intact). - form.scss: drop the dead .fmr-botcheck-box rules; theme <altcha-widget> via its light-DOM classes — map --altcha-* tokens onto Bootstrap-5 theme vars + the old pill geometry, flatten .altcha-main into a single centred row with hover/:focus-within ring and a green verified tint. - doc: bot_check_altcha.md §5b records the external-build-ships-CSS-separately gotcha and the theming approach. Verified end-to-end (solve -> submit -> Stop) in dev and production builds.
…ill, cookie init Five participant-facing v2 fixes reported by the user: - Submit items render as the page's big centred button in BOTH layouts (the 2d40496 goal that was dead code). FormRenderer stripped submit items in processItems()+render(), so solo's .fmr-solo-bigsubmit JS/CSS had no target and non-solo drew a generic auto Next button. Now: keep submit items rendered; exclude them from the completion count (getAllUnansweredItems, like block/blank) AND the client required-gate (feedback.js) so they can't block themselves; suppress the duplicate auto page-nav in non-solo and tag the submit button data-fmr-next; centre + enlarge via form.scss. Verified e2e: solo + non-solo reach the Stop page. - VAS extremes unreachable: the thumb width was widened to $vas-hit (~44px), but a native range thumb insets value travel by thumbW/2 at each end. Narrow thumb width back to $vas-mark-w (the 44px tap target is the input height + full-width track, not the thumb width). Verified: track-end clicks yield 0/100. - rating_button: the generic .btn-group .btn skin clobbered the v1 analogue_rating_scale / square* / blank_button geometry from custom_item_classes.css (equal specificity, later source order). Re-assert v1 geometry scoped to the modifier classes; scope the solo padding bump out of the fixed-size variants. analogue bars match v1 (20x15, #ddd, no border). - "Saved" indicator: incremental autosave was silent (flushPersist never called setSaveState). Surface the existing pill on a real persist, gated on the server's {status:'saved'} so an empty-page noop doesn't lie. - request_cookie: v2 never initialised vanilla-cookieconsent and called a non-existent window.showPreferences. Import common/js/cookieconsent.js (as v1's participant bundle does) + the showPreferences module export. Full PWA-item implementation + e2e to follow.
… phone hand-off) Completes the v2 PWA item family (after the request_cookie init fix): - push_notification: registerFormSW() was gated only on syncUrl, which is empty when offline_mode='off' — so a push-enabled study with offline off never registered the service worker and navigator.serviceWorker.ready hung, silently breaking push. Register the SW when EITHER syncUrl (offline) OR window.vapidPublicKey (push) is present; only post the sync-url message when a syncUrl exists. (The rest of the v2 push pipeline — subscribe/save/test/ unsubscribe + declarative iOS payload — was already wired.) - add_to_home_screen: the install modal never rendered because main.js called the webpack-imported AddToHomeScreen() default binding, but the prebuilt add-to-homescreen package is an IIFE that exports nothing and only sets window.AddToHomeScreen (so the binding was undefined → TypeError, same as v1 fixed by calling the global). Call window.AddToHomeScreen with a guard. Also only create <pwa-install> when a real manifest link exists (otherwise it probes /manifest.json at the origin root → 404). Add 'prompted' to AddToHomeScreen_Item::validateInput's allowlist (the JS writes it). - request_phone: ported v1's QR hand-off into the v2 leaf module (qr-code-styling was a dep but unimported in the v2 bundle). Desktop now renders the QR + a copy-link control encoding the resumable run URL (run_url + ?code=), injected server-side as window.formr.runResumeUrl since v2 has no _formr_code input. A REQUIRED request_phone hard-gates on desktop (keeps the validity set so the desktop submit is blocked) — the participant must open it on their phone, where the resumed session's mobile UA-sniff auto-answers is_phone (the v1 mechanism, no endpoint). An optional one lets desktop proceed and records is_desktop. Values use the PHP allowlist (is_phone/is_desktop/qr_scanned). E2E suite (Playwright + BrowserStack) follows.
…t_phone QR) New tests/e2e/pwa-items-v2.spec.js — load-time assertions (v2 renders all pages into one document and the item inits run at load, so no survey walk is needed): - add_to_home_screen (regression tests for the install fix): window.AddToHomeScreen resolves to the global factory (the import binding was undefined), the install button renders, <pwa-install> uses the run manifest (no /manifest.json root 404), no "AddToHomeScreen init failed" TypeError, and a captured beforeinstallprompt is handed to <pwa-install>. Green on local-chromium. - push_notification: permission button + hidden result input present + VAPID key exposed (subscribe/SW are [BS-only] — local-chromium blocks service workers). - request_phone: desktop renders the QR svg + copy-link control, and window.formr.runResumeUrl encodes /<run>/?code=<participant> (resumable session). The request_cookie modal test is test.fixme: the import + run() config + the showPreferences export are all bundled and the singleton specifier matches, but cookieconsent does not initialise in the headless probe (#cc-main never created, no show--consent, click opens nothing, no thrown error). Needs interactive Chrome DevTools to resolve (Playwright MCP was down) — so the cookie init fix (856e926) is NOT yet confirmed end-to-end. SW registration + push subscribe/delivery + native install + real iOS push remain [BS-only] (npm run test:bs) — to be run on a real device.
- formSyncAction: auth gates now precede the uuid dedup query (no unauthenticated ledger oracle) and the offline-sync path applies the same markClientHiddenItems resolution as formPageSubmitAction (a queued page with client-hidden showif items could block completion) - survey_r_calls dedup key now includes item_id: identical expressions on two items previously shared one row whose item_id pointed at the last-imported item, silently breaking /form-render-page's page-scoped join for the other item (patch 059 + extractor + CLAUDE.md) - patch 064 rewritten: unit_session_id INT(10) UNSIGNED (was BIGINT, blocked the FK), cascade FK to survey_unit_sessions, IF [NOT] EXISTS guards so phantom-054 hosts re-run cleanly; collations added to 059/060 new tables - survey_r_call_results: bounded write-time eviction (>1 day, LIMIT 500) on the cache-write path — the table previously grew forever - patch 067 documented as a deliberately dormant column (letter-key feature removed end-to-end in 5c6c50b) - UnitSession: declare public $layout (patch 068 column) - FormRenderer: memoize fetchPageMap (was queried twice per render)
- history.pushState now writes the server page number (data-fmr-page), not the array index — on resumed sessions only unanswered pages render, so back/forward/reload landed on the wrong page; Previous also updates ?page= now - service worker no longer wipes the offline answer queue on pushsubscriptionchange: browsers fire it for benign rotations (Chrome does on updates), destroying unsynced diary answers while the re-subscribe handler explicitly treats rotation as benign. Queue wipes remain on the logout paths only; logout matching now also covers subdomain-routed deploys (bare /logout) - recorders: iOS video/mp4 recordings get .mp4 (not .webm) extensions; AudioContext closed after each duration decode (iOS caps ~4 live contexts); MediaRecorder ctor failure releases the stream; an active recording is finalized on page navigation (new fmr:pagechange event) - drainQueue single-flight guard (online + load + Background Sync can interleave); save pill no longer sticks on an unparseable 200 - r-call dedup key recorded only on success so a network failure stays re-triggerable; altcha loader retries after a transient script-load failure; deferred-fill writes no longer stamp answered timestamps - Alpine: unchecked scalar checkbox syncs the hidden partner's "0" (matches $_POST/R semantics; null made mycheck==0 diverge) - error banner built with textContent (was the only innerHTML error path); request-phone copy fallback link no longer stacks; cookie consent poll stops after 30 min; header_image_path h()-escaped - altcha checkmark svg gets pointer-events:none — it overlaid the checkbox input and intercepted its click point (worked for humans via wrapper bubbling, broke native a11y + strict automation) - suppress focus ring on programmatically-focused page headings
…scheme
- screenshots: the template listed 43 device-specific apple-splash
PNGs that were never shipped — every page with an add_to_home item
logged 21 404s and <pwa-install> showed broken images. Replaced
with the two real screenshot_{landscape,portrait}.png entries
- shortcuts: settings-/privacy-/terms-icon.png don't exist either;
mapped to shipped icons (message.png, security.png, icon.png)
- protocol_handlers: web+formr{APP_NAME} is an invalid scheme for any
run name containing digits/dashes (Chrome drops the entry with a
warning); the suffix is now squeezed to lowercase letters via the
new {APP_PROTOCOL_SUFFIX} placeholder
Cached run manifests must be regenerated to pick this up (admin
manifest action or Run::generateManifest); done for all runs on dev.
…ut pin, CHANGELOG - bot_check: click the .altcha-checkbox wrapper, not the inner input — altcha's checkmark svg intercepted the input's click point and strict actionability refused it. Unblocks bot-check-v2 (5/5) AND both v2-finish completion walks, which silently stalled at the bot_check step (the solve timeout was swallowed and each loop burned the 12s submit timeout until the test budget expired) - request_cookie fixme resolved: vanilla-cookieconsent defaults hideFromBots=true and its bot check includes navigator.webdriver, so run() silently no-ops in every automated browser — real participants verified working end-to-end interactively. The test now masks webdriver in an init script and the regression assertion is live - form-v2-spec pins the shared all_widgets study to layout=default in beforeAll (v2-finish deliberately leaves it on solo, which broke this suite on the next session depending on execution order) - v2-finish default-lane budget raised so a future item-gate failure surfaces driveDefault's stuck-diagnostics instead of an opaque timeout; stale global-setup comment corrected (e2e_* runs are public) - CHANGELOG Unreleased consolidated: schema section now lists the real patches 057-069 (was stale pre-rebase 47-52 and missed 063-069), late-branch features added (solo layout, bot_check/Altcha, VAS, recorders, soft-delete, study_iteration, required gating, e2e suite), and the bare-R-in-value bullet now matches shipped behavior (bare R is accepted and auto-allowlisted)
…, uncommitted)
Backend:
- Showif batch was all-or-nothing — one erroring expression (if(NA), object-not-found) failed the whole OpenCPU batch: participants saw raw "problem evaluating showifs" banners and every item turned .has-error. Each showif/value is now tryCatch-wrapped and value-gating uses isTRUE(as.logical(...)). Verified: the previously-broken all-required run loads clean.
- Randomized-split double-display — later-page showifs were never server-evaluated, so ran_group == 0 and == 1 branches both rendered. Visibility for all pages now joins the initial batch (values/labels stay page-scoped); verified one branch pruned, the other shown, per session. Also stopped the first-page merge from resurrecting batch-pruned items.
- Multi-select answers broke every R re-render — the answer overlay emitted c(1, 2) into a one-row data.frame ("replacement has 2 rows, data has 1", in the live logs) the moment a participant ticked two boxes; empty selections emitted column-deleting c(). Now serialized as the ", "-joined string the results table actually stores, NA when empty.
- note_iframe rendered as nothing when its knit fails (root cause on dev: rbokeh missing from the slim OpenCPU image). Participants now get a neutral placeholder; test/admin sessions get the OpenCPU debug banner.
- PWA items un-importable — add_to_home_screen required choices (and request_phone rejected them) though both only use choices as a label override, leaving empty husk studies behind. Added Item::$choicesOptional; re-imported the two husk fixture studies (now 5 items each) and deleted my probe study.
- iOS install icon: apple-touch-icon now falls back to the same default icon the manifest uses.
Frontend (form bundle):
- Install button dead on arrival (blocker) — Alpine's showif reveal re-enabled input/select/textarea but never <button>, so a showif-revealed add_to_home_screen/push_notification button kept its server-rendered disabled forever. Verified: reveal now enables it.
- Solo stranding on invisible steps — when the seated step gets hidden by a showif flip (the block-guard scenario from agent 1), the controller now re-seats on the nearest navigable step and refreshes nav/progress (new fmr:showif-toggled event). Verified live.
- Solo desktop nav — Back/OK now align with the 720px step column (measured 360/1080) instead of the viewport edges (20/1316).
- choose_two_weekdays cap — nothing enforced "two" anywhere, not even v1; a third check is now reverted with an inline message.
- Server-error feedback no longer calls reportValidity(), whose native focus could land on a 0×0 tom-select input and yank a long page's scroll to the wrong end.
- Plus a latent e2e flake: v2-finish leaves the shared fixture at solo while solo-layout's first test assumed default — that test now sets its own precondition (was the one red in the regression run; 15/15 after).
…om for master 057-059
ParsedownExtra can throw PHP Error (not Exception) when DOMDocument returns unexpected structure on malformed HTML input (e.g. a label containing a bare <head> tag). The existing catch block in SurveyStudy::addItems used `Exception`, which silently re-threw any `Error`, crashing the survey import with a fatal. Widen all seven call sites to `catch (\Throwable $e)` and fall back to the raw text on failure, logging via formr_log_exception. Sites fixed: - SurveyStudy::addItems – item label + choice label - Run::saveSettings – description, public_blurb, footer_text, privacy, tos - Email::save – email body - Pause::save – pause body
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.