Feature/custom r functions and secrets#693
Merged
rubenarslan merged 16 commits intoJun 11, 2026
Merged
Conversation
Adds a new custom_r_path column to survey_runs storing a file path to user-defined R functions. The file content is injected into every OpenCPU evaluation and knitr rendering call for the run, so custom functions are available in showif, value, feedback, relative_to, branch conditions, external URLs, email body, and all other R evaluation contexts. Key design decisions: - File-based storage (same pattern as custom_css / custom_js) - Instance-level caching on Run model (file read at most once per request) - No function signature changes — opencpu_* functions auto-inject via a small opencpu_custom_r() helper that retrieves from the active RunSession - No caller modifications needed in RunUnit subclasses or SpreadsheetRenderer Changes: - sql/patches/057_custom_r_path.sql: new patch - application/Model/Run.php: property, caching getter, saveSettings, export - templates/admin/run/settings.php: R Functions tab with Ace editor - application/Functions.php: opencpu_custom_r() helper + inject in all opencpu_evaluate and knitr functions - application/Api/V1/RunResource.php: API read/write exposure
…e error
The opencpu_evaluate() function sends R code via the /base/R/identity
endpoint which parses the parameter value as an R expression. Defining
functions inside the IIFE's (function() { ... })() body caused an
'Unparsable argument' error from OpenCPU. Moving function definitions
to the outer { } compound expression (outside the IIFE) resolves this.
The IIFE's child scope inherits from the outer scope, so custom
functions remain callable from user code inside the IIFE. Functions
that need run data should accept it as parameters, since injected
data variables (survey results, etc.) are defined inside the IIFE.
Also added documentation in the admin UI about the scoping behavior.
…ction definitions OpenCPU's /base/R/identity parses POST parameter values as R expressions. Complex multi-line expressions containing function() definitions caused an 'Unparsable argument' error from OpenCPU's R parser. Wrap custom R code in eval(parse(text = json_encode(...))) so the parser only sees a string literal. The code is evaluated at runtime inside the IIFE, defining functions in the same scope as injected data variables and user code. This also keeps custom functions in the same environment as (data frames defined by getRunData()), so functions can reference run data directly rather than requiring parameters.
Two-part fix for SQLSTATE[22001] 'Data too long for column result_log': 1. Schema: bump result_log from TEXT (65KB) to MEDIUMTEXT (16MB) - sql/patches/058_result_log_mediumtext.sql - Consistent with state_log (already LONGTEXT) in the same table - Eliminates the crash ceiling for error dumps containing long R code 2. Code: add truncation guards at all three write paths - UnitSession::end() before the UPDATE - UnitSession::logResult() before the UPDATE - EmailQueue::logResult() before the UPDATE - Truncates at 60000 bytes (safe margin below original TEXT limit) - Uses mb_strcut() to avoid splitting multi-byte UTF-8 characters - Belt-and-suspenders: keeps result_log reasonable regardless of column capacity
Design: Instead of adding new DB columns, tables, encryption, or files, leverage the existing custom R code injection mechanism. Users define secrets as R string literals with a secret_ prefix (e.g. secret_api_key <- "sk-...") directly in the R Functions editor. These are already injected via eval(parse(text = ...)) into every OpenCPU evaluation — zero new code on the participant hot path. Redaction: Known secret values (extracted via regex from the custom R file) are replaced with [SECRET REDACTED] at every output boundary where they could leak: - opencpu_log() — exception messages and traces - opencpu_debug() — R code display, error responses, console, stdout, session info - notify_user_error() — error bodies shown to admin/test sessions - Run::export() — custom_r in export JSON - RunResource::getRun() — custom_r in API responses Security trade-offs: - No encryption at rest — encryption defends against DB-only reads at complexity cost; the highest-probability threats (log/output leakage) are only covered by redaction regardless. - Minimum 6-char filter prevents false positives on short strings that appear commonly in output. - Values must be string literals (no expressions) — documented in the UI. - The study admin owns both the secrets and the R code, so the R evaluation context naturally has access to them (same trust model as .formr). Also: Renamed tab from "R Functions" to "R Functions & Secrets" and updated the help text and placeholder to document the secret_ convention.
…n UI - New survey_run_secrets table with FK cascade (patch 059) - RunSecret model: encrypt at rest via Crypto::encrypt(), decrypt on read - Admin UI: separate R Functions and Secrets tabs with auto-save AJAX - Secrets accessible in R as .formr$secret_<name> in all evaluation paths - Redaction sourced from DB (not custom_r parsing); removed 6-char minimum - Redact OpenCPU errors in result_log (DB) and study admin email notifications - Removed custom_r defense-in-depth regex (separate tab makes it unnecessary) - Export includes secret names only (values cannot leave instance encryption) - All 144 unit tests pass
…blic blurb; increase for custom CSS, JS, and R functions
… the API token injection
… usage guidelines
…dy error export
Adds a server-side R syntax check via OpenCPU's base::parse() that runs
every time the admin saves custom R code in the run settings. The check
is non-blocking — saving always succeeds even when the code is broken,
keeping formr a power-tool that trusts the user to save work-in-progress.
New endpoint:
- AdminAjaxController::ajaxValidateRCode() — accepts r_code POST param,
delegates to opencpu_validate_r_code(), returns {valid, message} JSON.
New helper:
- opencpu_validate_r_code(string ): array in Functions.php — calls
OpenCPU's /base/R/parse with json_encode()d code (so the POST value
arrives as an R string literal, not as raw R code), strips the
'R Error: ' prefix from parse failures, returns valid=true/false/null
for success/parse-error/server-down.
UI changes in the R Functions settings tab:
- Merged separate Save + Check R Code buttons into one always-clickable
'Save & Test R Syntax' button (no .save_settings class, so
run_settings.js leaves it alone — stays enabled regardless of dirty
state).
- Sequential flow: save the form to disk first, then validate syntax.
- Result rendered in a <pre> with white-space: pre-wrap so R's
multi-line error format (line number, source line, caret pointer) is
preserved exactly as printed.
- On parse failure, a 'Copy for LLM' button appears below the error
that copies a structured block:
TASK: Debug this R code syntax error.
CODE:
<user's code>
ERROR:
<full R error message>
- Progress indicator cycles Saving… → Checking syntax… → result.
- Results display above the editor (always in view, not scrolled off).
Fixes to the secrets management UI (incidental):
- Enter key in secret-name or secret-value fields triggers Add.
- Focus returns to the name field after adding a secret.
…e debugger links
Correctness fixes (verified against live OpenCPU):
- json_encode for R string literals now uses JSON_UNESCAPED_SLASHES |
JSON_UNESCAPED_UNICODE: PHP's default \/ escape and \uXXXX surrogate
pairs are invalid R escapes, so any custom R containing a '/' (division,
URLs in comments) or emoji broke every evaluation / failed validation.
- Custom R + secrets injection unified in opencpu_run_prelude(): runs
AFTER library(formr) in all six contexts (opencpu_evaluate previously
ran it before), always wrapped in eval(parse(text=...)) so injected
code cannot terminate knitr chunks.
- Secret references inside custom R functions now trigger injection
(previously only the unit's own code was scanned).
- Secret values escape newlines (\n\r) so multi-line values can't break
Rmd chunks; redaction implements the documented 6-char minimum.
- result_log truncation (truncate_result_log helper) now matches the
MEDIUMTEXT limit from patch 058 instead of contradicting it at 60 KB.
- Restored clobbered opencpu_substitute_parsed_strings docblock.
Security hardening:
- Secrets are write-only in the admin UI: stored values are never
rendered into the page; replace-or-delete only (null in secrets_json
= keep). Import placeholders survive saves as empty-string secrets.
- Secret names validated server-side ([A-Za-z0-9_]{1,190}) and client-
side; names-only fetch (no decryption) for settings page and export.
- Secret rows built with DOM APIs; the old innerHTML path silently
stripped <>&"' from secret values, corrupting them.
New: "Open in R Fiddle" links in the OpenCPU debugger (R Markdown /
R Code panels). Secret-redacted code travels base64url-encoded in the
URL fragment (never sent to the fiddle host); configurable/disableable
via $settings['r_fiddle_url'].
Plus: CHANGELOG entry; unit tests for injection gating, escaping,
redaction, name validation, truncation, and fiddle URLs.
Browser-testing the debugger caught a leak the unit tests missed: the
R source shown by opencpu_debug (and embedded in the R Fiddle link)
contains the secret in its R single-quoted escaped form (' -> \'), so
the literal str_replace on the raw value sailed past it. Redact the
addcslashes form written by opencpu_inject_secrets and the
JSON-encoded form (quoted request payloads in error output) alongside
the raw value.
Adds tests/e2e/run-secrets-custom-r.spec.js (8 specs) covering the scenarios verified manually for PR rubenarslan#693: invalid/valid custom R validation, client-side secret-name validation, write-only secret rendering, untouched-field no-save, end-to-end custom-function + secret injection into a participant render, and — the centerpiece — a leak sweep over the OpenCPU debugger and the R Fiddle link payload. Writing the sweep caught a second leak the unit tests missed: the debugger's Request panel renders OpenCPU_Request::__toString(), which addslashes() the already R-escaped secret assignment (" -> \"), a form the addcslashes-only redaction variant didn't match. opencpu_redact_secrets now closes over the escaper set (addcslashes / addslashes / json / rawurlencode) composed to depth 2 and redacts longest-match-first. Unit tests extended with the composed-escaping cases. New tests/e2e/helpers/admin.js: admin login + cached storageState, run create/delete, and an Ace-aware setAceValue helper.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Run-level custom R functions, secrets, and syntax validation
Adds the ability to define custom R functions and encrypted secrets at the run level, injected into every R evaluation context. Includes an inline R syntax validator with feedback and LLM-ready error export.
Features
custom_r): Define named R functions and globals in a dedicated settings tab. Injected before every R evaluation — stored in.formrpackage env.-> Possible ToDo: Add "smart" injection only when defined function is requested
secret_naming convention and auto-redaction from result logs. Selective injection via.formr$to R code.result_logoverflow from long custom R code (switched toMEDIUMTEXT).Fixes
Schema changes
sql/patches/057_custom_r_path.sql— custom R functions storagesql/patches/058_result_log_mediumtext.sql—result_logcolumn type changesql/patches/059_create_run_secrets.sql— secrets table