Formula agent AI slop attempt - #10
Open
markusenglund wants to merge 10 commits into
Open
Conversation
This introduces a dedicated formula-relationship AI use case with repository storage and migration support, and updates check-formula-relationship to detect formula columns and print relationships with column-name context for clearer output. Made-with: Cursor
For each AI-identified row-level relationship, evaluate the Python expression over every data row in a persistent python3 subprocess and report how many rows hold within rounding tolerance. Tolerance is computed via interval arithmetic: each numeric cell is treated as [value ± 0.5 * 10^-decimals] based on its stored precision, and the Python worker enumerates corner evaluations to produce an expected range that is compared for overlap with the observed range. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Python worker is now a generic sandboxed evaluator: compile expression once, eval at each provided point scope, return a numeric value or error. No domain knowledge, no itertools, no random sampling. TypeScript now owns extractOperands (regex on uppercase identifiers), generateCorners (Cartesian product ≤6 operands, random sampling for more), and min/max aggregation. checkRelationship sends all corners for all rows as a flat batch to Python in a single round trip. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds an explanation field to the AI response so the reasoning is always logged, even when no relationships are found. Also instructs the AI to report at most one formula per group of related columns to avoid redundant algebraic rearrangements. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restructures the prompt using markdown sections (matching the reviewResults pattern), adds article title/abstract/data description for Dryad datasets and fileCaption/fullText for PMC datasets. Also extends the command to accept both Dryad (numeric) and PMC (PMCxxxxxxx) dataset IDs, and removes redundant prompt instructions the model doesn't need. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The model was using sample values to verify formulas and rejecting real relationships when they didn't hold exactly (e.g. SLA = LA / LM fails on aggregated means). Column name and scientific context are sufficient for identification; the Python runner handles verification. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Each sheet in checkFormulaRelationship triggers a separate Gemini call, so the 5x6 copy-paste limits were wasteful. Split the constants into copy-paste (5x6) and formula (2x3) variants and thread an optional sheet limit into the two loaders the formula command uses. Also filter out empty-header columns when listing columns for the AI formula prompt and in the column log line. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Adds `test-formula-prompt` CLI that runs the formula agent against hand-crafted cases and checks both that expected relationships are found (across a list of acceptable algebraic forms) and that no unexpected extras are reported. First case is extId 26240. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Trim defensive checks for cases that can't actually arise, simplify countDecimals (drop scientific-notation handling), rename misnamed files (cornerGeneration→extractOperands, intervalCheck→countDecimals), and extract the shared identify→check→revisit loop used by both the CLI command and the test harness into runRelationshipLoop. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.