Product principles and design decision framework

[sacchen]

This issue is the living strategy document for phil.

Purpose:

• Keep big-picture decisions coherent across roadmap and implementation.
• Preserve reasoning history (including dead ends) without treating early thoughts as final.

Status Legend

• Hypothesis: idea under exploration; not a commitment.
• Leaning: current preference; still open to change.
• Provisional Decision: active default policy pending more evidence.
• Accepted: stable policy for current milestones.
• Superseded: kept for history, no longer active.

Current Position (Not Final)

• Provisional Decision: phil is terminal-first, exact-first, recovery-first, and script-safe.
• Provisional Decision: reliability regressions block release.
• Leaning: one-theme-per-release reduces scope drift and improves user clarity.
• Leaning: use SymPy-native primitives where possible; keep custom code for safety + UX.
• Hypothesis: decimal exactness policy may need a calculator-mode default or stronger artifact hinting.

Decision Rules (Apply to Roadmap + PRs)

For any change, answer:

1. Does it improve correctness, time-to-answer, or time-to-recovery on real workflows?
2. Does it preserve one-shot/REPL parity and script output contracts?
3. Is failure behavior explicit and tested (E: + actionable hint:)?
4. Can it ship as a small coherent slice?
5. If reliability regresses, is release blocked?

If any answer is "no", de-scope, split, or defer.

Open Questions (Active)

1) Performance and Recovery Policy

Open:

• Explicit thresholds for normal response, guarded failure response, and startup budget.
• Fail-fast vs continue-symbolic boundaries.

Evidence needed:

• benchmark timings on fixed workflow set,
• REPL recovery latency checks on pathological inputs.

2) Simplification Policy

Open:

• broad simplify() boundaries vs targeted transform defaults.
• structure-preservation expectations.

Evidence needed:

• behavior diffs on representative expressions,
• user-facing predictability review.

3) Ambiguity Policy

Open:

• accept/reject/hint table for shorthand ambiguity (sin x^2, 1/2x, etc.).

Evidence needed:

• parser behavior matrix + tests,
• error-recovery quality checks.

4) Exactness / Numeric Policy

Open:

• mixed decimal behavior policy (artifact handling, exactness defaults, hinting).

Evidence needed:

• explicit decision on decimal parsing/rationalization strategy,
• regression tests for decimal artifact class.

5) Release Scope Discipline

Open:

• criteria to move items from Later -> Next.

Evidence needed:

• impact estimate + reliability risk + testability.

Known Bug Class To Resolve

• Provisional Decision: treat float-artifact cancellation as a product-level exactness decision, not a one-off bug.

Example:

phil> 1/(0.1 + 0.2 - 0.3)
1.80143985094820e+16

Competitor Win Conditions (Working)

• vs WolframAlpha: faster terminal-first completion for common workflows.
• vs qalc: clearer symbolic/exact recovery after mistakes.
• vs Python/SymPy REPL: fewer keystrokes and less setup for core tasks.

Status: Hypothesis until benchmarked.

Evidence Loop (Lightweight)

Per milestone:

1. Run fixed workflow benchmark set (core + pathological).
2. Gather small real-user observation sample.
3. Track minimal metrics:
   • time to first correct answer,
   • error -> success recovery time,
   • post-error abandonment/tool-switch,
   • repeat usage signal.
4. Use metrics as release gate input.

Update Protocol (for Archivability)

• New exploration should be added as comments first.
• Once stable enough, summarize in this body with a status label.
• Do not delete old reasoning; mark superseded items as Superseded.
• Keep this top section concise; keep detailed exploration in linked comment archive.

Exploration Archive (Non-Binding)

These comments capture useful thought process and references but are not final decisions by themselves:

• Decimal artifact case and questions: Product principles and design decision framework #9 (comment)
• ExactCalculator/constructive-real takeaways: Product principles and design decision framework #9 (comment)
• Reading-to-decision map + priority reads: Product principles and design decision framework #9 (comment)
• Lightweight decision-process note: Product principles and design decision framework #9 (comment)
• Branch->PR process note: Product principles and design decision framework #9 (comment)

Related Issues

• Umbrella: prioritize math branch roadmap across feature tracks #6 Umbrella: prioritize roadmap across tracks
• Prefer SymPy built-ins over custom math plumbing #8 SymPy built-ins consolidation
• Add exact-first stats/probability helper surface #4 Stats/probability track
• Define CS/applied-math feature track (excluding #1/#4) #3 CS/applied-math track

[sacchen]

Concrete bug case to track under exactness/recovery policy:

phil> 1/(0.1 + 0.2 - 0.3)
1.80143985094820e+16

Why this matters:

• This violates user expectation for calculator-style exactness and can look like a correctness bug.
• It should be treated as a product decision case for mixed float/exact semantics, not just a documentation note.

Questions this should drive:

1. Should decimal literals default to exact rationals in core calculator mode?
2. If not, what explicit hinting/rationalization policy should apply on near-zero cancellation?
3. Do we want an opt-in/out mode for exact decimal parsing?

Suggested acceptance target for this class of input:

• Deterministic behavior + explicit hinting when binary-float cancellation artifacts are likely.

[sacchen]

Learnings from Android/ExactCalculator + constructive-real references that are directly relevant to phil:

Sources:

• ExactCalculator AOSP change thread/diff context: https://android.googlesource.com/platform/packages/apps/ExactCalculator/%2B/4d44fcd061329d121965bee229d3fca5d282e5c1%5E2..4d44fcd061329d121965bee229d3fca5d282e5c1/
• Constructive reals/API paper context: https://research.google/pubs/towards-an-api-for-the-real-numbers/
• Hans Boehm CR calculator notes: https://www.hboehm.info/crcalc/

What we should carry into phil:

1. Avoid long uninterruptible work.

• Reliability is largely about keeping recovery immediate.
• Reinforces our guardrail-first stance for giant powers/towers/factorials.

2. Prefer goal-directed evaluation over eager materialization.

• Compute only as far as needed to answer the user query.
• In phil terms: symbolic-first simplification and cancellation before heavy numeric expansion.

3. Preserve exactness where cheap and useful.

• ExactCalculator tries exact fractions when easy.
• Relevant to our decimal artifact class (1/(0.1 + 0.2 - 0.3)): decide explicit policy for decimal exactness/rationalization.

4. Make approximation/uncertainty explicit.

• Avoid UX where users cannot tell exact from approximate behavior.
• In phil: hinting/output policy should make exact-vs-float paths visible when it matters.

5. Specialize for performance where semantics remain clear.

• Practical targeted fast paths can beat elegant but slow general methods.
• Apply selectively with tests and parity guarantees.

Suggested integration:

• Treat these as supporting rationale for Product principles and design decision framework #9 decisions on performance thresholds, exactness policy, and fast-recovery gates.

[sacchen]

Reading-to-decision map for phil (what each source helps us decide), plus priority order.

Most Relevant to Read First (highest leverage)

1. SymPy Best Practices

• https://docs.sympy.org/dev/explanation/best-practices.html
• Why first: directly informs our simplification policy and reduces custom-math regressions.

2. SymPy Parsing docs

• https://docs.sympy.org/latest/modules/parsing.html
• Why first: impacts ambiguity policy, exactness handling (rationalize), and parser behavior consistency.

3. SymPy Assumptions + refine/ask

• https://docs.sympy.org/latest/guides/assumptions.html
• https://docs.sympy.org/latest/modules/assumptions/refine.html
• https://docs.sympy.org/latest/modules/assumptions/ask.html
• Why first: critical for correctness on domain-sensitive expressions and “no silent semantic drift.”

4. Python floating-point tutorial

• https://docs.python.org/3.13/tutorial/floatingpoint.html
• Why first: frames policy decisions for decimal artifact cases like 1/(0.1 + 0.2 - 0.3).

5. ExactCalculator/constructive-real references

• https://android.googlesource.com/platform/packages/apps/ExactCalculator/%2B/4d44fcd061329d121965bee229d3fca5d282e5c1%5E2..4d44fcd061329d121965bee229d3fca5d282e5c1/
• https://research.google/pubs/towards-an-api-for-the-real-numbers/
• https://www.hboehm.info/crcalc/
• Why first: reinforces fast-recovery/interruptibility and goal-directed evaluation strategy.

Source -> Decision Mapping

A) Performance/recovery policy (fast answer + fast failure)

• ExactCalculator + constructive real refs (above)
• pyperf (benchmark method): https://pypi.org/project/pyperf/

B) Simplification policy (broad simplify vs targeted transforms)

• SymPy Best Practices
• SymPy simplify docs: https://docs.sympy.org/latest/modules/simplify/simplify.html

C) Ambiguity policy (accept/reject/hint)

• SymPy Parsing docs
• SymPy parsing transformations in practice (convert_equals_signs, rationalize, etc.)

D) Exactness/numeric policy (mixed decimal behavior)

• Python floating point tutorial
• Python decimal module: https://docs.python.org/3/library/decimal.html
• SymPy parsing + rationalization behavior

E) Stats/probability implementation strategy

• SymPy stats: https://docs.sympy.org/latest/modules/stats.html

F) Competitive UX baselines

• qalc manual: https://qalculate.github.io/manual/qalc.html
• Wolfram|Alpha API overview: https://products.wolframalpha.com/api

Practical reading sequence (suggested)

Week 1 (decision-critical):

1. SymPy Best Practices
2. SymPy Parsing docs
3. SymPy Assumptions/refine/ask
4. Python floating-point tutorial
5. ExactCalculator diff notes

Week 2 (execution + benchmarking):

1. SymPy simplify details
2. pyperf docs
3. SymPy stats module
4. qalc / WA references for competitor benchmark framing

Goal: by end of Week 1, lock policies for simplification, ambiguity, and decimal exactness in #9.

[sacchen]

Keep-in-mind note (tentative until we validate relevance):

Decision approach we can use:

1. Start from phil principles first (correctness, fast answer, fast recovery, stable CLI contract).
2. Use primary docs to narrow options (SymPy/Python docs) before custom design.
3. Use competitor patterns as reference, not authority.
4. Record decisions explicitly (context, options, choice, tradeoffs, tests).
5. Validate with evidence loop (benchmarks + real usage observations).
6. Gate releases on decision adherence (block if reliability/parity contracts regress).

If this feels too heavyweight, we can apply only to high-impact decisions (simplification, ambiguity, exactness, recovery policy) and keep everything else lightweight.

[sacchen]

Process proposal to keep scope and traceability tight (inspired by perfect-commit discipline):

Prefer branch -> PR -> main for user-visible or multi-file changes, with explicit issue linking (Fixes #... / Refs #...).

Reason: it keeps each change coherent, makes review/rollback safer, and preserves a clear map from roadmap issue -> implementation -> release note.

I also added this as a recommended workflow in CONTRIBUTOR.md.

[sacchen]

Comment index / changelog for #9 (for readability + archivability)

Purpose:

• Keep all exploratory comments as historical record.
• Provide a fast map of what is still active context vs what is now summarized in the issue body.

Current source of truth

• Issue body is the canonical summary (status labels, open questions, update protocol).

Comment status map

1. Decimal artifact case and questions

• Product principles and design decision framework #9 (comment)
• Status: Active context (tracked in body: "Known Bug Class To Resolve").

2. ExactCalculator/constructive-real learnings

• Product principles and design decision framework #9 (comment)
• Status: Active context (summarized in body + supports performance/exactness questions).

3. Reading-to-decision map + priority reads

• Product principles and design decision framework #9 (comment)
• Status: Active context (reference list; non-binding).

4. Lightweight decision-process note

• Product principles and design decision framework #9 (comment)
• Status: Superseded by issue body sections:
  • "Decision Rules"
  • "Evidence Loop"
  • "Update Protocol"

5. Branch->PR process note

• Product principles and design decision framework #9 (comment)
• Status: Active process note; implemented in CONTRIBUTOR.md.

Maintenance rule for future comments

• New exploration: add as comment first.
• Once stable: fold into issue body with status label.
• If superseded: reply "superseded by body" (do not delete).

[sacchen]

Consolidated archive note (v2) for readability.

This comment is the single archive pointer for prior exploration.

Active references

• Canonical strategy summary: issue body
• Active bug class reference: decimal artifact example in issue body

Archived explorations (now folded into body)

• ExactCalculator/constructive-real learnings
• Reading-to-decision source map
• Lightweight decision-process note
• Branch -> PR process note (implemented in CONTRIBUTOR.md)
• Prior comment-index/changelog note

Archival policy

• Older exploratory comments are minimized as outdated (not deleted).
• New exploration should go in comments first, then be summarized into body with status labels.