Skip to content

Latest commit

 

History

History
406 lines (363 loc) · 30.6 KB

output-contract.md

File metadata and controls

406 lines (363 loc) · 30.6 KB

PCResolve Output Contract

Ownership uses the stable schema_version="1.0" provenance contract and one lexical scope model. Experimental value flow uses the separate flow-0.2 contract and does not change ownership output.

CLI

pcresolve project                  # human summary
pcresolve project --json           # full provenance JSON
pcresolve project --json-summary   # compact summary JSON (CI)
pcresolve a.py                     # one file, human summary
pcresolve a.py --json              # one file, full provenance JSON
pcresolve a.py --value-flow --entry a:function
pcresolve project --explain-library numpy
pcresolve project --explain-symbol x
pcresolve project --explain-call "np.array"
pcresolve project --strict
pcresolve project --usage-summary --top 20
printf '/path/to/project\n' | pcresolve --stdin --json-summary
  • Lexical scope analysis is the only supported scope semantics.
  • --json is the primary machine-consumption format.
  • --json-summary is the recommended CI format.
  • Long options do not accept abbreviations. A partial option such as --js or --json-f exits 2 as unrecognized.
  • --strict exits non-zero when ownership error diagnostics are present. In text, debug, and explain modes, the diagnostics causing that exit are shown even without --verbose. JSON modes retain diagnostics in their payload.
  • Choose one output mode: full JSON (--json), --json-summary, --debug-dump, or one --explain-* option. Conflicting modes exit 2 with an argument error.
  • JSON output rejects --verbose, --usage-summary, and --quiet. Explain output rejects --usage-summary and --quiet; --debug-dump rejects --quiet. These combinations exit 2 instead of ignoring options.
  • Text diagnostics are printed once. --verbose also includes diagnostics in debug/explain output and reports stats.skipped_modules when diagnostics are present. --quiet suppresses the summary and filters diagnostics to errors, including when combined with --verbose. --usage-summary can include library usage in quiet mode.
  • --top N requires a non-negative integer (default 20, 0 means unlimited). It limits libraries in the text and usage summaries, calls and symbols in explain output, and per-library files lists in summary JSON. Total counts remain unchanged. Full JSON, debug facts, diagnostics, import lists, and per-file statistics in explain-library output are not truncated. --top is rejected with full JSON and with a debug dump that has no appended --usage-summary, rather than being silently ignored.
  • Explain queries must contain a non-whitespace name; empty queries exit 2.
  • Explain queries with no matches complete successfully and report that result on stdout consistently for libraries, symbols, and calls.
  • --explain-call NAME matches func_name or resolved_func by exact name or a dotted path suffix at a name boundary. Matching is case-sensitive and excludes call arguments and string contents. Series matches Series, pd.Series, and aliases resolved to pandas.core.series.Series; it does not match ABCSeries, SeriesExtra, or Series.to_numpy. Qualified queries such as np.array, pandas.core.series.Series, and core.series.Series select matching callable paths. The reported count includes all matches before the --top display limit is applied.
  • Ownership and value-flow positional inputs accept a project directory or one existing .py/.pyi file, using relative or absolute paths. File mode reads only that source; it does not discover or follow sibling sources. Use directory mode for cross-file analysis. Value flow also accepts repeated --source-file options when an explicit multi-file source set is required.
  • Missing paths, missing positional ownership input, empty stdin, and unsupported file inputs exit 2 as argument errors rather than producing a successful empty analysis.
  • In file mode, the path-normalization root is the file's parent directory, or the directory above its outermost enclosing regular Python package (identified by __init__.py or __init__.pyi). This retains package module names, including for a selected __init__.py, without adding other sources. Ownership JSON keeps schema_version="1.0" and the same fields.
  • --stdin reads one directory or source-file input path from standard input. It is an alternative to the positional path; using both is an error. In value-flow mode it also conflicts with --source-file, so the selected input source never depends on whether stdin happens to be empty.
  • --max-seconds, --max-work and --max-memory-mb optionally limit one ownership or value-flow analysis. Values must be positive and finite. An exceeded limit exits 2 with an error on stderr and emits no result JSON. Checks are cooperative; output serialization is outside the limit. These options do not change semantic proof budgets or output schemas.

File and directory inputs use the same ownership semantics. Given the same analyzed source set and module mapping root, their call classifications are the same. Missing module implementations limit both input forms: for example, from factory import create_app; create_app() retains the import-backed owner factory when no additional project evidence is available. The import path alone does not prove that the module is project-local or identify a different implementation owner.

Python API

The stable entry points use these signatures:

analyze_project(project_root, analysis_limits=None, reuse_source_facts=False)
analyze_source(source, file_path="<string>")
ProjectAnalyzer(project_root, analysis_limits=None, reuse_source_facts=False)
SingleFileAnalyzer(module_name=None, is_package=False, file_path="")

analyze_project and ProjectAnalyzer also accept a .py/.pyi file path, with the same source selection and path-normalization root as the ownership CLI. They return the existing ProjectAnalysis result for that single file.

analyze_source accepts Python source text. Its file_path parameter supplies location metadata; it does not select or read a file from disk.

PCResolve uses one lexical scope model. There is no public scope-model selector or stats.scope_model field.

AnalysisLimits(max_seconds=None, max_work=None, max_memory_mb=None) applies optional resource limits; exhaustion raises AnalysisLimitExceeded. No partial ProjectAnalysis or FlowAnalysis is returned. reuse_source_facts=True on an ownership analyzer retains independent copies of pristine visitor facts for unchanged files across repeated runs. It is disabled by default and does not reuse completed cross-file semantic results. See Architecture for cache invalidation and resource-check details.

Experimental value-flow contract

Value-flow analysis is an additive, experimental API. It does not change the ownership entry points or the ownership JSON schema below:

FlowAnalyzer(
    source_files=None,
    import_roots=None,
    project_root=None,
    return_summaries=None,
    parameter_shapes=None,
)
FlowAnalyzer.analyze(entry, max_depth=1, max_functions=500,
                     max_call_contexts=2000)
FlowAnalyzer.expand(result, call_id, additional_depth=1)

Exactly one of project_root and source_files is required. entry is a FunctionRef; its module and dotted qualname identify a function, nested function, or method. File path and definition line can disambiguate duplicate definitions in the Python API.

FlowAnalysis.to_dict() emits schema_version="flow-0.2" with entry, inputs, functions, calls, and boundaries. find_calls() and describe_call_flow() select exact call-site records; trace_parameter() composes only the summaries already present in the snapshot. Empty flow lists or status="unknown" are not no-flow proofs when boundaries remain.

The PCBench-driven extensions remain additive within flow-0.2:

Callable identity guards can add callable_type and callable_evidence to existing parameter/capture roots, with the equality branch in conditions. They describe a stable plain source function under that condition, not a runtime identity assertion or an alternative inferred from inequality. Individually exact ordinary callable/immutable scalar facts through a dynamic expansion can retain a normal_call_binding condition. This condition records the call ID, parameter and explicit argument without upgrading binding status or claiming that duplicate-key exception paths enter the callee body.

Located definitions inside conditional statement blocks can be analyzed while retaining conditional_definition_unconfirmed boundaries. Their conditions record activation witnesses; no located definition is a runtime-existence proof. Complete bounded mapping effects can include direct same-module helper calls; their normal-return status does not assert exception-path completion.

Call-time capture_bindings retain raw sources and can additionally expose current_sources and exact container_binding shape/identity, unknown-content status, evidence and conditions. An explicitly expanded open captured container retains symbolic capture roots; finite, conflicting or effect-unknown contents do not authorize an open container merely from its lexical name.

Location Additive field Meaning
calls[*] target_candidates Bounded source targets; a single target remains the selected static candidate when available.
calls[*] implicit_protocol Empty for ordinary calls. Synchronous with records method="__enter__" or "__exit__", lookup="type_mro", context expression/with ID/item index, source definition/hashes, assumptions and issues. IDs use :with_enter:<index> or :with_exit:<index>:<cleanup-order>. Exit facts add incoming_exit (kind/known exception/origin) and witness cleanup_order; the source-sorted call list is not execution order. These are not syntactic ownership calls; runtime_target_confirmed remains false.
functions[*] context_managers Optional synchronous with facts: saved manager sources, as_target, enter call/target/return/effect status, source hashes and exit target/effect/completion status. exit.executions links cleanup call IDs, witness order, incoming exit, status/issues. Complete bounded exits have source_modeled or source_noop effects; unproved exits remain unmodeled/unconfirmed. Entry/exit proofs are conditional on normal operations and snapshot assumptions, not runtime completion guarantees.
instance value-source provenance context_manager_return Enter-result endpoint with identity="receiver_alias" or "return_instance", enter_call_id, opaque caller-instantiated object_source and return evidence. Source dependence alone does not prove alias identity. Unknown returns must not inherit the manager type.
value-source conditions[*] context_enter Enter-call ID with completion="normal_return" and snapshot basis. Conditions describe possible normal-entry paths, not confirmed execution or completed cleanup.
value-source/effect conditions[*] context_exit Exit-call ID with incoming exit kind/exception. Links a cleanup path witness, not a guaranteed completed operation. Effect completion="normal_return" applies only to that operation's successful path.
exit argument-source provenance context_exception Symbolic exception triple components type, value, traceback with known exception spelling when available. Non-exception exits bind three literal None arguments; no runtime exception objects or traceback are synthesized.
exit implicit_protocol exit_outcomes, pending_return_sources Possible outgoing exit kinds/known exception spellings and raw selected pending-return references. The latter preserve object selection before cleanup, not final contents; mutated return contents are materialized after cleanup. completion remains unconfirmed when exit or result-truth behavior is unproved.
calls[*] receiver_type_evidence For a constructor-backed field returned by a local accessor, records the field, nominal instance type, assignment, all proven return statements, accessor call, and downstream receiver call. These are source candidates, not runtime dispatch guarantees.
calls[*] callable_instance_evidence For a uniquely assigned module-level constructor-backed callable object, records assignment, class, constructor sources and invocation. Optional binding gives conditional source_bound/unconfirmed status, exact target/definition, implicit receiver, finite constructor state, issues, assumptions and source/snapshot hashes. The target remains a source candidate, not a runtime dispatch guarantee.
calls[*] decorator_identity_evidence For a source-proven identity-return decorator, records its definition, unchanged input parameter, and normal return statements. This proves returned object identity; unmodeled decorator effects remain a boundary.
import-source provenance import_incomplete, import_modified Optional flags for a merge containing a missing binding, or a visible attribute write to an imported object. A true flag prevents unique-target resolution from that binding.
calls[*] binding_status, binding_issues Complete/uncertain/invalid binding state and statically proven missing, duplicate, unresolved, or dynamic-expansion facts.
calls[*].parameter_bindings[*] destination_kind parameter, var_positional, var_keyword, or unresolved; variadic element positions remain in target_path.
calls[*] result_sources Supported local container results, source-proven class-valued returns, and nominal constructor/allocation results, with evidence and conditions. This does not substitute arbitrary parameter types or prove runtime dispatch.
value-source provenance class_type, instance_type Optional serialized FunctionRef for a source class value or nominal instance. Class-valued returns retain kind="class", module, and source; receiver parameters keep their parameter roots.
value-source provenance container_object Source-proven ordinary-argument alias returned by a narrow local helper. Contains the opaque caller-local object source, builtin container_shape (dict, list, tuple), identity="argument_alias", returned parameter, target function, and optional parameter_container. Evidence remains on the value source.
builtin container result container_object.identity, copied_from shallow_copy denotes a fresh top-level dict/list/tuple snapshot of one proven input object, recorded by copied_from; tuple-of-tuple uses argument_alias, not a false fresh identity. element_alias identifies a proven container element returned by setdefault. Nested mutable objects remain shared. These caller-local facts are not general helper-return or runtime-type proofs.
calls[*].effects[*] kind="container_copy", input_container, output_container A bounded builtin copy/conversion reads current contents and key/position markers on normal completion. It never restores earlier dependencies after clear/pop. Output shape and identity remain separate from parameter origins; builtin protocols have no fabricated source target.
mapping setdefault effect state_before, insertion, object_source, completion Constant-key lookup is present, absent or unknown; default insertion is never, always or conditional. Result and inserted dependencies select existing/default values accordingly. completion="normal_return" is not a guarantee of successful execution.
value-source conditions[*] mapping_membership An implicit setdefault branch records object_source, element_path and state="present" or "absent", with call evidence. It is a bounded key-presence condition, not an evaluated predicate or unconditional default write.
value-source provenance object_only True on a returned-container call endpoint: it records object identity, not an element dependency. Consumers must not substitute this endpoint through the helper's old input contents; current element roots are materialized separately.
value-source provenance value_incomplete Optional true flag for a missing/unknown alternative or visible receiver attribute mutation. Prevents unique class/instance/container-object inference while retaining dependency roots.
functions[*] return_objects Optional symbolic ordinary-parameter alias proof (parameter, identity, assignment/return evidence), without caller-local object IDs. This is separate from element returns.
functions[*] mapping_effects Element-specific membership, pop, delete, update, and merge facts with bounded state and branch conditions.
instantiated helper mapping effects completion, may_raise, binding, call_id, call_evidence A complete straight-line constant-string-key dict removal instantiated from a proven ordinary argument. completion="normal_return" excludes exceptional completion; may_raise="KeyError" applies to required-key pop/delete. Binding and both source sites explain which actual object was modified.
functions[*].return_objects[*] requires_shapes, completion Optional receiver-parameter shape constraints and normal-completion scope for a symbolic alias proof that includes fully modeled effects. No caller-local object IDs are stored in the symbolic summary.
functions[*] receiver_contexts Source class receiver contexts used for classmethod/__new__ summaries. Multiple contexts merge conservatively into one source-call record; conflicting targets remain bounded alternatives.
functions[*] decorator_effects Bounded source-decoration facts on the exact original FunctionRef, including entries. Separates identity_status from body_binding_status (preserved, modified, unknown); records expression/target, optional constructor/receiver facts, input-function attribute effects, conditions, source evidence/hashes, assumptions and unresolved boundaries. runtime_target_confirmed remains false; a same-object return alone is not body/binding preservation.
calls[*] callable_sources Callable expression provenance, including an optional source-proven callable_type and callable_evidence on ordinary parameter roots. String values or annotations do not establish callable identity.
calls[*] class_method_bindings Source-snapshot classmethod lookup/binding explanations. source_bound requires a direct source class/import/alias, a proven builtin descriptor, matching local member or source MRO, and no recognized mutation/escape/customization. Records receiver versus definition class, implicit binding, source hashes/evidence, conditions, assumptions and issues; runtime_target_confirmed is always false. unconfirmed is not a binding proof.
calls[*] analysis_contexts Per-expansion facts tied to incoming_call_id: symbolic parameter_facts, target/candidates/status, binding status, argument/callable/result sources, effects, conditional returns and conditions. The top-level call is a conservative union; do not combine facts from different contexts.
calls[*] conditional_returns Guarded ordinary-parameter alias facts for a supported early-return path, with requires_literal_guard. Proven incoming objects optionally add container_shape, caller-local object_source, binding and input_evidence. completeness="partial_paths" does not authorize unconditional alias/content updates or prove the whole helper pure.
calls[*].argument_sources[*] literal, value_evidence Bounded source-proven builtin scalar/tuple/list values used for branch selection and finite-key constraints, separate from dependency roots. Mutable-container writes and unknown escapes invalidate stale literal facts.
returned object facts conditions Branch evidence and the exact incoming literal binding selecting the alias path. Effects are applied before alias instantiation, and the caller-local object identity refers to current contents.
mapping element effects sequence_binding, loop_evidence, operation_conditions Finite incoming tuple/list string keys, their binding/source evidence, and the per-iteration membership guard. A complete summary describes normal_return; a partial source site has completion="unproven", reachability="not_proven", no strong update and unknown final state.
class/instance sources attribute_provenance, instance_candidates Restricted class-attribute write/read evidence and source allocation alternatives. value_incomplete=true marks an open configuration state; the downstream call remains unresolved even when known source candidates exist. Initial None is not a runtime configuration proof.
boundaries[*] affected_scope, affected_values Known roots and element paths affected by the boundary, or explicit none/unknown scope.
source boundaries entry_relation, relation_basis, unaffected_values Static import reachability and entry parameters excluded from value impact because they are never read in the entry body; reflection suppresses exclusions. Neither is a runtime reachability proof.

Existing fields and schema_version="flow-0.2" are unchanged. Consumers must continue to treat a selected method or decorated target/class as conservative when a corresponding override/decorator boundary is present. target_status="builtin_allocation" reports a supported unshadowed builtin allocator result without inventing a Python definition or target candidate. builtin_allocator_source_unavailable retains the allocator body boundary. dynamic_construction, dynamic_class_receiver_override_possible, class_return_recursion, and class_return_budget preserve uncertainty in construction, class dispatch, and bounded class-return proofs. decorator_effects is a positive, finite syntax proof, not the absence of a mutation warning. First-phase preserved body/binding proofs permit only modeled input-function __doc__ writes, local aliases and bounded source helper/super composition. Recognized code/default writes report modified; signature or other unsupported attribute writes do not receive a preservation proof. Unproved dispatch, opaque calls/escape, implicit returns, unsupported syntax, multiple decorators, recursion and exhausted proof budgets retain unknown. decorator_effect_unconfirmed exposes function-scoped issues. Proven decorations can annotate their own decorated_caller_semantics / decorated_target_candidate boundaries with boundary_kind="assumption", body_binding_status="preserved" and explicit assumptions; independent receiver, effect and budget boundaries remain. These facts neither confirm runtime targets nor claim the decorator pure. Additive decoration effects distinguish documentation_cache construction, conditional setdefault and formatting mapping_lookup from writes to input_function. Source helper_call effects retain exact targets/bindings; partial effects never substitute for positive body/binding proof. Method-specific builtin_method_binding evidence in effects or receiver_type_evidence does not give a dict subclass builtin container shape. An inherited builtin method has no fabricated Python target; builtin_method_candidate retains dynamic/unknown key boundaries and does not perform a heap strong update. Source-unavailable decoration calls can include source_target, source_available and affected_roles without declaring the call safe. Stable ownership 1.0 is unchanged. For a proven source classmethod binding, the existing dynamic_class_receiver_override_possible boundary remains but has boundary_kind="assumption", binding_status="source_bound" and explicit assumptions. This distinguishes theoretical external replacement from a recognized uncertainty; it does not confirm a runtime target or make argument binding complete. Unproved binding or a visible change retains an uncertainty boundary, with class_method_binding_unconfirmed exposing evidence-bearing issues. Per-expansion analysis_contexts also carries class_method_bindings; consumers must keep a proof with its corresponding incoming context. A merged call can contain both proven and unproved facts, so a single source_bound record does not cover other contexts. Returned-container proofs also require complete binding and an already proven builtin input shape. container_return_unproven includes an explanatory detail and affected input roots; container_return_budget preserves proof truncation. Capture dictionaries, copies, mixed returns, generators, decorators, and helpers with unmodeled effects do not acquire an original-object alias. See returned container objects for the distinction between dependencies, shape, identity, and current contents. mapping_effect_unproven prevents partial, unknown-shape, unstable-callable, or uncertain-binding helper summaries from imposing strong updates; mapping_effect_budget records truncated proofs. Required-key removals retain mapping_effect_exception_path. Effects joined with an unproven context have state_after="conditional" and status="partial_context", not an unconditional final-absence guarantee. All extensions stay in flow-0.2.

The CLI selects this contract with --value-flow and --entry MODULE:QUALNAME. In that mode, --json emits flow JSON rather than the ownership schema. Commands without --value-flow retain the stable ownership contract. See the value-flow CLI and pandas example for the complete usage and boundary semantics.

Full provenance JSON (--json)

all_api_calls is the primary consumption surface. all_symbol_provenance provides the explanation and evidence layer: it records which imports, aliases, assignments, parameters, returns, attributes, containers, and decorators support each call classification.

{
  "schema_version": "1.0",
  "profile": "full",
  "project_root": ".",
  "stats": {},
  "diagnostics": [],
  "files": [],
  "all_api_calls": [],
  "all_symbol_provenance": [],
  "library_usage": {}
}

all_api_calls[*] stable fields

Field Type Description
expression string Full call expression text
func_name string Function name without arguments
parameters string Argument text
top_library string Resolved primary owner: import-backed top-level library name, or local, python, unknown. PCResolve does not distinguish stdlib from PyPI: any import-backed owner keeps its top-level name (e.g. json, pathlib, requests, numpy).
base_symbol string Root/base symbol used for resolution
reason string DIRECT_IMPORT, RETURN_PROPAGATION, FLOW_MERGE, ...
confidence float 0.0–1.0
alternatives list Alternative top libraries
decorated_by list Decorator library evidence
file_path string Relative POSIX path from project_root
lineno int
col_offset int
end_lineno int
end_col_offset int
chain list Trace chain
resolved_func string Fully qualified function path
resolved_chain list Resolved trace chain

resolved_func semantics

resolved_func attempts to qualify the called function through its receiver's provenance. It is a best-effort display hint, not a guaranteed precise resolution:

  • Constructor calls: when PCResolve identifies the receiver class via import_from_symbols, resolved_func includes the class path (e.g. requests.Session.get, flask.Flask.test_client).
  • Factory returns: when the receiver traces through a local function that returns an import-backed library-owned object, the class info is typically not preserved. PCResolve may produce a library-level function (e.g. requests.get) when the call path normalizes; otherwise the original receiver expression is preserved while top_library carries the ownership.
  • Local / unknown: stays as the original expression or local.
  • Zero-argument super() methods: for a statically known direct single external base, the hint uses the base's import path and method name (e.g. tensorflow.keras.layers.Layer.get_config). Unsupported or ambiguous inheritance and explicit super(...) retain the original function name. This does not identify the method's internal defining class. Class decorators remain conservative, with a narrow exception for an import-backed tensorflow.keras.utils.register_keras_serializable(...) call using literal package/name configuration on a module-level class. This requires a direct named import (including a symbol alias); wildcard imports, visible rebinding, and module-attribute decorator calls remain unsupported. Its registration implementation returns the original class; unknown or ambiguous decorators still prevent expansion, including when stacked with this registration decorator.
  • resolved_func must not be treated as an importable symbol; it may not exist at that path in the library.

resolved_chain is [func_name, resolved_func, top_library].

Summary JSON (--json-summary)

{
  "schema_version": "1.0",
  "profile": "summary",
  "project_root": ".",
  "stats": {},
  "diagnostics": [],
  "libraries": {}
}

Summary excludes all_api_calls, all_symbol_provenance, and per-file symbols/chains.

Path normalization

All paths use POSIX separators (/) relative to project_root. External paths use the <external>/... prefix.

Reason constants

Reason Meaning
DIRECT_IMPORT Call traced directly to an import alias or from-import
TRANSITIVE_IMPORT Call traced through a re-export or transitive module chain
LOCAL_DEFINITION Call is a locally defined function, method, or class
BUILTIN Call is a Python builtin (no import required)
PARAMETER_PROPAGATION Source traced through a function parameter
RETURN_PROPAGATION Source traced through a function return value
FLOW_MERGE Multiple branches/sources merged (if/else, multi-return, SourceSet)
UNRESOLVED Trace could not reach a terminal origin

PARAMETER_PROPAGATION remains a stable reason value. Parameter evidence often resolves before final classification, so the resulting call may instead report RETURN_PROPAGATION, FLOW_MERGE, or TRANSITIVE_IMPORT.

Confidence rules

Reason Confidence
DIRECT_IMPORT 1.0
LOCAL_DEFINITION 1.0
BUILTIN 1.0
PARAMETER_PROPAGATION 0.9
RETURN_PROPAGATION 0.9
TRANSITIVE_IMPORT 0.9
FLOW_MERGE (single) 0.85
FLOW_MERGE (N alts) max(1/N, 0.2)
FLOW_MERGE with a conservative local primary 0.5
UNRESOLVED 0.0