Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,14 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- coverage.py dependency

### Added
- **Versioned exact effect presets**: expanded resolver-attested PyMongo/Motor,
filesystem, typed requests/httpx/aiohttp, and typed S3 contracts. Added a
message-bus preset containing only typed confluent-kafka `Producer.produce`,
with staged queue timing. Untyped message clients and Redis's timing-ambiguous
shared sync/async owners are omitted; receiver HTTP clients and S3 object
operations abstain from incomplete URL or Key-only resource identities.
This preset tranche does not include real-world evaluation, and Issue #97
remains open for unsupported families and evaluation.
- **Mypy integration**: Added mypy's build API for type-aware dependency analysis
- New `_get_module_dependencies_via_mypy()` method for full dependency graph extraction
- New `_module_to_file_path()` helper for module resolution
Expand Down
43 changes: 29 additions & 14 deletions docs/effect-contracts.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,47 +127,62 @@ differ from the analyzed snapshot.

Six conservative, independently versioned exact-symbol presets are bundled:

- `redis-v1`
- `mongodb-v1`
- `filesystem-v1`
- `http-clients-v1`
- `message-bus-v1`
- `object-storage-v1`
- `sqlalchemy-v1`

Validate one without copying package data:

```bash
fastapi-endpoint-detector validate-effect-contracts --preset redis-v1 --format json
fastapi-endpoint-detector validate-effect-contracts --preset filesystem-v1 --format json
```

Select exactly one preset in configuration:

```yaml
analysis:
effect_preset: redis-v1
effect_preset: filesystem-v1
```

`effect_preset` and `effect_contracts` are mutually exclusive. Presets preserve the
same exact `(canonical symbol, invocation)` matcher and evidence-only behavior as
user documents. Version ranges are audited support metadata, not runtime package
checks. Direct positional/keyword finite strings produce hashed resource
identities. `filesystem-v1` 2.0 additionally traces an exact `pathlib.Path(...)`
identities. `filesystem-v1` additionally traces an exact `pathlib.Path(...)`
or `builtins.open(...)` constructor through one unconditional local assignment
or active `with` binding into exact `_io`/`Path` instance methods. Reassignment,
escaped handles, control flow, aliases, captured handles, composition, dynamic
arguments, and unsupported factories fail closed.

Dynamic boto3 clients without `mypy-boto3-s3`, generic HTTP `request`/`send`,
mode-specific append classification, deferred cursors, Redis pipelines, and bare
method names are intentionally absent.

Each family has an independent identity and semantic hash. Filesystem receiver
origins, exact HTTP verb tables, and composite typed-S3 `(Bucket, Key)` identities
are version `2.0.0`; MongoDB and Redis remain `1.0.0`. HTTP contracts preserve
`GET`, `POST`, `PUT`, `PATCH`, `DELETE`, `HEAD`, or `OPTIONS` as structured
contract semantics while finite URLs remain hashed resource evidence. Typed S3
contracts fail closed unless both bucket and key are finite. The v1 changelog and
known exclusions are frozen in `benchmarks/results/effect-presets-v1/README.md`.
mode-specific `open()` classification, deferred cursor consumption, Redis, and
bare method names are intentionally absent. Redis sync and async clients share
mypy declaration owners, but their immediate-versus-await timing cannot be
expressed by the current exact contract schema. Direct typed-S3 get, put, and
delete operations use the ordered schema-v4 `(Bucket, Key)` composite identity;
copy-source and other compound identities continue to abstain when every
component cannot be represented exactly. Untyped aiokafka, kafka-python, pika,
and kombu rows are also omitted. The message-bus preset contains only typed
confluent-kafka `Producer.produce`, conservatively declared as a staged queue
operation.

Each family has an independent identity and semantic hash. Filesystem and HTTP
contracts are version `3.0.0`; MongoDB/Motor and typed S3 contracts are version
`2.0.0`; message bus starts at `1.0.0`. Requests support starts at its resolver-
typed `2.34` release. HTTP receiver-client contracts abstain from URL resource
identity because constructor `base_url` can make the call argument incomplete;
top-level requests/httpx convenience calls retain finite URL evidence. aiohttp
request timing is conservatively `await`. HTTP contracts preserve `GET`, `POST`,
`PUT`, `PATCH`, `DELETE`, `HEAD`, or `OPTIONS` as structured semantics. The v1
changelog and known exclusions are frozen in
`benchmarks/results/effect-presets-v1/README.md`. That historical artifact is not
real-world evaluation and is not modified by this tranche. Issue #97 remains open
for unsupported package families, additional compound/base-URL identities,
applicability enforcement, version-matrix expansion, and controlled or real-world
evaluation.
Multiple presets are not silently merged because the current provenance model has
one authoritative contract source per analysis.

Expand Down
10 changes: 5 additions & 5 deletions docs/resource-coupling.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,18 +11,18 @@ Configure an effect source and a separate namespace-qualified coupling document:

```yaml
analysis:
effect_preset: redis-v1
effect_preset: filesystem-v1
resource_coupling: .resource-coupling.yaml
```

```yaml
schema_version: 1
mode: report_only
groups:
- id: orders-cache
resource_space: production-orders-redis-db0
producer_contract_ids: [redis-delete, redis-set]
consumer_contract_ids: [redis-get]
- id: generated-files
resource_space: application-data-directory
producer_contract_ids: [pathlib-write-text]
consumer_contract_ids: [pathlib-read-text]
limits:
max_endpoint_links_per_resource: 32
max_edges: 1000
Expand Down
2 changes: 1 addition & 1 deletion src/fastapi_endpoint_detector/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -82,9 +82,9 @@ class AnalysisConfig(BaseModel):
Literal[
"filesystem-v1",
"http-clients-v1",
"message-bus-v1",
"mongodb-v1",
"object-storage-v1",
"redis-v1",
"sqlalchemy-v1",
]
| None
Expand Down
2 changes: 1 addition & 1 deletion src/fastapi_endpoint_detector/models/effect_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,11 +28,11 @@
BUNDLED_EFFECT_PRESETS = {
"filesystem-v1": Path(__file__).parent.parent / "presets" / "effects_filesystem_v1.yaml",
"http-clients-v1": Path(__file__).parent.parent / "presets" / "effects_http_clients_v1.yaml",
"message-bus-v1": Path(__file__).parent.parent / "presets" / "effects_message_bus_v1.yaml",
"mongodb-v1": Path(__file__).parent.parent / "presets" / "effects_mongodb_v1.yaml",
"object-storage-v1": (
Path(__file__).parent.parent / "presets" / "effects_object_storage_v1.yaml"
),
"redis-v1": Path(__file__).parent.parent / "presets" / "effects_redis_v1.yaml",
"sqlalchemy-v1": Path(__file__).parent.parent / "presets" / "effects_sqlalchemy_v1.yaml",
}

Expand Down
173 changes: 155 additions & 18 deletions src/fastapi_endpoint_detector/presets/effects_filesystem_v1.yaml
Original file line number Diff line number Diff line change
@@ -1,69 +1,206 @@
schema_version: 1
preset:
id: stdlib-filesystem-effects
version: 2.0.0
version: 3.0.0
provenance:
kind: preset
source: fastapi-endpoint-detector/effects_filesystem_v1.yaml
revision: "2"
revision: "3"
contracts:
- id: pathlib-read-text
symbol: pathlib.Path.read_text
invocation: instance_method
operation: read
channel: filesystem
resource: {kind: receiver}
package: {python: ">=3.10,<3.14"}
package: &python {python: ">=3.10,<3.14"}
- id: pathlib-read-bytes
symbol: pathlib.Path.read_bytes
invocation: instance_method
operation: read
channel: filesystem
resource: {kind: receiver}
package: {python: ">=3.10,<3.14"}
package: *python
- id: pathlib-write-text
symbol: pathlib.Path.write_text
invocation: instance_method
operation: write
channel: filesystem
resource: {kind: receiver}
value: {kind: argument, index: 0}
package: *python
- id: pathlib-write-bytes
symbol: pathlib.Path.write_bytes
invocation: instance_method
operation: write
channel: filesystem
resource: {kind: receiver}
value: {kind: argument, index: 0}
package: *python
- id: pathlib-touch
symbol: pathlib.Path.touch
invocation: instance_method
operation: write
channel: filesystem
resource: {kind: receiver}
package: *python
- id: pathlib-mkdir
symbol: pathlib.Path.mkdir
invocation: instance_method
operation: write
channel: filesystem
resource: {kind: receiver}
package: *python
- id: pathlib-unlink
symbol: pathlib.Path.unlink
invocation: instance_method
operation: delete
channel: filesystem
resource: {kind: receiver}
package: *python
- id: pathlib-rmdir
symbol: pathlib.Path.rmdir
invocation: instance_method
operation: delete
channel: filesystem
resource: {kind: receiver}
package: *python
- id: pathlib-rename
symbol: pathlib.Path.rename
invocation: instance_method
operation: update
channel: filesystem
resource: {kind: receiver}
value: {kind: argument, index: 0}
package: *python
- id: pathlib-replace
symbol: pathlib.Path.replace
invocation: instance_method
operation: update
channel: filesystem
resource: {kind: receiver}
value: {kind: argument, index: 0}
package: *python

- id: io-text-read
symbol: _io._TextIOBase.read
invocation: instance_method
operation: read
channel: filesystem
resource: {kind: receiver}
package: {python: ">=3.10,<3.14"}
package: *python
- id: io-text-write
symbol: _io._TextIOBase.write
invocation: instance_method
operation: write
channel: filesystem
resource: {kind: receiver}
value: {kind: argument, index: 0}
package: {python: ">=3.10,<3.14"}
package: *python
- id: io-buffered-read
symbol: _io._BufferedIOBase.read
invocation: instance_method
operation: read
channel: filesystem
resource: {kind: receiver}
package: {python: ">=3.10,<3.14"}
package: *python
- id: io-buffered-write
symbol: _io.BufferedWriter.write
invocation: instance_method
operation: write
channel: filesystem
resource: {kind: receiver}
value: {kind: argument, index: 0}
package: {python: ">=3.10,<3.14"}
- id: pathlib-write-text
symbol: pathlib.Path.write_text
invocation: instance_method
package: *python

- id: os-remove
symbol: os.remove
invocation: function
operation: delete
channel: filesystem
resource: {kind: argument, index: 0}
package: *python
- id: os-unlink
symbol: os.unlink
invocation: function
operation: delete
channel: filesystem
resource: {kind: argument, index: 0}
package: *python
- id: os-rmdir
symbol: os.rmdir
invocation: function
operation: delete
channel: filesystem
resource: {kind: argument, index: 0}
package: *python
- id: os-mkdir
symbol: os.mkdir
invocation: function
operation: write
channel: filesystem
resource: {kind: receiver}
resource: {kind: argument, index: 0}
package: *python
- id: os-makedirs
symbol: os.makedirs
invocation: function
operation: write
channel: filesystem
resource: {kind: argument, index: 0}
package: *python
- id: os-rename
symbol: os.rename
invocation: function
operation: update
channel: filesystem
resource: {kind: argument, index: 0}
value: {kind: argument, index: 1}
package: *python
- id: os-replace
symbol: os.replace
invocation: function
operation: update
channel: filesystem
resource: {kind: argument, index: 0}
value: {kind: argument, index: 1}
package: *python

- id: shutil-copyfile
symbol: shutil.copyfile
invocation: function
operation: write
channel: filesystem
resource: {kind: argument, index: 1}
value: {kind: argument, index: 0}
package: {python: ">=3.10,<3.14"}
- id: pathlib-write-bytes
symbol: pathlib.Path.write_bytes
invocation: instance_method
package: *python
- id: shutil-copy
symbol: shutil.copy
invocation: function
operation: write
channel: filesystem
resource: {kind: receiver}
resource: {kind: argument, index: 1}
value: {kind: argument, index: 0}
package: {python: ">=3.10,<3.14"}
package: *python
- id: shutil-copy2
symbol: shutil.copy2
invocation: function
operation: write
channel: filesystem
resource: {kind: argument, index: 1}
value: {kind: argument, index: 0}
package: *python
- id: shutil-move
symbol: shutil.move
invocation: function
operation: update
channel: filesystem
resource: {kind: argument, index: 0}
value: {kind: argument, index: 1}
package: *python
- id: shutil-rmtree
symbol: shutil.rmtree
invocation: function
operation: delete
channel: filesystem
resource: {kind: argument, index: 0}
package: *python
Loading
Loading