Skip to content

Commit 830dc28

Browse files
committed
Document canonical source generation
Explain immutable format context, path and import factoring, extensions, and executable source generation from the canonical pycodify owner. Verification: documentation validator (4 sources, 12 Python blocks), detached strict Sphinx build (4 sources), typed-extension smoke, and cached diff check.
1 parent 76d58e7 commit 830dc28

6 files changed

Lines changed: 215 additions & 174 deletions

File tree

README.md

Lines changed: 14 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ class Config:
1414
value: int = 42
1515

1616
config = Config(name="production", value=100)
17-
code = generate_python_source(Assignment("config", config), clean_mode=True)
17+
code = generate_python_source(Assignment("config", config))
1818
print(code)
1919
# Output:
2020
# from __main__ import Config
@@ -27,9 +27,12 @@ print(code)
2727
|--------|:--------:|:-----------:|:--------:|:---------------:|:-------------:|
2828
| pickle ||||||
2929
| JSON/YAML ||||||
30-
| Python source ||||| |
30+
| Python source ||||| Depends on imported APIs |
3131

32-
Binary formats like `pickle` cannot be diffed, inspected, or edited. Text formats like JSON lose type information. Python source code has all desired properties: it is diffable, inspectable, editable, type-preserving, and cross-version stable.
32+
Binary formats like `pickle` cannot be diffed, inspected, or edited. Text
33+
formats like JSON lose type information. Python source is diffable, inspectable,
34+
editable, and type-preserving, but replay still depends on the imported APIs and
35+
constructor signatures remaining compatible.
3336

3437
## Features
3538

@@ -41,6 +44,14 @@ Binary formats like `pickle` cannot be diffed, inspected, or edited. Text format
4144

4245
## Documentation
4346

47+
Explicit mode is the default (``clean_mode=False``), preserving every field for
48+
reproducibility. Pass ``clean_mode=True`` only when concise, default-eliding
49+
source is desired.
50+
51+
Generated output is executable Python, not a sandboxed data format. Review it
52+
before execution and never call `exec` on source generated from untrusted
53+
objects, custom formatters, headers, or edited files.
54+
4455
Full documentation available at [pycodify.readthedocs.io](https://pycodify.readthedocs.io)
4556

4657
## Installation

docs/api.rst

Lines changed: 23 additions & 97 deletions
Original file line numberDiff line numberDiff line change
@@ -1,106 +1,32 @@
1-
API Reference
2-
=============
1+
API orientation
2+
===============
33

4-
Core API
5-
--------
4+
``Assignment`` and ``CodeBlock``
5+
Structured top-level source items.
66

7-
.. automodule:: pycodify.core
8-
:members:
9-
:undoc-members:
10-
:show-inheritance:
11-
12-
Formatters
13-
----------
14-
15-
.. automodule:: pycodify.formatters
16-
:members:
17-
:undoc-members:
18-
:show-inheritance:
19-
20-
Import Resolution
21-
-----------------
22-
23-
.. automodule:: pycodify.imports
24-
:members:
25-
:undoc-members:
26-
:show-inheritance:
27-
28-
Main Functions
29-
--------------
30-
31-
.. py:function:: generate_python_source(assignment: Assignment, clean_mode: bool = True) -> str
32-
33-
Generate executable Python source code from an object.
34-
35-
:param assignment: Assignment object specifying variable name and value
36-
:type assignment: Assignment
37-
:param clean_mode: If True, omit fields matching defaults. If False, include all fields.
38-
:type clean_mode: bool
39-
:return: Executable Python source code with imports
40-
:rtype: str
41-
42-
**Example:**
43-
44-
.. code-block:: python
45-
46-
from pycodify import Assignment, generate_python_source
47-
from dataclasses import dataclass
48-
49-
@dataclass
50-
class Config:
51-
name: str = "default"
52-
value: int = 42
7+
``generate_python_source(obj, header="", clean_mode=False, *, context=None)``
8+
Generate complete executable source and required imports. A caller-supplied
9+
``FormatContext`` is retained across both render passes and owns clean mode.
5310

54-
config = Config(name="production", value=100)
55-
code = generate_python_source(Assignment("config", config))
56-
print(code)
11+
``to_source(value, context=None)``
12+
Format one value into a ``SourceFragment``.
5713

58-
.. py:class:: Assignment
14+
``SourceFormatter``
15+
Nominal extension point. Subclasses implement ``can_format`` and ``format``;
16+
application-specific formatters register without changing pycodify core.
5917

60-
Represents a variable assignment for code generation.
18+
``resolve_imports``
19+
Resolve import requirements and aliases for colliding names.
6120

62-
.. py:method:: __init__(name: str, value: Any)
21+
``FormatContext`` and ``SourceFragment``
22+
Immutable typed formatting state and output records. ``FormatContext`` may
23+
carry exact-type ``extensions`` and exposes them through ``extension(type)``.
6324

64-
Create an assignment.
25+
The canonical import surface is ``pycodify.__all__``.
6526

66-
:param name: Variable name
67-
:type name: str
68-
:param value: Value to serialize
69-
:type value: Any
70-
71-
.. py:class:: SourceFormatter
72-
73-
Base class for custom formatters.
74-
75-
.. py:method:: can_format(value: Any) -> bool
76-
77-
Check if this formatter can handle the value.
78-
79-
:param value: Value to check
80-
:type value: Any
81-
:return: True if this formatter can format the value
82-
:rtype: bool
83-
84-
.. py:method:: format(value: Any, context: FormatContext) -> SourceFragment
85-
86-
Format a value to source code.
87-
88-
:param value: Value to format
89-
:type value: Any
90-
:param context: Formatting context with state
91-
:type context: FormatContext
92-
:return: Source code fragment with imports
93-
:rtype: SourceFragment
94-
95-
.. py:class:: SourceFragment
96-
97-
Represents generated source code with its imports.
98-
99-
.. py:attribute:: code
100-
101-
The generated Python source code.
102-
103-
.. py:attribute:: imports
104-
105-
Set of (module, name) tuples required for the code.
27+
Public API
28+
----------
10629

30+
.. automodule:: pycodify
31+
:members:
32+
:member-order: bysource

docs/architecture.rst

Lines changed: 53 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -8,9 +8,12 @@ Generating executable source requires knowing import aliases before emitting cod
88

99
pycodify solves this with two passes:
1010

11-
1. **Collection Pass**: Traverse the object, emit code fragments, collect ``(module, name)`` import pairs
12-
2. **Resolution**: Detect collisions (same name, different modules), generate deterministic aliases
13-
3. **Regeneration Pass**: Re-traverse with resolved ``name_mappings``, emit final code
11+
1. **Collection pass**: Traverse the object, emit code fragments, and collect
12+
``(module, name)`` import pairs.
13+
2. **Resolution**: ``resolve_imports`` sorts the pairs, detects names exported by
14+
more than one module, and derives aliases from each module path.
15+
3. **Regeneration pass**: Re-traverse with the resolved ``name_mappings`` and
16+
emit final code using those local names.
1417

1518
This is not an optimization—it is structurally necessary. A single-pass algorithm cannot know whether ``Config`` needs aliasing until it has seen all types that might also be named ``Config``.
1619

@@ -45,13 +48,39 @@ Core Components
4548
Represents a piece of generated code with its required imports.
4649

4750
**FormatContext**
48-
Carries state during formatting: ``name_mappings``, ``visited_objects``, etc.
49-
50-
**ImportResolver**
51-
Detects collisions and generates deterministic aliases for imports.
52-
53-
**SourceGenerator**
54-
Orchestrates the two-pass algorithm and produces final executable code.
51+
Carries indentation depth, ``clean_mode``, the ``(module, name)`` to
52+
local-name mapping, and caller-owned typed extensions. ``indented()`` returns
53+
a copied context and does not mutate its caller.
54+
55+
**resolve_imports**
56+
Detects collisions and returns import lines plus the name mapping used by the
57+
regeneration pass.
58+
59+
**generate_python_source**
60+
Orchestrates collection, resolution, and regeneration to produce a complete
61+
source string.
62+
63+
Immutable typed context extensions
64+
----------------------------------
65+
66+
``FormatContext`` is frozen. Its default mappings are read-only mapping proxies,
67+
and context evolution uses dataclass replacement rather than mutation. A host
68+
may supply an ``extensions`` mapping keyed by the exact nominal type of each
69+
rendering policy or state record. A formatter retrieves one value with
70+
``context.extension(ExtensionType)``; pycodify does not search base classes,
71+
string keys, or a fallback chain.
72+
73+
``generate_python_source(..., context=context)`` threads the caller's context
74+
through collection and regeneration. Import resolution replaces only
75+
``name_mappings`` for the second pass, so typed extensions and all other context
76+
fields survive. When a context is supplied, its ``clean_mode`` is authoritative;
77+
the separate ``clean_mode=`` argument is used only when pycodify constructs the
78+
context.
79+
80+
Extensions provide formatting context, not formatter dispatch. New value
81+
families still extend the ``SourceFormatter`` registry, while the exact extension
82+
type allows that formatter to obtain host-owned rendering information without
83+
adding host imports or fields to pycodify core.
5584

5685
Clean Mode
5786
----------
@@ -64,6 +93,20 @@ Clean mode omits fields matching their default values. This requires:
6493

6594
Explicit mode includes all fields for complete reproducibility.
6695

96+
Stability and trust boundary
97+
----------------------------
98+
99+
Import ordering and collision aliases are deterministic for a fixed set of
100+
``(module, name)`` pairs. The library does not promise that arbitrary object
101+
graphs produce byte-identical text across runs: container iteration, object
102+
``repr`` implementations, custom formatters, and dependency versions can all
103+
affect output or replay.
104+
105+
The output is executable Python. pycodify owns formatting and import resolution,
106+
not sandboxing or validation of generated programs. Hosts must restrict inputs
107+
and custom formatters to trusted sources, review persisted code before running
108+
it, and define their own compatibility policy for imported domain APIs.
109+
67110
Lazy Dataclass Integration
68111
---------------------------
69112

@@ -86,4 +129,3 @@ Module Structure
86129
- **core.py**: Main API (``Assignment``, ``generate_python_source``)
87130
- **formatters.py**: Built-in formatters (dataclass, enum, primitive types)
88131
- **imports.py**: Import resolution and collision handling
89-

docs/conf.py

Lines changed: 4 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,16 +1,17 @@
11
"""Sphinx configuration for pycodify."""
22
import os
33
import sys
4+
from pathlib import Path
45

56
# Add source to path
6-
sys.path.insert(0, os.path.abspath("../src"))
7+
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
78

89
# Project information
910
project = "pycodify"
1011
copyright = "2024, Tristan Simas"
1112
author = "Tristan Simas"
12-
version = "0.1.0"
13-
release = "0.1.0"
13+
version = "0.1"
14+
release = "0.1.2"
1415

1516
# General configuration
1617
extensions = [
@@ -54,7 +55,6 @@
5455
html_theme = "sphinx_rtd_theme"
5556
html_theme_options = {
5657
"logo_only": False,
57-
"display_version": True,
5858
"prev_next_buttons_location": "bottom",
5959
"style_external_links": False,
6060
"vcs_pageview_mode": "",
@@ -70,4 +70,3 @@
7070
intersphinx_mapping = {
7171
"python": ("https://docs.python.org/3", None),
7272
}
73-

docs/index.rst

Lines changed: 14 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -1,50 +1,27 @@
1-
pycodify Documentation
2-
======================
1+
pycodify
2+
========
33

4-
**pycodify** serializes Python objects to executable Python source code with automatic import resolution.
4+
pycodify serializes Python objects to executable Python source while resolving
5+
imports and name collisions.
56

6-
The key insight: **Python source code is a serialization format.** Rather than inventing a format and writing loaders, pycodify emits code that Python itself interprets. The import system becomes the deserializer.
7-
8-
Quick Start
7+
Quick start
98
-----------
109

1110
.. code-block:: python
1211
13-
from pycodify import Assignment, generate_python_source
1412
from dataclasses import dataclass
13+
from pycodify import Assignment, generate_python_source
1514
1615
@dataclass
1716
class Config:
1817
name: str = "default"
1918
value: int = 42
2019
2120
config = Config(name="production", value=100)
22-
code = generate_python_source(Assignment("config", config), clean_mode=True)
23-
print(code)
24-
# Output:
25-
# from __main__ import Config
26-
# config = Config(name='production', value=100)
27-
28-
Features
29-
--------
30-
31-
- **Complete Executable Source**: Generates imports + code, not just expressions
32-
- **Type-Preserving**: Enums, Paths, callables serialize as themselves
33-
- **Collision Handling**: Automatic aliasing for name collisions across modules
34-
- **Clean Mode**: Omit fields matching defaults for concise output
35-
- **Extensible**: Register custom formatters for domain-specific types
36-
37-
Why Python Source?
38-
------------------
21+
code = generate_python_source(Assignment("config", config))
3922
40-
| Format | Diffable | Inspectable | Editable | Type-preserving | Cross-version |
41-
|--------|:--------:|:-----------:|:--------:|:---------------:|:-------------:|
42-
| pickle | ✗ | ✗ | ✗ | ✓ | ✗ |
43-
| JSON/YAML | ✓ | ✓ | ✓ | ✗ | ✓ |
44-
| Python source | ✓ | ✓ | ✓ | ✓ | ✓ |
45-
46-
Contents
47-
--------
23+
Explicit mode is the default (``clean_mode=False``) and includes default-valued
24+
fields. Pass ``clean_mode=True`` only when default elision is desired.
4825

4926
.. toctree::
5027
:maxdepth: 2
@@ -53,20 +30,9 @@ Contents
5330
api
5431
architecture
5532

56-
API Reference
57-
-------------
58-
59-
.. autosummary::
60-
:toctree: _autosummary
61-
62-
pycodify.core
63-
pycodify.formatters
64-
pycodify.imports
65-
66-
Indices and Tables
67-
------------------
68-
69-
* :ref:`genindex`
70-
* :ref:`modindex`
71-
* :ref:`search`
33+
Ownership
34+
---------
7235

36+
pycodify owns source fragments, imports, formatter registration, collision
37+
handling, and generic Python formatting. Host packages own their domain
38+
formatters and round-trip policies.

0 commit comments

Comments
 (0)