feat(arxjit): resolve a compilable signature from Python annotations - #103
feat(arxjit): resolve a compilable signature from Python annotations#103Jaskirat-s7 wants to merge 3 commits into
Conversation
|
thanks for working on that @Jaskirat-s7 some comments from my side: 1. High — explicit-signature arity is still unchecked
Therefore, this still succeeds incorrectly: def add(a, b):
return a + b
resolve_signature(extract_source(add), i64(i64))
An explicit signature should override annotation types, but it should not be allowed to describe a different number of arguments from the function. 2. Medium — annotation types are still resolved by spellingBare names spelled For example: int = str
def identity(value: int) -> int:
return valueThe resolver derives
CIThe package tests, language tests, and linter pass. The Once the two reconciliation issues are addressed and the branch is updated, this should be ready for another review. |
Wiki Issue 2 leaves open how Python annotations interact with an explicit signature. New arxjit.reconcile decides the Signature a validated function will be compiled against: an explicit signature= wins outright and the annotations are not read at all, so a function may annotate freely without having to agree with it and there is deliberately no mismatch error; otherwise the signature is derived from the annotations; a function with neither is reported at WARNING severity and stays interpreted, while a function that annotates some things but not others is asking to be compiled, so every remaining gap is an ERROR. Annotations are read from the ast rather than from __annotations__. A module using "from __future__ import annotations" turns every annotation into a string at runtime, which would otherwise have to be evaluated back; reading the ast makes that irrelevant and lets each diagnostic point at the exact annotation that caused it. Only a bare int, float or bool is accepted, so a subscripted, dotted or quoted annotation is reported against its own location. Note this can never produce i32 or f32, since Python has no annotation that distinguishes widths: the narrow types stay explicit-signature only. Positional-only parameters are included in the derived signature. Validation permits them and they are ordinary positional arguments to a compiled function, so leaving them out would silently drop an argument. Move the diagnostic location helpers out of validation into arxjit.locations, now that a second stage builds diagnostics from ast nodes, so the two cannot drift on the column contract. resolve_signature is exported and tested but not yet consulted by @jit; wiring it into the decorator is the next change. 160 tests, arxjit coverage 100%.
The douki schema accepts a parameter entry as either a string or a mapping of type/description/optional/default/variadic. Three test helpers with deliberately unannotated parameters were left with a bare key, which parses as None and matches neither, so the douki-arxjit hook failed in CI while reporting the files as unchanged. Give those parameters a description; douki normalizes the string form to a description mapping and is then idempotent.
Review found the explicit-signature short-circuit was too broad. It was
written for the rule that an explicit signature overrides annotation
*types*, but it returned before every other check, so a signature could
also contradict the definition itself:
def add(a, b): ...
resolve_signature(extract_source(add), i64(i64)) # succeeded
How many parameters a function has is a fact of the definition, not a
choice the caller restates, and a signature that disagrees would compile
to a calling convention Python cannot satisfy. The structural checks now
run before the explicit branch, so both paths are held to them: the
argument shape, which the short-circuit also skipped so an explicit
signature bypassed the variadic guard as well, and the argument count. An
unsupported shape stops the check there, since counting the positional
parameters of a function that also takes *args reports a number no caller
could act on.
Also stop resolving annotation types by spelling alone. A name matching
int, float or bool was mapped without asking what it currently refers to,
so a module that rebinds one derived a signature for a type it does not
use:
int = str
def identity(value: int) -> int: ... # derived i64(i64)
This is the same shape as the shadowed range in arxlang#99, and the namespace
added for it settles this too: an annotation name now has to resolve to
the builtin it is spelled after. Only module-level rebinding is
consulted. A function's own locals cannot apply, because annotations are
evaluated in the enclosing scope when the def executes, and an enclosing
function's rebinding is not observable here since freevars records only
the names the body reads. Absent a namespace the name is assumed to be
the builtin, matching the documented fail-open in arxlang#99.
Auditing the new code for lines that earn nothing turned up three, none
of which line coverage could see: the mismatch message pluralized the
declared count but no test produced the plural, since both halves sit on
one conditional expression; the builtin lookup carried a default that was
unreachable and, had it fired, would have reported every module merely
defining the name as shadowing it; and _annotation_type existed only to
prove a node was an ast.Name, losing that narrowing across the call so
its one caller had to cast it back. The first gained a test, the second
lost its default, and the third was merged into its caller.
179 tests, arxjit coverage 100% including branches. Verified on CPython
3.10, 3.11 and 3.14.
a59bad1 to
6b9c3c5
Compare
|
Both good catches, thanks, fixed in the latest commit. for the second change I was matching on spelling and never asking what the name currently pointed at. One thing I'd like your call on: I've only consulted While auditing before pushing I also pulled out three things that weren't earning their place, none of which line coverage could see: the mismatch message pluralized the count but no test ever produced the plural (both halves sit on one ternary); the builtin lookup had an unreachable default that would have reported every module merely defining the name as shadowing it, if it had ever fired; and |
Follows #102. Closes the remaining open question in wiki Issue 2: "decide how Python annotations interact with explicit signatures."
The rules
New
arxjit.reconciledecides whichSignaturea validated function will be compiled against:signature=Because an explicit signature short-circuits, a function may annotate freely without having to agree with it — there is deliberately no mismatch error, per the decision in the thread. The last row is the judgement call worth flagging: once a function annotates anything, it is read as asking to be compiled, so a gap elsewhere is an error rather than a silent fallback.
Annotations are read from the ast, not
__annotations__A module using
from __future__ import annotationsturns every annotation into a string at runtime, which would otherwise have to be evaluated back witheval_str. Reading the ast makes that irrelevant, and lets each diagnostic point at the exact annotation that caused it rather than at the function.Only a bare
int,floatorboolis accepted; a subscripted (list[int]), dotted (np.float64) or quoted ("int") annotation is reported against its own location. One consequence worth naming: this can never producei32orf32, since Python has no annotation that distinguishes widths, so the narrow types stay explicit-signature only. That partly answers the "should we trim i32/f32?" question from #91 — they remain reachable, just not from annotations.Positional-only parameters are included in the derived signature. Validation permits them and they are ordinary positional arguments to a compiled function, so leaving them out would silently drop an argument.
Fail-closed on argument shapes
Signatureis a fixed list of positional argument types, so a variadic or keyword-only parameter has nowhere to go. Validation rejects those shapes first, which makes this unreachable through@jit— butresolve_signatureis exported, and without a check it would leave the parameters out and hand back a signature that is quietly wrong (i64()for a function taking*args). It now reports that it cannot derive one instead. Four regression tests cover*args,**kwargs, keyword-only, and the mixed positional-plus-variadic case.Location helpers moved
_char_columnand_diagnosticmove out ofvalidation.pyinto a newarxjit.locations, now that a second stage builds diagnostics from ast nodes. Single-sourcing them keeps the two stages from drifting on the column contract (one-based Unicode characters, converted from ast's zero-based UTF-8 byte offsets at the boundary). No behaviour change — a move, plus aseverityparameter so reconciliation can emit WARNING.Note on scope
resolve_signatureis exported and fully tested but not yet consulted by@jit—core.pyis untouched here. Wiring it into the decorator, along with thestrictflag and the warn-and-fall-back behaviour, is the next PR. Splitting it this way keeps the resolution rules reviewable on their own, separately from the decorator behaviour change.Verification
169 tests, arxjit coverage 100%. All gates green locally: pytest+cov, ruff format/check, mypy strict (
cd packages/arxjit && mypy src), bandit-iii -lll, vulture 80, mccabe 10,douki syncidempotent. Verified on CPython 3.10.19, 3.11.11 and 3.14.6.