Skip to content

feat(arxjit): resolve a compilable signature from Python annotations - #103

Open
Jaskirat-s7 wants to merge 3 commits into
arxlang:mainfrom
Jaskirat-s7:feat/arxjit-signature-reconcile
Open

feat(arxjit): resolve a compilable signature from Python annotations#103
Jaskirat-s7 wants to merge 3 commits into
arxlang:mainfrom
Jaskirat-s7:feat/arxjit-signature-reconcile

Conversation

@Jaskirat-s7

Copy link
Copy Markdown
Contributor

Follows #102. Closes the remaining open question in wiki Issue 2: "decide how Python annotations interact with explicit signatures."

The rules

New arxjit.reconcile decides which Signature a validated function will be compiled against:

given result
explicit signature= wins outright; annotations are not read at all
annotations only signature derived from them
neither reported at WARNING severity, function stays interpreted
some annotations, others missing or unsupported every remaining gap is an ERROR

Because an explicit signature short-circuits, a function may annotate freely without having to agree with it — there is deliberately no mismatch error, per the decision in the thread. The last row is the judgement call worth flagging: once a function annotates anything, it is read as asking to be compiled, so a gap elsewhere is an error rather than a silent fallback.

Annotations are read from the ast, not __annotations__

A module using from __future__ import annotations turns every annotation into a string at runtime, which would otherwise have to be evaluated back with eval_str. Reading the ast makes that irrelevant, and lets each diagnostic point at the exact annotation that caused it rather than at the function.

Only a bare int, float or bool is accepted; a subscripted (list[int]), dotted (np.float64) or quoted ("int") annotation is reported against its own location. One consequence worth naming: this can never produce i32 or f32, since Python has no annotation that distinguishes widths, so the narrow types stay explicit-signature only. That partly answers the "should we trim i32/f32?" question from #91 — they remain reachable, just not from annotations.

Positional-only parameters are included in the derived signature. Validation permits them and they are ordinary positional arguments to a compiled function, so leaving them out would silently drop an argument.

Fail-closed on argument shapes

Signature is a fixed list of positional argument types, so a variadic or keyword-only parameter has nowhere to go. Validation rejects those shapes first, which makes this unreachable through @jit — but resolve_signature is exported, and without a check it would leave the parameters out and hand back a signature that is quietly wrong (i64() for a function taking *args). It now reports that it cannot derive one instead. Four regression tests cover *args, **kwargs, keyword-only, and the mixed positional-plus-variadic case.

Location helpers moved

_char_column and _diagnostic move out of validation.py into a new arxjit.locations, now that a second stage builds diagnostics from ast nodes. Single-sourcing them keeps the two stages from drifting on the column contract (one-based Unicode characters, converted from ast's zero-based UTF-8 byte offsets at the boundary). No behaviour change — a move, plus a severity parameter so reconciliation can emit WARNING.

Note on scope

resolve_signature is exported and fully tested but not yet consulted by @jitcore.py is untouched here. Wiring it into the decorator, along with the strict flag and the warn-and-fall-back behaviour, is the next PR. Splitting it this way keeps the resolution rules reviewable on their own, separately from the decorator behaviour change.

Verification

169 tests, arxjit coverage 100%. All gates green locally: pytest+cov, ruff format/check, mypy strict (cd packages/arxjit && mypy src), bandit -iii -lll, vulture 80, mccabe 10, douki sync idempotent. Verified on CPython 3.10.19, 3.11.11 and 3.14.6.

@xmnlab

xmnlab commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

thanks for working on that @Jaskirat-s7

some comments from my side:

1. High — explicit-signature arity is still unchecked

resolve_signature() still returns an explicit signature immediately. It does not compare the number of explicit.arg_types with the function’s positional and positional-only parameters.

Therefore, this still succeeds incorrectly:

def add(a, b):
    return a + b

resolve_signature(extract_source(add), i64(i64))

Signature accepts any argument-type tuple and does not perform that structural check itself.

An explicit signature should override annotation types, but it should not be allowed to describe a different number of arguments from the function.

2. Medium — annotation types are still resolved by spelling

Bare names spelled int, float, or bool are accepted without checking what those names resolve to in the function’s namespace.

For example:

int = str

def identity(value: int) -> int:
    return value

The resolver derives i64(i64), even though Python resolves both annotations to str.

ExtractedSource already carries globalns and freevars, so the resolver has most of the context needed to reject shadowed builtin annotation names or deliberately document them as reserved ArxJIT syntax.

CI

The package tests, language tests, and linter pass. The main workflow currently fails only because the PR branch is no longer up to date with origin/main; it needs to be rebased or updated.

Once the two reconciliation issues are addressed and the branch is updated, this should be ready for another review.

Wiki Issue 2 leaves open how Python annotations interact with an explicit
signature. New arxjit.reconcile decides the Signature a validated
function will be compiled against: an explicit signature= wins outright
and the annotations are not read at all, so a function may annotate
freely without having to agree with it and there is deliberately no
mismatch error; otherwise the signature is derived from the annotations;
a function with neither is reported at WARNING severity and stays
interpreted, while a function that annotates some things but not others
is asking to be compiled, so every remaining gap is an ERROR.

Annotations are read from the ast rather than from __annotations__. A
module using "from __future__ import annotations" turns every annotation
into a string at runtime, which would otherwise have to be evaluated
back; reading the ast makes that irrelevant and lets each diagnostic
point at the exact annotation that caused it. Only a bare int, float or
bool is accepted, so a subscripted, dotted or quoted annotation is
reported against its own location. Note this can never produce i32 or
f32, since Python has no annotation that distinguishes widths: the
narrow types stay explicit-signature only.

Positional-only parameters are included in the derived signature.
Validation permits them and they are ordinary positional arguments to a
compiled function, so leaving them out would silently drop an argument.

Move the diagnostic location helpers out of validation into
arxjit.locations, now that a second stage builds diagnostics from ast
nodes, so the two cannot drift on the column contract.

resolve_signature is exported and tested but not yet consulted by @jit;
wiring it into the decorator is the next change.

160 tests, arxjit coverage 100%.
The douki schema accepts a parameter entry as either a string or a mapping
of type/description/optional/default/variadic. Three test helpers with
deliberately unannotated parameters were left with a bare key, which parses
as None and matches neither, so the douki-arxjit hook failed in CI while
reporting the files as unchanged.

Give those parameters a description; douki normalizes the string form to a
description mapping and is then idempotent.
Review found the explicit-signature short-circuit was too broad. It was
written for the rule that an explicit signature overrides annotation
*types*, but it returned before every other check, so a signature could
also contradict the definition itself:

    def add(a, b): ...
    resolve_signature(extract_source(add), i64(i64))   # succeeded

How many parameters a function has is a fact of the definition, not a
choice the caller restates, and a signature that disagrees would compile
to a calling convention Python cannot satisfy. The structural checks now
run before the explicit branch, so both paths are held to them: the
argument shape, which the short-circuit also skipped so an explicit
signature bypassed the variadic guard as well, and the argument count. An
unsupported shape stops the check there, since counting the positional
parameters of a function that also takes *args reports a number no caller
could act on.

Also stop resolving annotation types by spelling alone. A name matching
int, float or bool was mapped without asking what it currently refers to,
so a module that rebinds one derived a signature for a type it does not
use:

    int = str
    def identity(value: int) -> int: ...   # derived i64(i64)

This is the same shape as the shadowed range in arxlang#99, and the namespace
added for it settles this too: an annotation name now has to resolve to
the builtin it is spelled after. Only module-level rebinding is
consulted. A function's own locals cannot apply, because annotations are
evaluated in the enclosing scope when the def executes, and an enclosing
function's rebinding is not observable here since freevars records only
the names the body reads. Absent a namespace the name is assumed to be
the builtin, matching the documented fail-open in arxlang#99.

Auditing the new code for lines that earn nothing turned up three, none
of which line coverage could see: the mismatch message pluralized the
declared count but no test produced the plural, since both halves sit on
one conditional expression; the builtin lookup carried a default that was
unreachable and, had it fired, would have reported every module merely
defining the name as shadowing it; and _annotation_type existed only to
prove a node was an ast.Name, losing that narrowing across the call so
its one caller had to cast it back. The first gained a test, the second
lost its default, and the third was merged into its caller.

179 tests, arxjit coverage 100% including branches. Verified on CPython
3.10, 3.11 and 3.14.
@Jaskirat-s7
Jaskirat-s7 force-pushed the feat/arxjit-signature-reconcile branch from a59bad1 to 6b9c3c5 Compare August 4, 2026 16:54
@Jaskirat-s7

Copy link
Copy Markdown
Contributor Author

Both good catches, thanks, fixed in the latest commit.
@xmnlab , @yuvimittal
On the arity one, you're right, and it turned out wider than your example. The short-circuit was written for the rule that an explicit signature overrides annotation types, but it returned before everything else, so it was also skipping the variadic guard from the last round ,@jit(signature=i64(i64)) on a def f(*args) sailed straight through too. So rather than just adding an arity check, I moved the structural checks ahead of the explicit branch, and both paths get them now. An unsupported shape stops there and doesn't also report a count, since "takes 0 arguments" for a *args function is a number nobody can act on.

for the second change I was matching on spelling and never asking what the name currently pointed at.
Annotation names now have to resolve to the builtin they're spelled after, using the namespace already on ExtractedSource from that work.

One thing I'd like your call on: I've only consulted globalns. A function's own locals can't matter, since annotations are evaluated in the enclosing scope when the def runs. But an enclosing function rebinding int is real and i don't think it's observable here — freevars only records names the body reads, and an annotation isn't part of the body. Is module-level enough, or would you rather I document the enclosing case as out of contract with a test pinning it, like we did for the method forms in #102?

While auditing before pushing I also pulled out three things that weren't earning their place, none of which line coverage could see: the mismatch message pluralized the count but no test ever produced the plural (both halves sit on one ternary); the builtin lookup had an unreachable default that would have reported every module merely defining the name as shadowing it, if it had ever fired; and _annotation_type existed only to prove a node was an ast.Name and then lose that narrowing, so its one caller had to cast it back. First got a test, second lost the default, third got merged in.
179 tests, 100% coverage including branches, checked on 3.10/3.11/3.14.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants