Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -153,7 +153,7 @@ lazy val dafny = (project in file("modules/dafny"))

lazy val lint = (project in file("modules/lint"))
.settings(noTestWarts *)
.dependsOn(ir, parser % Test)
.dependsOn(ir, convention, parser % Test)
.settings(
name := "spec-lint",
libraryDependencies ++= commonMainDeps ++ commonTestDeps
Expand Down
4 changes: 2 additions & 2 deletions docs/content/docs/design/architecture.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ flowchart LR
The parser auto-injects a short preamble (`isValidURI`, `isValidEmail`). See
[Parser Implementation Notes](/pipelines/parser-implementation).
2. IR build. CST becomes `ServiceIR`, a Scala 3 enum/case-class ADT, with span tracking.
3. Verify. `verify` runs structural lints (L01-L06; see
3. Verify. `verify` runs structural lints (L01-L07; see
[structural lints](/spec-language#structural-lints)) and then translates each obligation
to either Z3 SMT-LIB or Alloy (non-overlapping routing) and checks it. The translator's
correctness is mechanically validated by the universal `translate_soundness_standalone`
Expand Down Expand Up @@ -130,7 +130,7 @@ in [#391](https://github.com/HardMax71/spec_to_rest/issues/391).
modules/
├── ir/ # IR ADT (Isabelle-extracted SpecRestGenerated.scala), circe Serialize, PrettyPrint, VerifyError
├── parser/ # ANTLR4 grammar, Parse, Builder, Preamble injection
├── lint/ # Structural lints L01-L06
├── lint/ # Structural lints L01-L07
├── convention/ # M1-M10 classifier, naming, path, schema, validate, Builtins registry (single source of truth for spec-language builtin functions)
├── profile/ # Deployment profiles, type mapping, annotation
├── verify/ # Z3 + Alloy translators, backends, Consistency, Diagnostic, Narration
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ lints over the IR; `verify` hands the operation contracts to a solver. They catc

## Structural lints (`check`)

Once a parse succeeds, `check` runs six lints over the IR. Each carries a stable code. An error
Once a parse succeeds, `check` runs seven lints over the IR. Each carries a stable code. An error
exits non-zero; a warning exits zero.

| Code | Level | Catches |
Expand All @@ -19,6 +19,7 @@ exits non-zero; a warning exits zero.
| L04 | warning | two operations with the same input and output signature and equivalent `requires` |
| L05 | warning | an entity declared but never referenced |
| L06 | error | mutually recursive predicates or functions, which the verifier's inlining cannot unfold |
| L07 | warning | an operation whose resolved `204 No Content` success status would drop its declared outputs |

The lints are deliberately narrow. L01 fires only on literals whose type admits no operator at all,
so it catches obvious mistakes without standing in for a full type checker.
Expand Down
2 changes: 1 addition & 1 deletion docs/content/docs/roadmap.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ Per-feature ledger for the verifier. For the prose explanation of each entry, se
| Machine-readable (`--json` / `--json-out`) diagnostics output | shipped |
| Richer suggested-fix templates (op/invariant/field-aware; opt-out via `--no-suggestions`) | shipped |
| Human-readable "why this fails" narration (opt-out via `--no-narration`) | shipped ([#89](https://github.com/HardMax71/spec_to_rest/issues/89)), covers `invariant_violation_by_operation`, `contradictory_invariants`, `unreachable_operation` |
| Structural spec lints (type mismatch, unused entity, ...) | shipped, `check` runs L01-L06 ([#81](https://github.com/HardMax71/spec_to_rest/issues/81)); L04 ships a syntactic over-approximation, SAT-based overlap is candidate verify-side work |
| Structural spec lints (type mismatch, unused entity, ...) | shipped, `check` runs L01-L07 ([#81](https://github.com/HardMax71/spec_to_rest/issues/81)); L04 ships a syntactic over-approximation, SAT-based overlap is candidate verify-side work |
| Per-check VC dump (`--dump-vc <dir>`), Z3 SMT-LIB and Alloy `.als` artifacts | shipped |
| Unsat-core extraction (`--explain`), surface contributing spec spans on `unsat` diagnostics | shipped (Z3 always; Alloy when `minisat.prover` is bundled) |
| Native Alloy CLI cross-check job in CI | shipped |
Expand Down
10 changes: 9 additions & 1 deletion docs/content/docs/spec-language.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -180,7 +180,7 @@ directly into Python and SMT-LIB: `len`, `dom`, `ran`, `max`, `min`, `now`, `day

## Structural lints

`spec-to-rest check <file>.spec` runs six solver-free structural lints over the IR after parsing
`spec-to-rest check <file>.spec` runs seven solver-free structural lints over the IR after parsing
succeeds. Each diagnostic carries a stable code; warnings allow exit `0`, errors cause exit `1`.

| Code | Level | What it catches |
Expand All @@ -191,6 +191,7 @@ succeeds. Each diagnostic carries a stable code; warnings allow exit `0`, errors
| L04 | warning | Two operations share input/output signature **and** have equivalent `requires` |
| L05 | warning | `entity` declared but never referenced in state, operations, invariants, or types |
| L06 | error | Mutually-recursive predicates / functions; verifier inlining would diverge |
| L07 | warning | Operation declares outputs but its success status resolves to `204 No Content`, so they never reach the response |

L01 is intentionally narrow: it only fires on literals whose class admits no operator overload
(e.g., `flag and 5`, `count + true`, `count > true`). The DSL uses `+`/`-` for set/map union and
Expand All @@ -205,6 +206,13 @@ caller picks the operation by name; ambiguity only matters at the dispatch layer
overlap check on `requires_A and requires_B` is candidate work for `verify` and is tracked
separately.

L07 resolves the success status the same way the convention engine does: a per-operation
`http_status_success` override wins, otherwise the default follows from the resolved HTTP method
and operation kind (DELETE and delete-like operations default to `204`). A single `Bool` output is
exempt because that is the designed delete pattern: the flag selects 204-versus-404 and never
becomes a response body. For anything else, set `<Op>.http_status_success = 200` in the
`conventions` block.

## Test-generation coverage

`spec-to-rest compile` (test emission is on by default; opt out with `--no-tests`) walks
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,13 @@ class ArchitectureTest extends CatsEffectSuite:
.layer("bench").definedBy("specrest.bench..")
.layer("cli").definedBy("specrest.cli..")
.whereLayer("parser").mayOnlyBeAccessedByLayers("bench", "cli")
.whereLayer("convention").mayOnlyBeAccessedByLayers("profile", "codegen", "testgen", "cli")
.whereLayer("convention").mayOnlyBeAccessedByLayers(
"profile",
"codegen",
"testgen",
"lint",
"cli"
)
.whereLayer("dafny").mayOnlyBeAccessedByLayers("synth", "cli")
.whereLayer("profile").mayOnlyBeAccessedByLayers("codegen", "testgen", "cli")
.whereLayer("verify").mayOnlyBeAccessedByLayers("bench", "cli")
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,8 +25,16 @@ object AdminRouter:
val seeds = ScalarState
.fieldsWithSeeds(ir)
.map((sf, seed) => s"${ScalarState.columnName(stfName(sf))}=$seed")
.mkString(", ")
s"\n await session.execute(sa_update(ServiceState).values($seeds))"
val single =
s" await session.execute(sa_update(ServiceState).values(${seeds.mkString(", ")}))"
val stmt =
if single.length <= 100 then single
else
(" await session.execute(" ::
" sa_update(ServiceState).values(" ::
seeds.map(s => s" $s,") :::
List(" )", " )")).mkString("\n")
s"\n$stmt"
else ""
val deleteStatements =
if entities.isEmpty && !hasScalars then " pass"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -476,10 +476,11 @@ object EmitPython:
private def enumAliasDef(ir: ServiceIRFull, typeName: String): Option[String] =
svcEnums(ir)
.find(e => enumNameFull(e) == typeName)
.map(e =>
enumAliasName(typeName) + " = " +
enumValuesFull(e).map(v => s"\"$v\"").mkString("Literal[", ", ", "]")
)
.map: e =>
val values = enumValuesFull(e).map(v => s"\"$v\"").mkString(", ")
val single = s"${enumAliasName(typeName)} = Literal[$values]"
if single.length <= 100 then single
else s"${enumAliasName(typeName)} = Literal[\n $values\n]"

private def schemaInputField(
f: ProfiledField,
Expand Down
10 changes: 10 additions & 0 deletions modules/codegen/src/test/scala/specrest/codegen/EmitTest.scala
Original file line number Diff line number Diff line change
Expand Up @@ -634,6 +634,16 @@ class EmitTest extends CatsEffectSuite:
s"url_mapping schema should not import SecretStr; got:\n$schema"
)

test("wide enum Literal alias wraps under the ruff line limit"):
SpecFixtures.loadProfiled("ecommerce").map: profiled =>
val files = Emit.emitProject(profiled).map(f => f.path -> f.content).toMap
val schema = files("app/schemas/order.py")
val wrapped =
"""OrderStatusValue = Literal[
| "DRAFT", "PLACED", "PAID", "SHIPPED", "DELIVERED", "CANCELLED", "RETURNED"
|]""".stripMargin
assert(schema.contains(wrapped), s"expected wrapped Literal alias; got:\n$schema")

test(
"ci.yml renders GitHub Actions ${{ ... }} expressions literally (not Handlebars-substituted)"
):
Expand Down
49 changes: 49 additions & 0 deletions modules/lint/src/main/scala/specrest/lint/DroppedOutputs.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
package specrest.lint

import specrest.convention.Classify
import specrest.convention.Path
import specrest.ir.generated.SpecRestGenerated.*

object DroppedOutputs extends LintPass:
val code = "L07"

def run(ir: service_ir): List[LintDiagnostic] = ir match
case full: ServiceIRFull => svcOperations(full).flatMap(check(_, full))

private def check(op: operation_decl, ir: ServiceIRFull): List[LintDiagnostic] =
val outputs = operOutputs(op)
if outputs.isEmpty || isSingleBoolFlag(outputs) then Nil
else
val classification = Classify.classifyOperation(op, ir)
val conv = svcConventions(ir)
val opName = classificationOperationName(classification)
val method = resolveMethod(
Path.getConvention(conv, opName, "http_method").flatMap(parseHttpMethod),
classificationMethod(classification)
)
val status = resolveStatus(
Path.getConvention(conv, opName, "http_status_success"),
method,
classificationKind(classification)
)
if status == "204" then
val dropped = outputs.map(p => s"'${prmName(p)}'").mkString(", ")
List(
LintDiagnostic(
code,
LintLevel.Warning,
s"operation '$opName' resolves to HTTP 204 No Content, so its declared outputs ($dropped) never reach the response; set '$opName.http_status_success = 200' in the conventions block",
operSpan(op)
)
)
else Nil

// A lone Bool output on a 204 route is the designed delete pattern: the flag
// selects 204-vs-404 and is never a response body.
private def isSingleBoolFlag(outputs: List[param_decl]): Boolean =
outputs match
case p :: Nil =>
prmType(p) match
case NamedTypeF("Bool", _) => true
case _ => false
case _ => false
3 changes: 2 additions & 1 deletion modules/lint/src/main/scala/specrest/lint/Lint.scala
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,8 @@ object Lint:
MissingEnsures,
OperationOverlap,
UnusedEntity,
CircularPredicate
CircularPredicate,
DroppedOutputs
)

def run(ir: service_ir): List[LintDiagnostic] =
Expand Down
72 changes: 72 additions & 0 deletions modules/lint/src/test/scala/specrest/lint/DroppedOutputsTest.scala
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
package specrest.lint

import munit.CatsEffectSuite
import specrest.ir.generated.SpecRestGenerated.SpanT
import specrest.ir.generated.SpecRestGenerated.less_int
import specrest.lint.testutil.SpecFixtures

class DroppedOutputsTest extends CatsEffectSuite:

private def deleteSpec(output: String, conventions: String): String =
s"""service DropDemo {
| entity Item {
| id: Int
| name: String
| }
|
| state {
| items: Int -> Item
| }
|
| operation DeleteItem {
| input: id: Int
| output: $output
|
| requires:
| id in items
|
| ensures:
| id not in items'
| }
|$conventions}""".stripMargin

private val overrideBlock =
""" conventions {
| DeleteItem.http_status_success = 200
| }
|""".stripMargin

List(
("warns on entity output with defaulted 204", "item: Item", "", 1),
("silent when http_status_success overrides to 200", "item: Item", overrideBlock, 0),
("silent on single Bool flag output with 204", "deleted: Bool", "", 0)
).foreach: (label, output, conventions, expected) =>
test(s"L07 $label"):
SpecFixtures.buildFromSource("DropDemo", deleteSpec(output, conventions)).map: ir =>
val diags = DroppedOutputs.run(ir)
assertEquals(diags.length, expected, diags.map(_.message).toString)

test("L07 names the op, the 204, the dropped outputs, and the override"):
SpecFixtures.buildFromSource("DropDemo", deleteSpec("item: Item", "")).map: ir =>
val diags = DroppedOutputs.run(ir)
assertEquals(diags.length, 1)
val d = diags.head
assertEquals(d.code, "L07")
assertEquals(d.level, LintLevel.Warning)
assert(d.message.contains("DeleteItem"), d.message)
assert(d.message.contains("204"), d.message)
assert(d.message.contains("'item'"), d.message)
assert(d.message.contains("DeleteItem.http_status_success"), d.message)
assert(
d.span.exists { case SpanT(line, _, _, _) => less_int(BigInt(0), line) },
s"expected span, got ${d.span}"
)

test("L07 silent on the all-lints-pass fixture"):
SpecFixtures.loadLintIR("passing").map: ir =>
assertEquals(DroppedOutputs.run(ir), Nil)

List("url_shortener", "todo_list", "auth_service", "ecommerce").foreach: name =>
test(s"L07 silent on fixtures/spec/$name.spec"):
SpecFixtures.loadIR(name).map: ir =>
assertEquals(DroppedOutputs.run(ir), Nil)
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,13 @@ func (r clientResponse) JSON() any {
if len(r.body) == 0 {
return map[string]any{}
}
// UseNumber delivers numbers as json.Number: plain Unmarshal rounds
// integers through float64, conflating adjacent values above 2^53 before
// the runtime's exact int64 comparisons ever see them.
dec := json.NewDecoder(bytes.NewReader(r.body))
dec.UseNumber()
var v any
if err := json.Unmarshal(r.body, &v); err != nil {
if err := dec.Decode(&v); err != nil {
return map[string]any{}
}
return v
Expand Down
Loading
Loading