Use pinned pyspssio fork for file labels

Author: TonisOrmisson

README.md

@@ -39,8 +39,8 @@ openstatspec export --database-url postgresql+psycopg://... --dataset-id respons
 
 ## 0.1.0 support status
 
-The adapter requires pyspssio 0.5.1 or a compatible later release as its sole
-SPSS engine. There is no
+The adapter requires the pinned TonisOrmisson/pyspssio fork declared in
+pyproject.toml as its sole SPSS engine. There is no
 fallback reader or writer. It supports unencrypted SAV and ZSAV import and
 SAV/ZSAV export for the semantics exposed by that engine. SQLite is the local
 reference path.
@@ -60,7 +60,7 @@ intentional lossy export auditable.
 The matrix is also available to Python callers as
 `openstatspec.capability_matrix()`. It distinguishes supported semantics from
 unobservable and fail-closed paths; see the SAV profile for the exact
-`pyspssio` 0.5.1 boundary.
+the pinned pyspssio fork boundary.
 
 See [the SAV profile](docs/sav-profile.md) for feature boundaries and
 [release readiness](docs/release-readiness.md) for the pre-tag checklist.

THIRD_PARTY_NOTICES.md

@@ -5,8 +5,8 @@ SPSS engine is a separate third-party distribution:
 
 | Component | Version | Applicable terms |
 | --- | --- | --- |
-| pyspssio | 0.5.1 | MIT for the Python wrapper, except for its IBM I/O Module files |
-| IBM I/O Modules for IBM SPSS Statistics Data Files | 28.0.1.0, bundled by pyspssio | IBM International License Agreement for Non-Warranted Programs and the accompanying License Information / REDIST files |
+| pyspssio | TonisOrmisson/pyspssio pinned commit 0b3f879 | MIT for the Python wrapper, except for its IBM I/O Module files |
+| IBM I/O Modules for IBM SPSS Statistics Data Files | bundled by the pinned pyspssio fork | IBM International License Agreement for Non-Warranted Programs and the accompanying License Information / REDIST files |
 
 pyspssio ships the IBM licence, redistribution list, and third-party notices
 inside its installed distribution, normally under pyspssio/spssio/license/.
@@ -15,7 +15,7 @@ Apache-2.0 licence.
 
 ## Distribution responsibility
 
-OpenStatSpec requires pyspssio==0.5.1; it has no alternate SPSS engine. This
+OpenStatSpec requires the pinned pyspssio fork declared in pyproject.toml; it has no alternate SPSS engine. This
 repository does not vendor, alter, or publish the IBM binaries. Anyone who
 distributes an application bundle containing the installed engine must review
 and comply with the IBM materials supplied with that engine. In particular, the

docs/pyspssio-spike.md

@@ -2,8 +2,8 @@
 
 **Status:** selected as the required, sole Python SPSS engine.
 
-OpenStatSpec Python uses [pyspssio](https://pypi.org/project/pyspssio/) 0.5.1
-for both SAV and ZSAV import and export. There is no alternate SPSS engine.
+OpenStatSpec Python uses the pinned TonisOrmisson/pyspssio fork, based on
+[pyspssio](https://pypi.org/project/pyspssio/), for both SAV and ZSAV import and export. There is no alternate SPSS engine.
 
 ## Probe result
 
@@ -28,7 +28,7 @@ that require explicit tests or upstream work include:
 
 1. variable-set round trips;
 2. independent print and write formats;
-3. file label and document text;
+3. ordered document text;
 4. legacy code-page files and long-string edge cases.
 
 Until a feature is proved, the adapter must report it as unsupported rather

docs/release-readiness.md

@@ -29,7 +29,7 @@ relevant diagnostic codes through `allow_loss` or repeats `--allow-loss CODE`
 in the CLI.
 
 The concrete pyspssio boundaries are documented in the [SAV profile](sav-profile.md):
-file labels/documents are unobservable, legacy compatible names are fail-closed
+document text is unobservable; file labels are preserved by the pinned fork, legacy compatible names are fail-closed
 on export, and only UTF-8 has a source-encoding fidelity contract. Separate
 print/write formats, variable sets, multiple-response sets, alignment, scalar
 attributes, and ordered custom-attribute arrays have verified paths. A caller

docs/sav-profile.md

@@ -7,8 +7,8 @@ chunks, or creates a cell/EAV representation.
 
 ## SPSS engine
 
-OpenStatSpec Python requires pyspssio 0.5.1 as its sole SPSS engine. There is
-no fallback reader or writer. The engine reads unencrypted .sav and .zsav
+OpenStatSpec Python requires the pinned TonisOrmisson/pyspssio fork declared in
+pyproject.toml as its sole SPSS engine. There is no fallback reader or writer. The engine reads unencrypted .sav and .zsav
 files, and writes both formats through the same implementation.
 
 ## Verified SQL profiles
@@ -35,9 +35,10 @@ The `openstatspec capabilities` command and the
 of this boundary. They deliberately distinguish a supported feature from a
 feature that the underlying engine cannot observe or write faithfully.
 
-| SPSS semantic | pyspssio 0.5.1 status | Behaviour |
+| SPSS semantic | Pinned pyspssio status | Behaviour |
 | --- | --- | --- |
-| File label and document text | Unobservable | The public engine API exposes neither. Import records `file-label-and-documents-unobservable`; export requires that audited loss to be accepted. |
+| File label | Supported through the pinned fork | The adapter persists it in the dataset catalog and writes it through the IBM I/O identifier-string API. |
+| Ordered document text | Unobservable | The engine exposes file-to-file document copying, but not reading or creating document text. Import records documents-unobservable; export requires that audited loss to be accepted. |
 | Print and write formats independently | Supported as raw IBM I/O tuples | The adapter stores both tuples separately and writes them without collapsing either value. |
 | Variable sets | Supported through raw IBM I/O | The adapter stores source sets in the extension catalog and writes them through the raw dictionary setter. Invalid target definitions fail before data are written. |
 | Legacy compatible variable names | Fail-closed on export | A compatible name that differs from the long source name is retained in the catalog, but cannot be set through the public writer. Export requires `compatible-variable-name-not-exportable`. |

pyproject.toml

@@ -10,7 +10,7 @@ readme = "README.md"
 requires-python = ">=3.11"
 license = {text = "Apache-2.0"}
 authors = [{name = "OpenStatSpec"}]
-dependencies = ["SQLAlchemy>=2.0", "pandas>=2.2", "pyspssio>=0.5.1"]
+dependencies = ["SQLAlchemy>=2.0", "pandas>=2.2", "pyspssio @ git+https://github.com/TonisOrmisson/pyspssio.git@0b3f879"]
 
 [project.optional-dependencies]
 dev = ["pytest>=8"]
@@ -24,5 +24,8 @@ testpaths = ["tests"]
 addopts = "-q"
 markers = ["services: requires a configured PostgreSQL or MySQL/MariaDB service"]
 
+[tool.hatch.metadata]
+allow-direct-references = true
+
 [tool.hatch.build.targets.wheel]
 packages = ["src/openstatspec"]

src/openstatspec/api.py

@@ -25,8 +25,8 @@ def capability_matrix() -> Mapping[str, Any]:
             "values": "supported", "variable_labels": "supported",
             "value_labels": "supported",
             "print_format": "supported",
-            "write_format": "unobservable",
-            "file_label": "unobservable",
+            "write_format": "supported",
+            "file_label": "supported",
             "documents": "unobservable",
             "source_encoding": {
                 "utf8": "supported",
@@ -35,11 +35,11 @@ def capability_matrix() -> Mapping[str, Any]:
             "measurement_level": "supported", "user_missing_rules": "supported",
             "multiple_response_sets": "supported",
             "variable_alignment": "supported",
-            "variable_sets": "fail-closed-on-export",
+            "variable_sets": "supported",
             "compatible_variable_names": "fail-closed-on-export",
             "custom_attributes": {
                 "scalar_values": "supported",
-                "ordered_value_arrays": "fail-closed",
+                "ordered_value_arrays": "supported",
             },
             "variable_role": "supported",
         },

src/openstatspec/spss/sav.py

@@ -178,7 +178,7 @@ def inspect_sav(source: str | Path) -> dict[str, Any]:
         "source_name": source_path.name,
         "source_sha256": _sha256(source_path),
         "source_encoding": metadata.get("encoding"),
-        "file_label": "",
+        "file_label": str(metadata.get("file_label") or ""),
         "documents": [],
         "file_attributes": dict(metadata.get("file_attributes") or {}),
         "case_weight_variable": metadata.get("case_weight_var") or None,
@@ -209,7 +209,7 @@ def import_sav_dataset(*, source: str | Path, database_url: str, dataset_id: str
         source_format=source_path.suffix[1:].upper(),
         rows=_rows(frame, variables),
         variables=variables,
-        file_label="",  # pyspssio 0.5.x has no public file-label API.
+        file_label=str(metadata.get("file_label") or ""),
         source_encoding=metadata.get("encoding"),
         source_sha256=_sha256(source_path),
         imported_at=datetime.now(UTC).isoformat(),
@@ -289,6 +289,7 @@ def _write_with_dictionary_bridge(
     metadata = _writer_metadata(dataset, variables)
     with pyspssio.Writer(str(destination), mode="w") as writer:
         writer.compression = 2 if destination.suffix.lower() == ".zsav" else 1
+        writer.file_label = str(dataset.get("file_label") or "")
         for variable in variables:
             writer._add_var(  # pylint: disable=protected-access
                 variable["source_name"],
@@ -384,9 +385,9 @@ def _typed_value_labels(variable: dict[str, Any]) -> dict[Any, str]:
 
 def _engine_loss_report(metadata: dict[str, Any]) -> dict[str, dict[str, Any]]:
     events: dict[str, dict[str, Any]] = {
-        "file-label-and-documents-unobservable": {
-            "code": "file-label-and-documents-unobservable",
-            "detail": "pyspssio 0.5.x exposes neither SAV file label nor document text through its public API.",
+        "documents-unobservable": {
+            "code": "documents-unobservable",
+            "detail": "pyspssio does not expose SAV document text through its public API; the file label is preserved separately.",
         },
     }
     variable_sets = metadata.get("_var_sets")

tests/conformance.py

@@ -86,10 +86,11 @@ def write_supported_semantics_fixture(destination: str | Path) -> dict[str, Any]
         "var_attributes": {"code": {"Origin": "fixture"}},
         "mrsets": {"$responses": {"label": "Responses", "counted_value": 1, "variable_list": ["resp_a", "resp_b"]}},
         "file_attributes": {"Fixture": "OpenStatSpec"},
+        "file_label": "OpenStatSpec supported fixture",
         "case_weight_var": "code",
     }
     pyspssio.write_sav(str(destination), frame, metadata=metadata)
-    return {"long_text": long_text}
+    return {"long_text": long_text, "file_label": metadata["file_label"]}
 
 
 def sys_float_max() -> float:
@@ -126,7 +127,7 @@ def compare_sav_semantics(source: str | Path, exported: str | Path) -> dict[str,
     except AssertionError:
         failures.append("values-or-case-order")
     for attribute in (
-        "encoding", "case_weight_var", "file_attributes", "mrsets", "var_types", "var_formats",
+        "encoding", "file_label", "case_weight_var", "file_attributes", "mrsets", "var_types", "var_formats",
         "var_labels", "var_alignments", "var_column_widths", "var_measure_levels", "var_roles",
         "var_value_labels", "var_attributes",
     ):

tests/test_attribute_catalog.py

@@ -10,7 +10,7 @@
 from openstatspec.sql.wide import create_wide_dataset, read_wide_dataset
 
 
-_REQUIRED_ENGINE_LOSS = ["file-label-and-documents-unobservable"]
+_REQUIRED_ENGINE_LOSS = ["documents-unobservable"]
 
 
 @pytest.mark.parametrize("suffix", [".sav", ".zsav"])