Skip to content

Add the de-facto required methods to the IFileSystem Protocol - #10

Open
ninowalker wants to merge 1 commit into
dbreunig:mainfrom
ninowalker:fix/ifilesystem-protocol-gap
Open

Add the de-facto required methods to the IFileSystem Protocol#10
ninowalker wants to merge 1 commit into
dbreunig:mainfrom
ninowalker:fix/ifilesystem-protocol-gap

Conversation

@ninowalker

Copy link
Copy Markdown

IFileSystem declares 17 methods, but the commands call four more without a hasattr guard:

method called by
lstat commands/readlink/readlink.py:55, interpreter/builtins/test.py:381 (test -L)
cp commands/cp/cp.py:136
mv commands/mv/mv.py:131
link commands/ln/ln.py:112 (ln without -s)

So a backend that implements exactly the Protocol type-checks, passes a smoke test, and then raises AttributeError the first time a script runs cp, mv, ln, readlink, or test -L. Those call sites catch only OSError subclasses, so it escapes bash.exec() as a Python exception rather than becoming a nonzero exit.

I ran into it writing a custom backend over a remote store — I worked from the Protocol, everything looked complete, and cp blew up at runtime.

mypy already knew. On main, mypy src/just_bash/ reports exactly these:

commands/readlink/readlink.py:55: error: "IFileSystem" has no attribute "lstat"; maybe "stat"?  [attr-defined]
commands/mv/mv.py:131: error: "IFileSystem" has no attribute "mv"  [attr-defined]
commands/ln/ln.py:112: error: "IFileSystem" has no attribute "link"  [attr-defined]
commands/cp/cp.py:136: error: "IFileSystem" has no attribute "cp"  [attr-defined]
interpreter/builtins/test.py:381: error: "IFileSystem" has no attribute "lstat"; maybe "stat"?  [attr-defined]

This PR removes all five (total goes 1003 -> 997).

The change

Four declarations added to IFileSystem, signatures copied from the backends. All four of InMemoryFs, ReadWriteFs, OverlayFs and MountableFs already agree exactly on each one, so there was nothing to reconcile:

async def cp(self, src: str, dest: str, recursive: bool = False) -> None: ...
async def mv(self, src: str, dest: str) -> None: ...
async def lstat(self, path: str) -> "FsStat": ...
async def link(self, existing_path: str, new_path: str) -> None: ...

No runtime behaviour changes — the Protocol is not runtime_checkable and nothing subclasses it.

Deliberately left out: readdir_with_file_types. All four backends have it, but MountableFs guards it with hasattr and falls back to readdir + stat, so it is genuinely optional and declaring it would make it mandatory for existing third-party backends. It also returns DirentEntry, which lives in fs/in_memory_fs.py, so declaring it would make types.py import from fs/ — which imports from types.py. Same reasoning for get_all_paths, also hasattr-guarded.

The test

tests/test_fs/test_ifilesystem_conformance.py reads the method list off the Protocol by introspection and asserts:

  1. the four unguarded call sites are declared on IFileSystem
  2. every built-in backend implements everything the Protocol declares, with async-ness matching

12 cases, no fixtures. It would have caught #6 (utimes missing from ReadWriteFs) as well as this one.

It does surface one pre-existing gap I have left alone: realpath is declared on IFileSystem but only InMemoryFs implements it, so pwd -P and readlink -f raise AttributeError on the other three backends. Rather than guess at the right semantics for OverlayFs (symlink handling there looks security-relevant) I listed it in a commented KNOWN_GAPS set, so the test guards everything else without hiding it. Happy to send a separate PR for that, or fold it into this one if you would rather.

One other thing worth knowing: CI runs pytest tests/test_commands/ tests/test_network.py, so tests/test_fs/ — this new file included — is not currently exercised there. I did not want to touch the workflow in this PR.

Testing

  • pytest tests/test_commands/ tests/test_network.py (the CI command): 2314 passed, 1 skipped
  • everything except tests/spec_tests and tests/comparison_tests: 2925 passed, 3 skipped
  • tests/spec_tests + tests/comparison_tests: identical results to main

Fixes #8

cp, mv, link and lstat are called by the cp/mv/ln/readlink commands and
the test builtin without a hasattr guard, so a backend implementing
exactly the Protocol type-checks and then raises AttributeError at
runtime. Declare them, with signatures matching all four built-in
backends.

Add a conformance test that reads the method list off the Protocol and
checks every built-in backend implements it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

IFileSystem Protocol is narrower than what the commands require

1 participant