Skip to content

feat: add file upload integration tests and dependencies#36

Open
AndriiS-DevBrother wants to merge 4 commits into
mainfrom
andriis/large-files-tests
Open

feat: add file upload integration tests and dependencies#36
AndriiS-DevBrother wants to merge 4 commits into
mainfrom
andriis/large-files-tests

Conversation

@AndriiS-DevBrother

@AndriiS-DevBrother AndriiS-DevBrother commented Mar 18, 2026

Copy link
Copy Markdown
Collaborator

Summary

Add end-to-end file upload/download integration tests that verify the full cross-node pipeline across real Docker nodes.

New test suite — test_file_upload.py

Cross-node replication E2E test covering:

  • Upload a file to the owner node
  • Replicate to the readonly observer
  • Download from the readonly node and verify content integrity
  • Owner-delete and verify the file is gone
  • Unauthorized-delete attempt and verify the file remains intact

Uses the session-scoped shard fixture with heartbeat-driven block creation.

Note: Logic already covered by Scala unit tests (SyntheticDeploySpec, FileUploadCostSpec, DownloadFileAPITest, FileSystemProcessSpec, FileReplicationSpec, etc.) is intentionally not duplicated — this suite focuses on the cross-node E2E flow only.

Dependencies

@NazarY-DevBrother NazarY-DevBrother left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LG2M, 2 small comments:

)


def _download_file(node: Node, file_hash: str, offset: int = 0,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test_file_upload.py:_download_file — This helper accesses client._deploy_stub.downloadFile(...) directly, bypassing the public download_file() method from pyf1r3fly. Consider extending download_file() in PR #4 to accept timeout and grpc_options, so the integration test can use the public API instead of reaching into private internals.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done! Updated the integration test to securely use the newly expanded client.download_file() SDK method with the timeout parameter from pyf1r3fly PR #4, so we no longer bypass the public API here. I've also updated the poetry lock to capture that.

Comment thread services.yml
f1r3node:
url: git@github.com:F1R3FLY-io/f1r3node.git
branch: dev
branch: feature/streaming-file-upload-registry

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

f1r3node branch is changed from dev to feature/streaming-file-upload-registry. This is fine for testing, but should be reverted to dev (or the target merge branch) before merging this PR, otherwise CI will break when the feature branch is deleted.

@AndriiS-DevBrother
AndriiS-DevBrother force-pushed the andriis/large-files-tests branch from f1b2096 to c6ea15f Compare March 22, 2026 09:43

@spreston8 spreston8 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_wait_for_finalization (line 118) calls node.is_finalized(block_hash) but this method does not exist on the Node class. Only node.last_finalized_block() is available (returns the LFB as BlockInfo).

Options to fix:

  1. Add an is_finalized(block_hash) method to Node wrapping the gRPC isFinalized call
  2. Rewrite _wait_for_finalization to poll last_finalized_block() and check if blockNumber >= target_block_number

This will fail at runtime as-is.

Co-Authored-By: Claude noreply@anthropic.com

@spreston8
spreston8 dismissed their stale review March 25, 2026 03:05

Reposting as comment instead of request-changes

@spreston8 spreston8 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_wait_for_finalization (line 118) calls node.is_finalized(block_hash) but this method does not exist on the Node class. Only node.last_finalized_block() is available (returns the LFB as BlockInfo).

Options to fix:

  1. Add an is_finalized(block_hash) method to Node wrapping the gRPC isFinalized call
  2. Rewrite _wait_for_finalization to poll last_finalized_block() and check if blockNumber >= target_block_number

This will fail at runtime as-is.

Co-Authored-By: Claude noreply@anthropic.com

@AndriiS-DevBrother

Copy link
Copy Markdown
Collaborator Author

_wait_for_finalization (line 118) calls node.is_finalized(block_hash) but this method does not exist on the Node class. > Only node.last_finalized_block() is available (returns the LFB as BlockInfo).

oops, forgot to commit this change 😕

@AndriiS-DevBrother
AndriiS-DevBrother removed the request for review from nashef March 26, 2026 12:06
Oracle Public Cloud User and others added 4 commits March 26, 2026 12:43
- Add test_file_upload.py with cross-node replication tests
- Update poetry.lock and pyproject.toml with new dependencies
- Update services.yml configuration
@AndriiS-DevBrother
AndriiS-DevBrother force-pushed the andriis/large-files-tests branch from a6164ed to 3ab45c8 Compare March 27, 2026 21:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants