Skip to content

Update the benchmark to load datasets from multiple S3 buckets when None is specified #607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
R-Palazzo wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Update the benchmark to load datasets from multiple S3 buckets when None is specified #607

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
828570f
load dataset only once
R-Palazzo May 25, 2026
cba80aa
use downlad_demo from sdv
R-Palazzo May 25, 2026
3803c19
update _resolve_dataset
R-Palazzo May 25, 2026
d040382
fix lint
R-Palazzo May 25, 2026
2118c9a
cleaning
R-Palazzo May 25, 2026
b3481aa
add validation
R-Palazzo May 26, 2026
a62376b
move dataset loading to execution
R-Palazzo May 27, 2026
f0eedea
fix tests
R-Palazzo May 27, 2026
c349d70
rename _get_dataset_bucket_mapping -> dataset_to_bucket
R-Palazzo May 28, 2026
2884e39
add tests
R-Palazzo May 28, 2026
5555441
stop compressing job_arg_list
R-Palazzo May 28, 2026
e3c2e90
undo pip install command
R-Palazzo May 28, 2026
b97fa0d
remove dataset_name from JobArgs
R-Palazzo May 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/integration.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,9 @@ jobs:
python -m pip install --upgrade pip
python -m pip install --no-cache-dir invoke .[test]
- name: Run integration tests
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
run: invoke integration
- if: matrix.os == 'ubuntu-latest' && matrix.python-version == 3.14
name: Upload integration codecov report
Expand Down
3 changes: 3 additions & 0 deletions .github/workflows/minimum.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,4 +39,7 @@ jobs:
python -m pip install --no-cache-dir invoke .[test]

- name: Test with minimum versions
env:
AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
run: invoke minimum
6 changes: 2 additions & 4 deletions pyproject.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,10 +64,8 @@ dependencies = [
'XlsxWriter>=1.2.8',
"rdt>=1.18.2;python_version<'3.14'",
"rdt>=1.20.0;python_version>='3.14'",
"sdmetrics>=0.21.0;python_version<'3.14'",
"sdmetrics>=0.26.0;python_version>='3.14'",
"sdv>=1.21.0;python_version<'3.14'",
"sdv>=1.33.0;python_version>='3.14'",
"sdmetrics>=0.28.0",
"sdv @ git+https://github.com/sdv-dev/SDV.git@main",
]

[project.urls]
Expand Down
4 changes: 1 addition & 3 deletions sdgym/_benchmark_launcher/utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,9 +285,7 @@ def _build_instance_artifact_filepaths(
return (
_build_s3_uri(output_destination, f'{artifact_key_prefix}/{metainfo_name}.yaml'),
_build_s3_uri(output_destination, f'{artifact_key_prefix}/{results_name}.csv'),
_build_s3_uri(
output_destination, f'{modality_prefix}/job_args_list_{metainfo_name}.pkl.gz'
),
_build_s3_uri(output_destination, f'{modality_prefix}/job_args_list_{metainfo_name}.pkl'),
)


Expand Down
Loading
Loading