Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 2 additions & 5 deletions sail-partitioned/benchmark.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,8 @@ echo "Install Python packages"
python3 -m venv myenv
source myenv/bin/activate
pip install --upgrade setuptools wheel
pip install --no-cache-dir "pysail>=0.4.2,<0.5.0"
pip install "pyspark-client==4.0.0" \
"protobuf==5.28.3" \
"grpcio==1.71.2" \
"grpcio-status==1.71.2" \
pip install --no-cache-dir "pysail>=0.4.6,<0.6.0"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two questions

  1. Can the script please fix the pysail version? Right now, it installs some unspecified version between 0.4.6 and 0.6. As far as I see, the latest version in 0.4.6. With the current approach, future benchmark runs may yield different results than today's benchmark runs.

  2. Looking quickly at the sail changelog, I couldn't find any changes related to performance compared to the previous update to 0.4.2. Should we wait with a ClickBench PR until the next major release (0.5.x)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rschu1ze

  1. My intention was to allow ClickBench to always use the latest patch version. Sail follows semantic versioning (MAJOR.MINOR.PATCH) with frequent patch releases roughly every two weeks, while minor releases happen about every two months, with 0.5 expected next week. Rather than opening a new ClickBench PR for every patch release, I thought it would be more practical to update ClickBench only on minor releases.
  2. Yeah forsure, happy to wait for the next minor release. Since 0.5 is coming next week, that was the motivation for preparing this PR 🙂

pip install "pyspark-client==4.1.1" \
pandas \
psutil

Expand Down
7 changes: 2 additions & 5 deletions sail/benchmark.sh
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,8 @@ echo "Install Python packages"
python3 -m venv myenv
source myenv/bin/activate
pip install --upgrade setuptools wheel
pip install --no-cache-dir "pysail>=0.4.2,<0.5.0"
pip install "pyspark-client==4.0.0" \
"protobuf==5.28.3" \
"grpcio==1.71.2" \
"grpcio-status==1.71.2" \
pip install --no-cache-dir "pysail>=0.4.6,<0.6.0"
pip install "pyspark-client==4.1.1" \
pandas \
psutil

Expand Down
Loading