Skip to content

chore: upgrade protobuf to 5.x.x#210

Closed
rishhavv wants to merge 1 commit intonumaproj:mainfrom
rishhavv:protobuf-upgrade-5.26.1
Closed

chore: upgrade protobuf to 5.x.x#210
rishhavv wants to merge 1 commit intonumaproj:mainfrom
rishhavv:protobuf-upgrade-5.26.1

Conversation

@rishhavv
Copy link

@rishhavv rishhavv commented Feb 3, 2025

Fixes #204

Update protobuf dependency to support version 5.x

Description

This PR updates the protobuf dependency in pyproject.toml from ">=3.20,<5.0" to "^5.26.1". This change allows pynumaflow to leverage the latest features and improvements available in protobuf 5.x.

Changes Made

  • Updated protobuf dependency constraint in pyproject.toml from ">=3.20,<5.0" to "^5.26.1"

Benefits

  • Access to performance improvements and bug fixes introduced in protobuf 5.x
  • Better compatibility with modern dependency requirements
  • Ensures pynumaflow stays up-to-date with the latest stable protobuf release

Testing

  • Verified compatibility by running the existing test suite
  • No breaking changes or performance regressions observed
  • All functionality remains intact with the updated dependency

Related Issue

Closes #204

Checklist

  • Updated dependency in pyproject.toml
  • Tested with protobuf 5.26.1
  • All tests passing
  • No breaking changes identified
  • Tested with UDFs
  • Tested with examples in the repo

Signed-off-by: rishhavv <rishav.katoch17300@gmail.com>
@rishhavv
Copy link
Author

rishhavv commented Feb 3, 2025

Fixes issue number #204

@kohlisid
Copy link
Contributor

kohlisid commented Feb 3, 2025

Hey @rishhavv
Thanks for testing this out!
Did you get a chance to test a numaflow deployment running with this updated UDF?

@codecov
Copy link

codecov bot commented Feb 3, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 94.13%. Comparing base (e6e336d) to head (79a9c8c).
Report is 3 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #210      +/-   ##
==========================================
- Coverage   94.17%   94.13%   -0.05%     
==========================================
  Files          55       55              
  Lines        2317     2317              
  Branches      119      119              
==========================================
- Hits         2182     2181       -1     
- Misses         98       99       +1     
  Partials       37       37              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

google-cloud = "^0.34.0"
google-api-core = "^2.11.0"
protobuf = ">=3.20,<5.0"
protobuf = "^5.26.1"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
protobuf = "^5.26.1"
protobuf = ">=3.20,<6.0"

let's not hardcode to 5.26.1, rather put some bound values? will the above work?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vigith this does not mean it is hard-coded. This means that it will take any version >=5.26.1 and <6.0
Nevertheless, I would recommend having a stricter constraint.

Suggested change
protobuf = "^5.26.1"
protobuf = "~5.26.1"

which translates to >=5.26.1 and <5.27

Copy link
Member

@vigith vigith Feb 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's put >=3.20,<5.27 then? earlier we were saying we support only a lower bound of >3.20, is it not longer valid?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, but assuming that things would still work with previous versions of protobuf (<5.0) that we support.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since protobuf follows semantic versioning, I am assuming that they won't break anything till they reach 6.0. Now the question is, is 3.x and 4.x still compatible with 5.x, if not we have to bound ourselves to strictly 5.x.

Since our tests were working with earlier version, i am gonna assume that we are good with 3.x and 4.x.

@kohlisid could you please verify these "assumptions"? 🙂

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vigith I did test with 5.x version - everything worked fine out of the box.
I can verify once with 3.x and 4.x once.

But do we know why we had a limit in the first place?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, I think it's better to set it as >=3.20,<6.0, breaking changes are not released in minor upgrades and knowing that protobuf is a library used by millions - it's even more unlikely that any breaking change will be released with minor upgrade.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC during our early version of pynumaflow, we used the 3.2 version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @rishhavv
Were you able to verify the compatibility?

@rishhavv
Copy link
Author

rishhavv commented Feb 8, 2025

Hey @rishhavv Thanks for testing this out! Did you get a chance to test a numaflow deployment running with this updated UDF?

hey @kohlisid, Yes i did test it with multiple UDFs

@vigith
Copy link
Member

vigith commented Mar 15, 2025

completed via #218

@vigith vigith closed this Mar 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Update protobuf Dependency to Support Version >=5.0

4 participants