dependency management workflow

[dongYoun2]

• Migrated to modern pyproject.toml - Dependencies are now declared in pyproject.toml with pip-compile generating pinned requirements.lock lockfile
• Added Dependabot Configuration + auto-regeneration workflow - GitHub Actions automatically updates requirements.lock when Dependabot modifies pyproject.toml
• Enforced lockfile constraints - Dev container and Dockerfile now install with -c requirements.lock for reproducible builds when installing additional dependencies for e2e testing
• Simplified setup.py - Reduced to only handling C++ extension; all metadata/dependencies moved to pyproject.toml
• Add dependency management workflow doc to README.md

[github-actions]

ℹ️ No lockfile changes needed

requirements.txt is already up-to-date with pyproject.toml.

[github-actions]

ℹ️ No lockfile changes needed

requirements.txt is already up-to-date with pyproject.toml.

[twom]

I have never done this, but the documentation. seems to suggest that depandabot supports pyproject.toml files and will also upate the requirements.txt

https://docs.github.com/en/code-security/dependabot/ecosystems-supported-by-dependabot/supported-ecosystems-and-repositories#pip-and-pip-compile

so not sure if you need this?

[dongYoun2]

Hi Tom, oh yes, it seems dependabot looks at both pyproject.toml and requirements.txt. In this case, would it be better to change requirements.txt -> requirements.lock so that dependabot only looks at pyproject.toml?

What I thought was specifying only direct dependencies that we use in Designer in pyproject.toml, and use pip-compile to generate requirements.txt file, which includes all direct and transitive dependencies with pinned version for full reproducibility.

My concern about dependabot updating the lock file (requirements.txt previously) was in situation like when dependabot first updates a transitive dependency in requirements.txt, and later we need to add a new direct dependency in pyproject.toml, which requires us to manually regenerate the updated requirements.txt using pip-compile. In this case, I believe we cannot guarantee that the transitive dependency updated by dependabot remains the same.

However, I’m not sure if this is a good practice for python dependency management workflow.

[twom]

For this project I don't think it will be a problem. When you run pip-compile later, with new direct dependencies you would expect it to pin the newest versions of your dependencies.

I think it is a good workflow to get started.

[dongYoun2]

I see, then I will make sure that dependabot only updates (direct) dependencies specified in pyproject.toml by changing requirements.txt to requirements.lock. So the workflow would be:

• dependabot updates dependencies in pyproject.toml: then, the github action runs pip-compile to regenerate requirements.lock (github action runs only when pyproject.toml changes)
• we manually update dependencies or add new dependencies in pyproject.toml: then, we also have to manually regenerate requirements.lock file with pip-compile and commit them

[github-actions]

ℹ️ No lockfile changes needed

requirements.lock is already up-to-date with pyproject.toml.