Skip to content

Update from setup.py to pyproject.toml#164

Merged
jsf9k merged 16 commits into
developfrom
improvement/use-pyproject-toml-file
Nov 20, 2025
Merged

Update from setup.py to pyproject.toml#164
jsf9k merged 16 commits into
developfrom
improvement/use-pyproject-toml-file

Conversation

@jsf9k

@jsf9k jsf9k commented Oct 28, 2025

Copy link
Copy Markdown
Member

🗣 Description

This pull request updates our project packaging configuration from using a setup.py file to using a pyproject.toml file with a setuptools build backend.

See also cisagov/skeleton-generic#235.

💭 Motivation and context

Resolves #163. Resolves #136.

🧪 Testing

All automated tests pass.

✅ Pre-approval checklist

  • This PR has an informative and human-readable title.
  • Changes are limited to a single goal - eschew scope creep!
  • All relevant type-of-change labels have been added.
  • I have read the CONTRIBUTING document.
  • These code changes follow cisagov code standards.
  • All new and existing tests pass.

@jsf9k jsf9k self-assigned this Oct 28, 2025
@jsf9k jsf9k added the improvement This issue or pull request will add or improve functionality, maintainability, or ease of use label Oct 28, 2025
@jsf9k jsf9k moved this to In Progress in Next Kraken Oct 28, 2025
@github-actions github-actions Bot added dependencies Pull requests that update a dependency file python Pull requests that update Python code labels Oct 28, 2025
@jsf9k jsf9k force-pushed the improvement/use-pyproject-toml-file branch from 540a8db to 336618a Compare October 28, 2025 17:26
@jsf9k jsf9k moved this from In progress to Review in progress in Skeleton Maintenance Oct 28, 2025
@jsf9k jsf9k added the kraken 🐙 This pull request is ready to merge during the next Lineage Kraken release label Oct 28, 2025
@github-actions github-actions Bot added the test This issue or pull request adds or otherwise modifies test code label Oct 28, 2025
@jsf9k

jsf9k commented Oct 28, 2025

Copy link
Copy Markdown
Member Author

@mcdonnnj - I'm not sure what kind of version bump to apply here, if any.

@jsf9k

jsf9k commented Oct 28, 2025

Copy link
Copy Markdown
Member Author

The flake8 config could be moved to the pyproject.toml file as well, but that would require the flake8-pyproject Python package. There is also the complication that the flake8 configuration comes from upstream. See cisagov/skeleton-generic#234.

The bandit config is only used when pre-commit runs bandit against the Python test code; otherwise, we use the default configuration for bandit. For completeness, also see cisagov/skeleton-generic#233.

@jsf9k jsf9k marked this pull request as ready for review October 28, 2025 17:59
@jsf9k jsf9k requested review from dav3r and mcdonnnj as code owners October 28, 2025 17:59
Comment thread pyproject.toml Outdated
@jsf9k

jsf9k commented Oct 28, 2025

Copy link
Copy Markdown
Member Author

I realize that we want wheel and pip to be the latest versions, but should we now remove setuptools from setup-env? If so then this would be an upstream change in cisagov/skeleton-generic.

@dav3r dav3r left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks like a solid start to me. Thanks for getting us switched over to pyproject.toml. 🎉

@dav3r dav3r requested a review from Copilot October 29, 2025 19:22

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request migrates the project from a setup.py-based build configuration to a modern pyproject.toml-based configuration using the Hatchling build backend. This modernization consolidates configuration files and follows current Python packaging best practices (PEP 517/518).

  • Migrates from setup.py to pyproject.toml with Hatchling as the build backend
  • Consolidates pytest.ini and .isort.cfg configurations into pyproject.toml
  • Updates GitHub labeler configuration to reference the new configuration file

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
setup.py Removed legacy setup.py file (116 lines)
pytest.ini Removed standalone pytest configuration file
.isort.cfg Removed standalone isort configuration file
pyproject.toml Added comprehensive project configuration including build system, dependencies, tool configurations, and metadata
.github/labeler.yml Updated file references from setup.py, .isort.cfg, and pytest.ini to pyproject.toml

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread pyproject.toml Outdated
@jsf9k

jsf9k commented Nov 5, 2025

Copy link
Copy Markdown
Member Author

@mcdonnnj - Any thoughts here? Are you still researching the current landscape of build backends?

@jsf9k

jsf9k commented Nov 6, 2025

Copy link
Copy Markdown
Member Author

In this PR I moved the isort config to pyproject.toml, but just now I realized that this file is actually inherited from cisagov/skeleton-generic. As a result I have dropped commit 1ae44fe.

Do we want to leave the .isort.cfg file as it is in cisagov/skeleton-generic or create a pyproject.toml in that repo to contain that tool's configuration? I'm for the latter, since that would make things cleaner here.

jsf9k added 3 commits November 6, 2025 11:55
This is because this file now contains configurations for test tools.

Also remove test tool config files that no longer exist.
These may be of use to folks who are editing pyproject.toml for a
descendant of this skeleton repository.
@jsf9k jsf9k force-pushed the improvement/use-pyproject-toml-file branch from 901f748 to 3b8d5fc Compare November 6, 2025 16:55
@dav3r

dav3r commented Nov 6, 2025

Copy link
Copy Markdown
Member

In a way you already do, since .isort.cfg is present. isort only sorts Python imports. That sounds pretty Python-specific to me.

LOL, I forgot about that. In that case, I agree with your plan to replace .isort.cfg with a section in pyproject.toml.

This gets rid of the .bandit.yml file that was being used only against
the test code.
@jsf9k

jsf9k commented Nov 9, 2025

Copy link
Copy Markdown
Member Author

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code. If you insist, I could add a simple setup.py like this as a placeholder:

import setuptools

setuptools.setup()

I don't see the point of that, though.

Also, it appears that hatchling doesn't support native code associated with Python libraries very well. Maybe that's one reason to use setuptools instead. As far as I can tell, setuptools is still the most common build backend and a lot of folks are saying the fact that the documentation appears to promote hatchling instead is misleading. Projects like numpy that have a lot of compiled native code tend to use something like PDM.

I'm kind of of the opinion that until we have native extensions it's a tossup between hatchling and setuptools.

I think this PR would change only slightly if we used setuptools as the build backend instead of hatchling. I'm happy to make those changes if that's where you're leaning.

@jsf9k

jsf9k commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

FYI, here are the necessary changes for moving to setuptools as the build backend:

diff --git a/pyproject.toml b/pyproject.toml
index 370ba01..727434b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,9 +1,9 @@
 # For more information about configuring project metadata for the
-# hatch build backend, please see
-# https://hatch.pypa.io/latest/config/metadata/
+# setuptools build backend, please see
+# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
 [build-system]
-build-backend = "hatchling.build"
-requires = ["hatchling"]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools"]
 
 [project]
 authors = [
@@ -38,11 +38,10 @@ dependencies = [
     "schema",
 ]
 description = "Example Python library"
-dynamic = ["version"]
+dynamic = ["readme", "version"]
 keywords = ["skeleton"]
 license = "CC0-1.0"
 name = "example"
-readme = "README.md"
 requires-python = ">=3.9"
 
 [project.optional-dependencies]
@@ -74,12 +73,12 @@ issues = "https://github.com/cisagov/skeleton-python-library/issues"
 mission = "https://www.cisa.gov/cybersecurity"
 source = "https://github.com/cisagov/skeleton-python-library"
 
-[tool.hatch.version]
-# Versions should comply with PEP440
-path = "src/example/_version.py"
-
 [tool.pytest.ini_options]
 # Increase verbosity, display extra test summary info for tests that
 # did not pass, display code coverage results, and enable debug
 # logging.
 addopts = "--verbose -ra --cov --log-cli-level=DEBUG"
+
+[tool.setuptools.dynamic]
+readme = {file = ["README.md"], content-type = "text/markdown"}
+version = {attr = "example._version.__version__"}

@jsf9k

jsf9k commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

In case it helps, this is how I cope with the current state of Python packaging:
image

@mcdonnnj

Copy link
Copy Markdown
Member

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code.

Keeping the setup.py file is the recommendation of PyPA in their guide to modernizing setup.py projects and it has not been deprecated.

@jsf9k

jsf9k commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code.

Keeping the setup.py file is the recommendation of PyPA in their guide to modernizing setup.py projects and it has not been deprecated.

Yes, in the first link they say it can coexist but nowhere do they say it must be present. My point is that if everything can be moved into the pyproject.toml file (as I show above) then there is no need for it. It is just a dumb placeholder file.

@jsf9k

jsf9k commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

I don't think the setup.py file is necessary unless you're compiling native code.

Actually, it isn't even necessary if you're compiling native code. So I don't see any reason to keep a setup.py file around.

The setuptools build backend:
- Supports native extensions, in contrast with the hatchling build
backend
- Is the most commonly used build backend among projects on PyPI
- Is an officially supported build backend from Python, in contrast
with the hatchling build backend
@jsf9k

jsf9k commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

FYI, here are the necessary changes for moving to setuptools as the build backend:

diff --git a/pyproject.toml b/pyproject.toml
index 370ba01..727434b 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -1,9 +1,9 @@
 # For more information about configuring project metadata for the
-# hatch build backend, please see
-# https://hatch.pypa.io/latest/config/metadata/
+# setuptools build backend, please see
+# https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html
 [build-system]
-build-backend = "hatchling.build"
-requires = ["hatchling"]
+build-backend = "setuptools.build_meta"
+requires = ["setuptools"]
 
 [project]
 authors = [
@@ -38,11 +38,10 @@ dependencies = [
     "schema",
 ]
 description = "Example Python library"
-dynamic = ["version"]
+dynamic = ["readme", "version"]
 keywords = ["skeleton"]
 license = "CC0-1.0"
 name = "example"
-readme = "README.md"
 requires-python = ">=3.9"
 
 [project.optional-dependencies]
@@ -74,12 +73,12 @@ issues = "https://github.com/cisagov/skeleton-python-library/issues"
 mission = "https://www.cisa.gov/cybersecurity"
 source = "https://github.com/cisagov/skeleton-python-library"
 
-[tool.hatch.version]
-# Versions should comply with PEP440
-path = "src/example/_version.py"
-
 [tool.pytest.ini_options]
 # Increase verbosity, display extra test summary info for tests that
 # did not pass, display code coverage results, and enable debug
 # logging.
 addopts = "--verbose -ra --cov --log-cli-level=DEBUG"
+
+[tool.setuptools.dynamic]
+readme = {file = ["README.md"], content-type = "text/markdown"}
+version = {attr = "example._version.__version__"}

Please see commit 461f872.

@jsf9k

jsf9k commented Nov 10, 2025

Copy link
Copy Markdown
Member Author

@mcdonnnj - At one point you said that if we used setuptools as the Python build backend then we shouldn't get rid of the setup.py file. Why was that? I don't think the setup.py file is necessary unless you're compiling native code. If you insist, I could add a simple setup.py like this as a placeholder:

import setuptools

setuptools.setup()

I don't see the point of that, though.

Also, it appears that hatchling doesn't support native code associated with Python libraries very well. Maybe that's one reason to use setuptools instead. As far as I can tell, setuptools is still the most common build backend and a lot of folks are saying the fact that the documentation appears to promote hatchling instead is misleading. Projects like numpy that have a lot of compiled native code tend to use something like PDM.

I'm kind of of the opinion that until we have native extensions it's a tossup between hatchling and setuptools.

I think this PR would change only slightly if we used setuptools as the build backend instead of hatchling. I'm happy to make those changes if that's where you're leaning.

Please see commit 461f872.

This ensures that the data file(s) are incorporated into the wheels
that are built.
This directory can be created by `pip install .`.
@jsf9k jsf9k requested a review from dav3r November 12, 2025 14:50

@dav3r dav3r left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks reasonable to me. 👍

@jsf9k

jsf9k commented Nov 12, 2025

Copy link
Copy Markdown
Member Author

I realize that we want wheel and pip to be the latest versions, but should we now remove setuptools from setup-env? If so then this would be an upstream change in cisagov/skeleton-generic.

This comment is no longer applicable since we moved to using setuptools as our build backend.

@jsf9k jsf9k merged commit 6d6043a into develop Nov 20, 2025
129 checks passed
@jsf9k jsf9k deleted the improvement/use-pyproject-toml-file branch November 20, 2025 15:05
@github-project-automation github-project-automation Bot moved this from In Progress to Done in Next Kraken Nov 20, 2025
@github-project-automation github-project-automation Bot moved this from Review in progress to Done in Skeleton Maintenance Nov 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file improvement This issue or pull request will add or improve functionality, maintainability, or ease of use kraken 🐙 This pull request is ready to merge during the next Lineage Kraken release python Pull requests that update Python code test This issue or pull request adds or otherwise modifies test code

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Upgrade from setup.py to pyproject.toml Setup.py link returns 404 error

4 participants