GH-45867: [Python] Fix `SetuptoolsDeprecationWarning` #47141

paddyroddy · 2025-07-20T08:33:33Z

Rationale for this change

When building locally, I get many errors along the lines of

Please ensure the files specified are contained by the root
of the Python package (normally marked by `pyproject.toml`).

By 2026-Mar-20, you need to update your project and remove deprecated calls
or your builds will no longer be supported.

See https://packaging.python.org/en/latest/specifications/glob-patterns/ for details.

What changes are included in this PR?

Make the licence SPDX compliant
Remove the licence classifier
Move the licence files from setup.cfg to pyproject.toml
Fix the disallowed glob patterns via symlinks
Bumped the minimum version of setuptools due to macOS CI failures (don't know why this happened, caching maybe?)

I appreciate the symlink change might prove controversial. See discussions in #45867, fixes #45867.

Are these changes tested?

When I rebuild locally, I get no errors any more.

Are there any user-facing changes?

Yes. The minimum required version of setuptools is now 77. However, this is available on >=3.9 so won't affect anyone really.

GitHub Issue: [CI][Python] A new version (77.0.2) of setuptools seems to have broken some of our builds #45867

github-actions · 2025-07-20T08:33:57Z

⚠️ GitHub issue #45867 has been automatically assigned in GitHub to PR creator.

github-actions · 2025-07-20T09:33:40Z

⚠️ GitHub issue #45867 has been automatically assigned in GitHub to PR creator.

rok · 2025-07-20T10:03:57Z

python/pyproject.toml

    # versions released after 2022
    "setuptools_scm[toml]>=8",
-    "setuptools>=64",
+    "setuptools>=77",


This is required due to setuptools<77 not recognizing the new license strings proposed in this PR (see pypa/setuptools#4903 for explanation). I'm not sure if this is controversial so bringing it up here. @raulcd

rok · 2025-07-20T10:22:54Z

@github-actions crossbow submit

github-actions · 2025-07-20T10:24:00Z

no tasks were provided for the job
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/16398862798

raulcd · 2025-07-20T10:25:16Z

@github-actions crossbow submit -g python -g wheels

github-actions · 2025-07-20T10:26:23Z

Invalid group(s) {'wheels'}. Must be one of {'example-cpp', 'python', 'nightly-release', 'cpp', 'verify-rc-source-macos', 'wheel', 'test', 'verify-rc-source', 'cuda', 'vcpkg', 'linux-amd64', 'conan', 'r', 'nightly', 'ruby', 'linux', 'c-glib', 'linux-arm64', 'verify-rc-source-linux', 'example', 'packaging', 'integration', 'example-python', 'fuzz', 'nightly-tests', 'verify-rc', 'homebrew', 'nightly-packaging'}
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/16398880949

raulcd · 2025-07-20T10:27:28Z

@github-actions crossbow submit -g python

github-actions · 2025-07-20T10:29:58Z

Revision: 6b1a08b

Submitted crossbow builds: ursacomputing/crossbow @ actions-2fdf3226a1

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.13
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-42-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

kou · 2025-07-20T11:31:45Z

python/LICENSE.txt

Does symbolic link work on Windows...?

Good point. I guess there's no other way to fix it then other than copying or creating new licence and notice files.

Have copied over the files for the tests @rok

kou · 2025-07-20T11:32:51Z

@github-actions crossbow submit -g wheel

github-actions · 2025-07-20T11:34:05Z

⚠️ GitHub issue #45867 has been automatically assigned in GitHub to PR creator.

github-actions · 2025-07-20T11:36:02Z

Revision: 6b1a08b

Submitted crossbow builds: ursacomputing/crossbow @ actions-fe79dfe84b

Task	Status
python-sdist
wheel-macos-monterey-cp310-cp310-amd64
wheel-macos-monterey-cp310-cp310-arm64
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-macos-monterey-cp312-cp312-amd64
wheel-macos-monterey-cp312-cp312-arm64
wheel-macos-monterey-cp313-cp313-amd64
wheel-macos-monterey-cp313-cp313-arm64
wheel-macos-monterey-cp313-cp313t-amd64
wheel-macos-monterey-cp313-cp313t-arm64
wheel-macos-monterey-cp39-cp39-amd64
wheel-macos-monterey-cp39-cp39-arm64
wheel-manylinux-2-28-cp310-cp310-amd64
wheel-manylinux-2-28-cp310-cp310-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-manylinux-2-28-cp312-cp312-amd64
wheel-manylinux-2-28-cp312-cp312-arm64
wheel-manylinux-2-28-cp313-cp313-amd64
wheel-manylinux-2-28-cp313-cp313-arm64
wheel-manylinux-2-28-cp313-cp313t-amd64
wheel-manylinux-2-28-cp313-cp313t-arm64
wheel-manylinux-2-28-cp39-cp39-amd64
wheel-manylinux-2-28-cp39-cp39-arm64
wheel-musllinux-1-2-cp310-cp310-amd64
wheel-musllinux-1-2-cp310-cp310-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-musllinux-1-2-cp312-cp312-amd64
wheel-musllinux-1-2-cp312-cp312-arm64
wheel-musllinux-1-2-cp313-cp313-amd64
wheel-musllinux-1-2-cp313-cp313-arm64
wheel-musllinux-1-2-cp313-cp313t-amd64
wheel-musllinux-1-2-cp313-cp313t-arm64
wheel-musllinux-1-2-cp39-cp39-amd64
wheel-musllinux-1-2-cp39-cp39-arm64
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp39-cp39-amd64

rok · 2025-07-20T12:43:29Z

@github-actions crossbow submit -g wheel

github-actions · 2025-07-20T12:46:14Z

Revision: df371b1

Submitted crossbow builds: ursacomputing/crossbow @ actions-4d54dadfaa

Task	Status
python-sdist
wheel-macos-monterey-cp310-cp310-amd64
wheel-macos-monterey-cp310-cp310-arm64
wheel-macos-monterey-cp311-cp311-amd64
wheel-macos-monterey-cp311-cp311-arm64
wheel-macos-monterey-cp312-cp312-amd64
wheel-macos-monterey-cp312-cp312-arm64
wheel-macos-monterey-cp313-cp313-amd64
wheel-macos-monterey-cp313-cp313-arm64
wheel-macos-monterey-cp313-cp313t-amd64
wheel-macos-monterey-cp313-cp313t-arm64
wheel-macos-monterey-cp39-cp39-amd64
wheel-macos-monterey-cp39-cp39-arm64
wheel-manylinux-2-28-cp310-cp310-amd64
wheel-manylinux-2-28-cp310-cp310-arm64
wheel-manylinux-2-28-cp311-cp311-amd64
wheel-manylinux-2-28-cp311-cp311-arm64
wheel-manylinux-2-28-cp312-cp312-amd64
wheel-manylinux-2-28-cp312-cp312-arm64
wheel-manylinux-2-28-cp313-cp313-amd64
wheel-manylinux-2-28-cp313-cp313-arm64
wheel-manylinux-2-28-cp313-cp313t-amd64
wheel-manylinux-2-28-cp313-cp313t-arm64
wheel-manylinux-2-28-cp39-cp39-amd64
wheel-manylinux-2-28-cp39-cp39-arm64
wheel-musllinux-1-2-cp310-cp310-amd64
wheel-musllinux-1-2-cp310-cp310-arm64
wheel-musllinux-1-2-cp311-cp311-amd64
wheel-musllinux-1-2-cp311-cp311-arm64
wheel-musllinux-1-2-cp312-cp312-amd64
wheel-musllinux-1-2-cp312-cp312-arm64
wheel-musllinux-1-2-cp313-cp313-amd64
wheel-musllinux-1-2-cp313-cp313-arm64
wheel-musllinux-1-2-cp313-cp313t-amd64
wheel-musllinux-1-2-cp313-cp313t-arm64
wheel-musllinux-1-2-cp39-cp39-amd64
wheel-musllinux-1-2-cp39-cp39-arm64
wheel-windows-cp310-cp310-amd64
wheel-windows-cp311-cp311-amd64
wheel-windows-cp312-cp312-amd64
wheel-windows-cp313-cp313-amd64
wheel-windows-cp313-cp313t-amd64
wheel-windows-cp39-cp39-amd64

martin-traverse · 2025-09-11T20:56:51Z

As per #47551, the test seems to be expecting this:

license = {text = "Apache-2.0"}

paddyroddy · 2025-09-30T16:04:31Z

As per #47551, the test seems to be expecting this:
license = {text = "Apache-2.0"}

I don't think it's this. Those tests are passing, mine is failing on the AMD64 Conda Integration Test. @rok any idea?

rok · 2025-10-01T15:27:55Z

@paddyroddy as per @raulcd there was a go failure (#47673) on master that was since fixed, so rebase should fix it.

…ptools-deprecation

paddyroddy · 2025-10-08T12:37:05Z

@rok passing again now. Sorry, was away again but actually back now.

pitrou · 2025-10-28T09:00:00Z

This essentially copies the license files from repo root to python, right? That's not a very good idea IMHO: we will now need to maintain two sets of identical files.

pitrou · 2025-10-28T09:10:14Z

An alternative would be to copy those files automatically when sdist is run. Here is some inspiration from a quick Google search:
https://github.com/pybind/pybind11/blob/3b30b0a51e48a3273918f30dfb222db604cea5e9/setup.py#L63-L77

rok · 2025-10-28T09:19:24Z

An alternative would be to copy those files automatically when sdist is run.

That seems like a good alternative to the proposed symlink solution.

rok · 2025-10-30T16:19:44Z

An alternative would be to copy those files automatically when sdist is run.

@paddyroddy would you be interested in adding this here?

paddyroddy · 2025-10-30T17:30:45Z

@paddyroddy would you be interested in adding this here?

Yes. Will try and find time tomorrow.

…ptools-deprecation

paddyroddy · 2025-11-01T16:44:16Z

An alternative would be to copy those files automatically when sdist is run. Here is some inspiration from a quick Google search: pybind/pybind11@3b30b0a/setup.py#L63-L77

Isn't already covered by the MANIFEST?

arrow/python/MANIFEST.in

Lines 2 to 3 in d165954

    
           include ../LICENSE.txt 
        
           include ../NOTICE.txt

I was copying the files here from a git point of view, but they should already be packaged

rok · 2025-11-03T10:58:22Z

I was copying the files here from a git point of view, but they should already be packaged

I don't think they are packaged currently, see #47932 (comment).

If I understand correctly @pitrou is suggesting we copy them at wheel-build-time so they do get packaged, but are not duplicated in the repo.

paddyroddy · 2025-11-03T11:49:22Z

I don't think they are packaged currently, see #47932 (comment).

Ah, so they are meant to be, but the current setup isn't working. Will look into the SDist thing.

paddyroddy added 4 commits July 20, 2025 10:23

Make licence SPDX compliant

0216dd7

Licence classifiers are deprecated

7b5fcce

Move license-files to pyproject.toml

5fc3f18

.. isn't valid so create symlinks and point to them

4be7410

paddyroddy requested review from AlenkaF, raulcd and rok as code owners July 20, 2025 08:33

github-actions bot added Component: Python awaiting review Awaiting review labels Jul 20, 2025

Set minimum setuptools as 77

6b1a08b

paddyroddy requested review from assignUser, jonkeane and kou as code owners July 20, 2025 09:20

rok reviewed Jul 20, 2025

View reviewed changes

github-actions bot added awaiting changes Awaiting changes and removed awaiting review Awaiting review labels Jul 20, 2025

kou reviewed Jul 20, 2025

View reviewed changes

kou changed the title ~~GH-45867: [PYTHON] Fix SetuptoolsDeprecationWarning~~ GH-45867: [Python] Fix SetuptoolsDeprecationWarning Jul 20, 2025

Copy over text files instead

df371b1

github-actions bot added awaiting change review Awaiting change review and removed awaiting changes Awaiting changes labels Jul 20, 2025

rok mentioned this pull request Sep 11, 2025

GH-47550: [Python] Use SPDX convention for license metadata on PyPI #47551

Open

Merge branch 'main' into paddyroddy/fix-setuptools-deprecation

f832a6d

Merge remote-tracking branch 'upstream/main' into paddyroddy/fix-setu…

b850194

…ptools-deprecation

rok mentioned this pull request Oct 24, 2025

[Release][Python] PyPI rejects our source distribution due to missing LICENSE.txt #47932

Open

paddyroddy added 2 commits November 1, 2025 10:09

Merge remote-tracking branch 'upstream/main' into paddyroddy/fix-setu…

8b214e1

…ptools-deprecation

Remove duplicate text files

3929753

GH-45867: [Python] Fix SetuptoolsDeprecationWarning #47141

Are you sure you want to change the base?

GH-45867: [Python] Fix SetuptoolsDeprecationWarning #47141

Uh oh!

Conversation

paddyroddy commented Jul 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

rok Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

rok commented Jul 20, 2025

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

raulcd commented Jul 20, 2025

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

raulcd commented Jul 20, 2025

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

kou Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

paddyroddy Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

paddyroddy Jul 20, 2025

Choose a reason for hiding this comment

Uh oh!

kou commented Jul 20, 2025

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

rok commented Jul 20, 2025

Uh oh!

github-actions bot commented Jul 20, 2025

Uh oh!

martin-traverse commented Sep 11, 2025

Uh oh!

paddyroddy commented Sep 30, 2025

Uh oh!

rok commented Oct 1, 2025

Uh oh!

paddyroddy commented Oct 8, 2025

Uh oh!

pitrou commented Oct 28, 2025

Uh oh!

pitrou commented Oct 28, 2025

Uh oh!

rok commented Oct 28, 2025

Uh oh!

rok commented Oct 30, 2025

Uh oh!

paddyroddy commented Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paddyroddy commented Nov 1, 2025

Uh oh!

rok commented Nov 3, 2025

Uh oh!

paddyroddy commented Nov 3, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

GH-45867: [Python] Fix `SetuptoolsDeprecationWarning` #47141

GH-45867: [Python] Fix `SetuptoolsDeprecationWarning` #47141

paddyroddy commented Jul 20, 2025 •

edited

Loading

paddyroddy commented Oct 30, 2025 •

edited

Loading