I'm having trouble getting a python package to build with python -m build .. setup.py fails on:
FileNotFoundError: [Errno 2] No such file or directory: 'requirements/requirements.txt'
It's caused by the fact that build is copying files to a temporary directory first. But it's only copying the source/, README.md, setup.py, setup.cfg. It's not copying requirements/.
For complex reasons my setup.py needs to reference other files at the root of the source repo - a directory containing multiple requirements.txt files. It's not really worth discussing why it needs to be structured this way, I've already been through that long debate with colleagues.
This works fine when we install the package through pip install -e . or as a git dependency git+ssh://... but fails when building before we push to a pypi repo.
setup.cfg
setup.py
source/
source/my_package/
requirements/
requirements/requirements.txt
requirements/some-other-requirements.txt
setup.py references this directory before calling setup().
from pathlib import Path
from setuptools import setup, find_namespace_packages
requirements_dir = Path("requirements")
# This is the line that fails:
with (requirements_dir / "requirements.txt").open() as f:
install_requires = list(f)
setup(
packages=find_namespace_packages(where="source", include=["acme_corp.*"], exclude=["tests", "tests.*"]),
package_dir={"": "source"},
install_requires=install_requires,
extras_require=optional_packages,
)
build does the correct job here. After building the source dist, it checks whether the built result can be actually installed. And without including the files under requirements into the source dist, the source dist can not be installed and is thus unusable. Try it out:
$ python -m build --sdist # builds fine, but the tar is broken:
$ cd dist
$ tar tf pkg-0.0.1.tar.gz | grep requirements # will be empty
$ pip wheel pkg-0.0.1.tar.gz # will fail
Processing ./pkg-0.0.1.tar.gz
File was already downloaded
Preparing metadata (setup.py) ... error
ERROR: Command errored out with exit status 1:
command:
...
Complete output (9 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-req-build-wfzz27_k/setup.py", line 9, in <module>
with (requirements_dir / "requirements.txt").open() as f:
File "/usr/lib64/python3.9/pathlib.py", line 1252, in open
return io.open(self, mode, buffering, encoding, errors, newline,
File "/usr/lib64/python3.9/pathlib.py", line 1120, in _opener
return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pip-req-build-wfzz27_k/requirements/requirements.txt'
To fix, write a MANIFEST.in alongside the setup.py that includes requirements directory in the source dist. Example file contents:
graft requirements
Building a source dist should work now; you can additionally verify that the sdist now contains all files to install from:
$ tar tf dist/pkg-0.0.1.tar.gz | grep requirements
pkg-0.0.1/requirements/
pkg-0.0.1/requirements/requirements.txt
...
This only concerns the source dist, as the wheel is built on host (your machine) already and does not contain a setup script anymore. Wheel building will also ignore the MANIFEST.in file.