Let pip Build the Distribution for You¶
Our wheel script is dandy, but once you need to add more features, it becomes difficult to manage and update. Therefore, Python packaging provides a standard framework, PEP 517, to develop tools that build wheels, so different projects can share those tools and avoid dealing with the details.
There are several PEP 517 backend implementations available. The most famous is Setuptools, but other tools like Flit and Poetry are also gaining traction. You should use them when developing your own project. But let’s roll our own here, because why not!
Note
If you’re wondering: pip is a PEP 517 frontend. So once we have a backend, we can use pip to call it to generate a wheel.
A PEP 517 backend must implement two Python functions:
build_wheel
build_sdist
Build a wheel¶
The build_wheel
hook can be implemented to, well, build wheels.
import pathlib
import tempfile
from .distinfo import iter_files
from .wheel import create_dist_info, create_wheel
_NAME = "my_package"
_VERSION = "2"
_TAG = "py3-none-any"
_PACKAGE = pathlib.Path("my_package")
def build_wheel(
wheel_directory, config_settings=None, metadata_directory=None,
):
with tempfile.TemporaryDirectory() as td:
if metadata_directory is None:
td_path = pathlib.Path(td)
dist_info = create_dist_info(
_NAME, _VERSION, _TAG, _PACKAGE, td_path,
)
else:
dist_info = pathlib.Path(metadata_directory)
wheel_path = create_wheel(
_NAME,
_VERSION,
_TAG,
_PACKAGE,
dist_info,
pathlib.Path(wheel_directory),
)
return wheel_path.name
Next, we need to tell pip where our backend is. This is done with PEP 518,
which introduces the configuration file pyproject.toml
, and a TOML table
for this:
[build-system]
requires = []
backend-path = [""]
build-backend = "packager.pep517"
requires
lists packages the frontend should install before running the backend. We leave it blank since we include the script in the project, and only use the standard library.backend-path
tells the frontend where to find the backend. Empty means where thepyproject.toml
file is.build-backend
tells the frontend how to import the backend functions.
Now we can let pip build the wheel for us:
$ pip wheel --no-deps /path/to/example-project
Processing /path/to/example-project
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Building wheels for collected packages: my-package
Building wheel for my-package (PEP 517) ... done
Created wheel for my-package: filename=my_package-2-py3-none-any.whl ...
Stored in directory: ...
Successfully built my-package
Or we can let pip build and install the wheel in one shot:
$ pip install /path/to/example-project
Processing /path/to/example-project
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Building wheels for collected packages: my-package
Building wheel for my-package (PEP 517) ... done
Created wheel for my-package: filename=my_package-2-py3-none-any.whl ...
Stored in directory: ...
Successfully built my-package
Installing collected packages: my-package
Attempting uninstall: my-package
Found existing installation: my-package 1
Uninstalling my-package-1:
Successfully uninstalled my-package-1
Successfully installed my-package-2
Build a source distribution¶
A source distribution, a.k.a. sdist, is an archive to destribute the source code with some simple descriptive metadata. It is most commonly used by packages that require binary compilation to support unknown platforms, and is also useful for auditing purposes.
When pip receives an sdist, it first extract the source code, and builds a wheel from it to install. [1] The relation can be drawn like this:
extract
+-----------------------+
| v
+-------+ build_sdist +--------------+
| sdist | <------------ | source code |
+-------+ +--------------+
|
| build_wheel
v
+--------------+
| wheel |
+--------------+
|
| install
v
+--------------+
| installation |
+--------------+
[1] | There are historical fallbacks to this, but let’s avoid the details here. The sdist-wheel relation is true in modern contexts. |
Since sdists need to be built to make sense anyway, their content is much less rigid than wheels’. The only constraint is basically to always use a gzipped POSIX.1-2001 pax tar with UTF-8 paths.
import tarfile
return sdist_path.name
Not much we haven’t seen here. This is very similar to how we built wheels, with only slight differences:
- We use tarfile instead of zipfile to create a gzipped pax tar.
- We need to bundle both
pyproject.toml
and thepackager
directory along, so the sdist can build itself later.
pip does not know how to buld an sdist because it’s a package installer, and
the installation process never needs to call build_sdist
, as shown in the
above graph. But we use the build
frontend instead:
$ py -m pip install build
$ py -m build /path/to/example-project --sdist --outdir /path/to/save/wheel
Note
The build
tool is still in early development and experiemental. Feel
free to report problems to FFY00/python-build.