Turn Your Package into a Distribution

What we came up with in distinfo.py is what package installers like pip does essentially, if you strip off all its bells and whistles. The difference is that, pip requires distributions to be a single-file archive, and structured in a certain way, so it knows exactly what to download and extract, and where to.

The modern format pip uses to install, called “wheels”, is specified in PEP 427. A wheel is a ZIP archive, and is unarchived into site-packages on installation. (Again, ignoring many bells and whistles that comes along.) Since Python has built-in ZIP support, we can easily extend our installer to do this.

packager/wheel.py
import argparse
import email.message
import pathlib
import tempfile
import zipfile

from .distinfo import (
    create_dist_info_dir,
    iter_files,
    write_metadata,
    write_record,
)


def _write_wheel_metadata(dist_info, tag):
    m = email.message.EmailMessage()
    m["Wheel-Version"] = "1.0"
    m["Generator"] = "packager/wheel.py"
    m["Root-Is-Purelib"] = "true"
    m["Tag"] = tag
    dist_info.joinpath("WHEEL").write_bytes(bytes(m))


def create_dist_info(name, version, tag, package, output_dir):
    dist_info = create_dist_info_dir(output_dir, name, version)
    write_metadata(dist_info, name, version)
    _write_wheel_metadata(dist_info, tag)
    write_record(dist_info, package)
    return dist_info


def create_wheel(name, version, tag, package, dist_info, output_dir):
    wheel_path = output_dir.joinpath(f"{name}-{version}-{tag}.whl")
    with zipfile.ZipFile(wheel_path, "w") as zf:
        for path, relative in iter_files((package, dist_info)):
            zf.write(path, relative.as_posix())
    return wheel_path


def _parse_args(argv):
    parser = argparse.ArgumentParser()
    parser.add_argument("output", type=pathlib.Path)
    return parser.parse_args(argv)


_NAME = "my_package"

_VERSION = "1"

_TAG = "py3-none-any"

_PACKAGE = pathlib.Path("my_package")


def main(argv=None):
    options = _parse_args(argv)
    options.output.mkdir(parents=True, exist_ok=True)
    with tempfile.TemporaryDirectory() as td:
        td_path = pathlib.Path(td)
        dist_info = create_dist_info(_NAME, _VERSION, _TAG, _PACKAGE, td_path)
        create_wheel(
            _NAME, _VERSION, _TAG, _PACKAGE, dist_info, options.output,
        )


if __name__ == "__main__":
    main()

Now we can build our wheel by calling

$ cd /path/to/example-project
$ py -m packager.wheel /path/to/save/wheel

And use pip to install the wheel:

$ cd /path/to/save/wheel
$ py -m pip install ./my_package-1-py3-none-any.whl

which can be correctly recognised (notice the version changed from 0 to 1):

$ py -m pip show my-package
Name: my-package
Version: 1
Summary: None
Home-page: None
Author: None
Author-email: None
License: None
Location: /path/to/site-packages
Requires:
Required-by:

Anatomy of a wheel

In the above example, the wheel is installed by simply extracted into the site-packages directory. So the archive contains exactly the same two components copied by our script in the previous section: code, and dist-info.

  • Code is exactly the same.
  • .dist-info is named the same.
  • METADATA is exactly the same.
  • RECORD is exactly the same.
  • INSTALLER is not present, since the wheel is not yet installed anywhere. pip automatically writes the INSTALLER value in the installed .dist-info directory when it installs.

There is a new file WHEEL that contains metadata that describes the currently used wheel format itself. We’re not getting into this too deep here since “real” distribution tools handle these automatically for you. Read the PEP for more information if you’re into these stuff.

  • The only valid Wheel-Version value at the current time is 1.0.
  • Generator marks the tools that generates the wheel.
  • Root-Is-Purelib describes where the package is installed. This does not matter in most situations, but for some Python installations that distinguish pure-Python and platform-dependent packages, setting this to false would cause the wheel to be installed under dist-packages instead.
  • Tag describes what platforms the wheel can be installed. For most pure-Python packages, this is always py3-none-any (or py2- if it’s for Python 2), indicating the major Python version, ABI, and platform information.

The wheel is named as {name}-{version}-{tag}.whl. This helps tools identify the wheel’s compatibility without extracting it, which is useful in network contexts.