Python toolchains in rules

Bazel is supposed to be repeatable, right? Your teammate runs the tests on her computer and should get the same result as you did. A build from a commit last month should be the same if you re-build it today. But in practice, this only works when we declare a hermetic build, where all the tools used are declared to Bazel using some pinned version. Bazel fetches these tools for us so that developers don't even have to think about whether they have the right version.

This has not been the case for Python, where we've seen clients trip over having different python interpreter versions in different environments.

Toolchains play a critical role in Bazel for achieving deterministic and hermetic builds. Today the team at Aspect has reached an important goal in rules_python when we landed the support for toolchains in the latest 0.7.0 release.

Here's how you start using it, based on the upstream example:

load(
    "@rules_python//python:repositories.bzl",
    "python_register_toolchains",
)

# Build steps will discover the right python via
# Bazel's toolchain support
python_register_toolchains(
    name = "python310",
    # Available versions are in
    # https://github.com/bazelbuild/rules_python/blob/0.7.0/python/versions.bzl#L39
    python_version = "3.10",
)

# However, repository rules run before toolchain resolution,
# so we explicitly load the "resolved" interpreter
# for the host platform...
load("@python310_resolved_interpreter//:defs.bzl", "interpreter")

load("@rules_python//python:pip.bzl", "pip_parse")

pip_parse(
    ...
    # ...and pass that interpreter to be used when
    # dependencies are resolved.
    python_interpreter_target = interpreter,
    ...
)

Any build or test actions using py_binary or py_test will pick up the hermetic toolchain automatically via bazel.build/docs/toolchains#toolchain-resol... However, repository rules like pip_parse run prior to that resolution, so we've used a trick here to load the resolved toolchain for the host platform, where the repository rules run.

It relies on the fantastic work from github.com/indygreg in github.com/indygreg/python-build-standalone. Consider sponsoring him on GitHub!

We encourage everyone to try the feature out.

A special thanks to github.com/UebelAndre and github.com/alexeagle for helping out.

Python toolchains in rules_python