This PR adds Python 3.14 support for Distribtests by updating relevant docker images and distribtest build files for Linux, MacOS and Windows environments. However, artifact generation in distribtests will be enabled in the next PR #40403
Prerequisites merged before this PR:
- #40317
- #40354
- #40383Closes#40289
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40289 from sreenithi:python_314_test_support 3b8bf9618da6113504edc5867d4787aa059b46ea
PiperOrigin-RevId: 802967999
This PR does the following:
- updates the base docker images to recent ones for manylinux-x86 and manylinux-x64
- updates Python version to be installed in the armv7 images
Related PRs updating other Python docker images:
- #40317
- #40354Closes#40383
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40383 from sreenithi:update_manylinux_docker_base_images dbfe777f80f0a622e5ceecec4614b70f42bf3f4d
PiperOrigin-RevId: 793828964
Multiple improvements to the python image templates that maximize docker build cache reuse.
Even for the clean build, I estimate the total build time be reduced by 1.5 hours.
This is achieved by grouping RUN commands to avoid intermediate layers.
Previously one image build spent about 10 minutes only on `RUN git config --global --add safe.directory ...` commands.
With this change all git config commands take a minute or less:
For the same reason, I grouped `RUN /opt/python/cp38-cp38/bin/pip` commands in `python_pip_builds.include` as well.
However, the most gains will be on building new versions of the same images. I've change the order of a typical python build:
```jinja
# before
{% include "../python_pip_builds.include" %}
{% include "../ccache.include" %}
{% include "../git_config.include" %}
# after
{% include "../git_config.include" %} # less likely to change
{% include "../ccache.include" %} # only on ccache version updates
{% include "../python_pip_builds.include" %} # likely to change more often
```
Other fixes/improvements:
- Added missing newlines at OEF in the templates - so we should get better diffs now.
- Fixed template extension in AUTOGENERATED notice
- Pre-installed ccache dependencies in musl images to avoid downloading them from internet (and a related build warning)
Closes#39613
PiperOrigin-RevId: 778114452
Updated almost all of our Docker images to make sure they're up-to-date and can be rebuilt. A few images as follows can't be updated right now because they're based on old, unsupported images. We'll need to fix those separately.
- tools/dockerfile/grpc_artifact_centos6_x64
- tools/dockerfile/grpc_artifact_centos6_x86
- tools/dockerfile/distribtest/csharp_centos7_x64
- tools/dockerfile/distribtest/python_centos7_x64
- tools/dockerfile/distribtest/python_dev_centos7_x64
- tools/dockerfile/interoptest/grpc_interop_aspnetcore
- tools/dockerfile/interoptest/grpc_interop_go1.8
Closes#38934
PiperOrigin-RevId: 734711029
This PR adds templating for Python versions and updates the maximum supported Python version to 3.13. The following major changes related to templating are added:
- Minimum supported Python version and list of supported versions in `setup.py` are fetched using new template generated files called `python_version.py`
- Dockerfiles for the different Python Linux builds are now template generated.
- The "Supported Python Versions" section from READMEs of ancillary and main packages have been removed
Note: All the `python_version.py` files and Linux build `Dockerfiles` except `tools/dockerfile/grpc_artifact_python_linux_armv7/Dockerfile` in the PR are generated from the respective templates.
Further non-templated additions to add support for Python 3.13:
- install scripts and artifacts for windows, macos and linux are added manually. Later, these can be templated as well.
- updated cython bounds to 3.x
- updated twine version to solve [cgi module import error](https://github.com/pypa/twine/issues/1046)
- the twine update introduces a dependency on cryptography>=2.0. But the cryptography package doesn't support 32-bit Linux images and hence `twine check` has been disabled for x86 manylinux and x86 musllinux artifacts.
Closes#37643
PiperOrigin-RevId: 678954495
Support Python 3.12.
### Testing
* Passed all Distribution Tests.
* Also tested locally by installing 3.12 artifact.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
Sreenithi Sridharan