Change was **not** created by the release automation script, because it doesn't handle a +2 version bump. See go/grpc-release
Closes#41291
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41291 from murgatroid99:v1.79.0-dev_bump 9a9bf54e5a891459390792dc9d547bdc17b7dd4d
PiperOrigin-RevId: 848168598
Python Bazel tests have been failing since yesterday after layering check was enabled in grpcio_tools build in commit: 756389e9e7
Temporarily disabling it after discussing IRL with @rishesh007
Closes#41142
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41142 from sreenithi:temp_fix_python_bazel_test 751c420bf3a27066d6cdd912e0e08e9c0acaebb8
PiperOrigin-RevId: 837494537
This PR removes some additional files that were included in the grpcio_tools package wheels after migrating to pyproject.toml build system in #40833Closes#40999
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40999 from sreenithi:fix_grpc_tools_wheels_2 a261b2160e3ddd34a5c0b74e6d84b2f8c1bda1cd
PiperOrigin-RevId: 827755815
This PR updates the minimum version of `setuptools` package required across different Python setup files to v77.0.1. This version contains Python 3.14 support as well as deprecates a format for defining project license in `pyproject.toml` files ([Reference](https://setuptools.pypa.io/en/stable/history.html#id71)) which is a prerequisite for #40833Closes#40931
PiperOrigin-RevId: 823008815
gRPC is currently getting formatted with two different clang-format implementations, and due to some weirdness they have different include file orderings. This change introduces clang-format configuration to ensure that the two systems align - it's *highly* expected that this will need some maintenance going forward as the two systems evolve.
Closes#40905
PiperOrigin-RevId: 819606209
Change was created by the release automation script. See go/grpc-release.
Closes#40796
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40796 from sergiitk:bump_dev_version_202509291139 e7aa910253d1706a72822da986b8b8e7bc87931d
PiperOrigin-RevId: 812961524
This PR enables Python 3.14 in all the different tests - Basic tests (Native Python tests), Bazel tests and Distrib tests to build Python 3.14 artifacts. In addition, it also updates all the public facing METADATA versions.
## Distribtests
Required pre-requisite changes to enable 3.14 artifacts are covered in #40289 .
## Bazel tests
Enabling Python 3.14 required updating the rules_python version to a more recent version that supports 3.14. This was done in #40602
## Basic tests
The following errors were caught by the Basic tests when running via Python 3.14 and resolved in this PR:
### 1) No running event loop for AsyncIO when run outside an async function
```
Traceback (most recent call last):
File "src/python/grpcio/grpc/_cython/_cygrpc/aio/common.pyx.pxi", line 184, in grpc._cython.cygrpc.get_working_loop
RuntimeError: no running event loop
```
This was caught by the `tests_aio.unit.outside_init_test.TestOutsideInit` and `tests_aio.unit.init_test.TestInit` tests, and was also previously reported in #39507 with the root cause.
Following some investigation, the fix is being worked on by @sergiitk in PR #40293. In order to parallelize the fix and this PR, these 2 tests are currently being skipped for Python 3.14 and above.
### 2) Pickling error from the `multiprocessing` library
```
_pickle.PicklingError: Can't pickle <function _test_well_known_types at 0x7f3937eee610>: it's not the same object as tests.unit._dynamic_stubs_test._test_well_known_types
when serializing dict item '_target'
when serializing multiprocessing.context.Process state
when serializing multiprocessing.context.Process object
```
This was caught by the `tests.unit._dynamic_stubs_test.DynamicStubTest` which runs test cases in a subprocess using the `multiprocessing` library.
Error root cause:
- The default start method of multiprocessing in linux has changed to `forkserver` instead of `fork` from Python 3.14.
- `forkserver` has a few extra restrictions for picklability as compared to `fork` (Ref: [Python Docs](https://docs.python.org/3.14/library/multiprocessing.html#the-spawn-and-forkserver-start-methods))
- All the [test case functions](0243842d5d/src/python/grpcio_tests/tests/unit/_dynamic_stubs_test.py (L115)) in the DynamicStubTest that are provided as `target` to the `multiprocessing.Process` use decorators. This causes problems when pickling them.
Hence to resolve this, we manually set the 'start method' of `multiprocessing` to use the `fork` start method.
Closes#40403
PiperOrigin-RevId: 804290760
A follow up on #40518, which regenerated xds-protos with protobuf 6.x.
Since PR #40518, xds-protos grpc gencode contains a poison pill that enforces grpcio runtime version to be equal or greater to the grpcio-tools version that generated the code.
This PR:
- Locks xds-protos grpcio dependencies to the same version as specified in generate_projects.sh
- Documents the need to keep these in sync
Closes#40521
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40521 from sergiitk:py/xds-protos/requirements 794fb65f2989da4f8afcc201fa6ac542db775df3
PiperOrigin-RevId: 800547324
As title
Closes#40484
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40484 from apolcyn:bump_dev_version_202508191952 e788be57e9dc7f5e8316bee4baadec26fba3f6e6
PiperOrigin-RevId: 798331971
Fix python import paths in bazel tests:
- `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
- `//examples/python/cancellation:test/_cancellation_example_test`
- `//examples/python/compression:test/compression_example_test`
Previously paths were set incorrectly. With bazel + python 3.11, this resulted in the following errors:
<details>
<summary>error examples for posterity</summary>
#### `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
```
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 141, in _test_syntax_errors
protos = protoc._protos("flawed.proto")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/protoc.py", line 110, in _protos
module = importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'flawed_pb2'
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 168, in test_combined_import
_run_in_subprocess(_test_combined_import)
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 48, in _run_in_subprocess
raise error_queue.get()
ModuleNotFoundError: No module named 'simple_pb2'
```
#### `//examples/python/cancellation:test/_cancellation_example_test`
```
test_graceful_sigint (__main__.CancellationExampleTest.test_graceful_sigint) ... Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3027/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/cancellation/test/_cancellation_example_test.runfiles/com_github_grpc_grpc/examples/python/cancellation/server.py", line 26, in
import search
ModuleNotFoundError: No module named 'search'
```
#### `//examples/python/compression:test/compression_example_test`
```
test_compression_example (__main__.CompressionExampleTest.test_compression_example) ... Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/client.py", line 24, in
import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/server.py", line 26, in
import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
FAIL
======================================================================
FAIL: test_compression_example (__main__.CompressionExampleTest.test_compression_example)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/test/compression_example_test.py", line 68, in test_compression_example
self.assertEqual(0, client_return_code)
AssertionError: 0 != 1
```
</details>
The main difference is as of Python 3.11 bazel no longer appends the directory of the file to the sys.path. For example, compression test sys.path:
1. Contains `examples/python/compression/test` in 3.10
2. Doesn't contain `examples/python/compression/test` in 3.11
Closes#40273
PiperOrigin-RevId: 786308459
In Python 3.8, multiprocessing module changed the default fork method from `fork` to `spawn`:
> On macOS, the spawn start method is now used by default.
> — https://docs.python.org/3/whatsnew/3.8.html#multiprocessing
Spawn method uses pickling to pass the target method to the fork. Pickling is only supported for top-level defined methods. This PR changes the test wrapper to use an annotation to collect test result instead of `_wrap_in_subprocess`, therefore making the method picklable by keeping the method definition at the top level. Similar to 9772d15077/src/python/grpcio_tests/tests/unit/_dynamic_stubs_test.py (L47).
Steps to reproduce:
```console
$ sw_vers
ProductName: macOS
ProductVersion: 15.5
BuildVersion: 24F74
$ cd tools/distrib/python/grpcio_tools/grpc_tools/test/
$ python3.9 -m venv --upgrade-deps .venv
$ source .venv/bin/activate
$ pip install grpcio grpcio-tools
$ python protoc_test.py
```
Before:
```console
$ python protoc_test.py
EEEEEEE
======================================================================
ERROR: test_combined_import (__main__.ProtocTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/sergiitk/Development/grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 168, in test_combined_import
_run_in_subprocess(_test_combined_import)
File "/Users/sergiitk/Development/grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 45, in _run_in_subprocess
proc.start()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/context.py", line 284, in _Popen
return Popen(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
_pickle.PicklingError: Can't pickle <function _test_combined_import at 0x10c2a0f70>: it's not the same object as __main__._test_combined_import
...
```
After
```console
$ python protoc_test.py
.......
----------------------------------------------------------------------
Ran 7 tests in 2.568s
OK
```
Closes#40253
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40253 from sergiitk:fix/macos/protoc_test 658fbecbbe3eca41249034819a2ae1a5a5c1e776
PiperOrigin-RevId: 786101961
This is a bug fix intended to unblock c-ares 1.34.5 upgrade: https://github.com/grpc/grpc/pull/39508
The bug can be seen by running any of the iomgr c-ares resolver `resolver_component_tests_runner_invoker`: //test/cpp/naming:resolver_component_tests_runner_invoker@experiment=no_event_engine_dns
In the general case, bug works like this:
1) c-ares initiates A and AAAA queries
2) c-ares opens fd 1 to some server that act as a black hole
3) query to server 1 times out and so c-ares opens fd 2 to query a new server b/c old one is not responding
c-ares 1.34.5 will internally increment a `consec_failures` field on the connection object when queries on fd 1 time out (see d3a507e920/src/lib/ares_process.c (L670))
4) answer to one of the queries comes in on fd 2
After processing the answer, c-ares [checks for connections to close out](d3a507e920/src/lib/ares_process.c (L233)) and closes connections that have `consec_failures > 0`, see d3a507e920/src/lib/ares_close_sockets.c (L119)
5) Now, c-ares will not indicate any interest in fd 1 from `ares_getsock` APIs
So, we'll now shutdown fd 1 which can have the effect of prematurely cancelling the overall request (see 06de3cc2ec/src/core/lib/event_engine/ares_resolver.cc (L590)) depending on ordering of events (reading another query's answer on fd 2 vs. when we run on_readable callback following shutdown of fd 1)
I guess this was not an issue before c-ares 1.34.5 b/c earlier c-ares did not have `consec_failures` tracking and so did not close sockets that had timed out earlier, after processing answers that came in on new fds, see 6360e96b5c/src/lib/ares_process.c (L1528)
---
Note: EE c-ares resolver does not have this early-cancellation issue b/c it shuts down no-longer-interesting fds with an OK status: 79769b35d0/src/core/lib/event_engine/ares_resolver.cc (L547)
PiperOrigin-RevId: 783612611
Change the gRPC Python Tools supported versions to match ones in
tools/distrib/python/grpcio_tools/python_version.py
Also update "Mac OS X" name to "macOS" as it is currently used by Apple.
---------
Signed-off-by: Arthit Suriyawongkul <arthit@gmail.com>
Change was created by the release automation script. See go/grpc-release.
Closes#40120
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40120 from veblush:bump_dev_version_202507041728 e403e80e566f334809e93e421d552d0b99076f68
PiperOrigin-RevId: 779336318
Update absl dependency to https://github.com/abseil/abseil-cpp/releases/tag/20250512.1
As part of this upgrade, remove gRPC's usage of internal absl helper functions.
A similar fix is needed for OpenCensus-Cpp, but since that repo is archived, I am choosing to add in a patch. The aim is to delete our OpenCensus plugin so that we don't have to maintain this forever as per https://github.com/grpc/proposal/pull/496.
Closes#39571
PiperOrigin-RevId: 777770232
Change was created by the release automation script. See go/grpc-release.
Closes#39614
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39614 from yashykt:bump_dev_version_202505211516 89b3566bfa45e56bc555a70f9a61832ba26274d5
PiperOrigin-RevId: 762218178
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#39392
PiperOrigin-RevId: 758868130
This change adds a "python-grpcio-tools-protoc" script. This complements the current way of invoking this tool, `python -m grpcio-tools.protoc`, with a CLI tool.
The main motivation is to be able to run it through [pipx](https://github.com/pypa/pipx).
Currently,
```sh
$ pipx run --spec grpcio-tools
'grpcio-tools' executable script not found in package 'grpcio-tools'.
Available executable scripts:
```
Closes#39090
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39090 from ikonst:patch-1 dbcfd8e9a626861fc50b9b7245d0dd11e3442e6f
PiperOrigin-RevId: 748390971
Change was created by the release automation script. See go/grpc-release.
Closes#39161
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39161 from XuanWang-Amos:bump_dev_version_202504041546 b481e8b65b4bd8fd25d4c966eb1e13c4798f2e43
PiperOrigin-RevId: 744818540
This PR upgrading Protobuf from v29 to v30 is mostly done mechanically except
- `-Wno-format-nonliteral` and `-Wno-language-extension-token` are added to the Windows Clang test
- `[ABSL|protobuf]_MSVC_STATIC_RUNTIME=ON` is added to `tools/run_tests/artifacts/build_artifact_protoc.bat` (Technically this is to fix the broken test on master)
Closes#38725
PiperOrigin-RevId: 734204682
With the Bazel build transitioning to BzlMod, all vendored Bazel Python targets must be removed. These targets are not available in the BCR and need be managed via a pip `requirements.bazel.txt` file. gRPC already uses this approach, so it is going to be extended to include those target. (e.g. see [test utils](e06ad82c3f/test/cpp/naming/utils/BUILD (L27-L45)) to understand how those targets are used)
Additionally, the generation of the `requirements.bazel.lock` file has been improved. Because this file is a lock file, including all transitive dependencies, manual maintenance is not managable. Its new source file, `requirements.bazel.txt` is created to list only the direct dependencies used by gRPC, along with instructions for generating the full lock file. This source file omits specific version requirements to use the latest available versions, but version constraints can be added as needed.
Closes#38692
PiperOrigin-RevId: 724427578
This is the [latest version ](4de3c74cf2) of envoy-API and made both bazel workspace and bzlmod use the same version.
Closes#38618
PiperOrigin-RevId: 721076378
This is a rebase of https://github.com/grpc/grpc/pull/32941 to fix conflicts and align to what changed in the meantime.
I'm very confused on how the tooling works here:
- no idea how to generate `tools/dockerfile/grpc_artifact_python_musllinux_1_1_aarch64/Dockerfile` so I copied the x64 one and update accordingly
- `tools/dockerfile/grpc_artifact_python_musllinux_1_1_aarch64.current_version` and `tools/dockerfile/distribtest/python_alpine_aarch64.current_version` need to be refreshed
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#38223
PiperOrigin-RevId: 720993782
- Updated Buildtools to 8.0.0
- Replaced `native.cc_proto_library` with the ones from `com_google_protobuf`. (Automatically fixed by the buildifier)
Closes#38446
PiperOrigin-RevId: 716355135
This PR makes necessary changes to Python build scripts for C++17 support, which were missed in #37919
- Fixed handling of C and C++ standard options for MSVC.
- Previously, a workaround relied on a specific method, `_compiler` found in the compiler class. [unixcompiler](ff11eed0c3/distutils/unixccompiler.py (L191)) has it but [msvccompiler](ff11eed0c3/distutils/_msvccompiler.py (L367)) doesn't. Therefore this won't work for MSVC even though it's supposed to work across all compilers.
- To resolve this, a new workaround has been implemented. This involves adding logic directly into the `_commandfile_spawn` function, which is considered the most suitable location for this fix. Although not ideal, this ensures that C and C++ standard options are managed correctly for MSVC.
- Removed `compiler_ok_with_extra_std function` as isolating C and C++ standard options is always recommended.
- Updated comments to correctly refer to C++17.
Closes#38410
PiperOrigin-RevId: 713442384
Change was created by the release automation script. See go/grpc-release.
Closes#38406
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38406 from stanley-cheung:bump_dev_version_202501080434 a3b2a3f2d996eb2dcf210c8fb03006d28223c4a9
PiperOrigin-RevId: 713317017
This bumps the minimum version of C++ to 17 from 14 per https://github.com/grpc/proposal/blob/master/L120-requiring-cpp17.md
Changes:
- Used C++17 instead of C++14 when building gRPC.
- Disabled `modernize-unary-static-assert` clang-tidy rule for the smooth transition. This rule will be re-enabled in a future update, along with a fix to address any identified issues.
- Added a post-install script for XCode/Cocoapod examples to enforce the use of C++17 across all projects, preventing build errors. (I'd like to have a better solution here but couldn't find it)
Closes#37919
PiperOrigin-RevId: 709073401
