Commit Graph

17 Commits

Author SHA1 Message Date
Sergii Tkachenko
0497da859e [CI] Use python 3.9+ everywhere (#40139)
- Upgrade the base image of  https://us-docker.pkg.dev/grpc-testing/testing-images-public/bazel to newer https://gcr.io/oss-fuzz-base/base-builder@sha256:4f3ca10accd14292556601d70e457fa85ad57180c913484427869d3379a07684, which comes with python3.11

  Before:
  ```

  Python 3.8.3 (default, Mar 12 2024, 03:21:22)
  [Clang 15.0.0 (https://github.com/llvm/llvm-project.git bf7f8d6fa6f460bf0a16ffe
  ```

  After:
  ```
  Python 3.11.13 (main, Jul 16 2025, 03:42:11)
  [Clang 18.1.8 (https://github.com/llvm/llvm-project.git 3b5b5c1ec4a3095ab096dd78
  ```
- Refactor and improve  https://us-docker.pkg.dev/grpc-testing/testing-images-public/sanity. Removed building python 3.7 from source.
- Switch Windows builds to use `3.9.13` (previously `3.8.10`)
- Changed minimum python version in bazel builds from 3.8 to 3.9.
- Updated pylint to use python3.11.
- Various minor compatibility fixes.

There's a few other dependencies/settings that can be upgraded now that we're using 3.9+. I'll make them in a follow-up PR.

Closes #40139

PiperOrigin-RevId: 786775922
2025-07-24 11:31:19 -07:00
Sergii Tkachenko
6c7e2a94f9 [Python] Fix bazel tests compatibility with python 3.11 (#40273)
Fix python import paths in bazel tests:
- `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
- `//examples/python/cancellation:test/_cancellation_example_test`
- `//examples/python/compression:test/compression_example_test`

Previously paths were set incorrectly. With bazel + python 3.11, this resulted in the following errors:

<details>
<summary>error examples for posterity</summary>

#### `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
```
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 141, in _test_syntax_errors
    protos = protoc._protos("flawed.proto")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/protoc.py", line 110, in _protos
    module = importlib.import_module(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "", line 1204, in _gcd_import
  File "", line 1176, in _find_and_load
  File "", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'flawed_pb2'

Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 168, in test_combined_import
    _run_in_subprocess(_test_combined_import)
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 48, in _run_in_subprocess
    raise error_queue.get()
ModuleNotFoundError: No module named 'simple_pb2'
```

#### `//examples/python/cancellation:test/_cancellation_example_test`
```
test_graceful_sigint (__main__.CancellationExampleTest.test_graceful_sigint) ... Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3027/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/cancellation/test/_cancellation_example_test.runfiles/com_github_grpc_grpc/examples/python/cancellation/server.py", line 26, in
    import search
ModuleNotFoundError: No module named 'search'
```

#### `//examples/python/compression:test/compression_example_test`
```
test_compression_example (__main__.CompressionExampleTest.test_compression_example) ... Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/client.py", line 24, in
    import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/server.py", line 26, in
    import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
FAIL

======================================================================
FAIL: test_compression_example (__main__.CompressionExampleTest.test_compression_example)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/test/compression_example_test.py", line 68, in test_compression_example
    self.assertEqual(0, client_return_code)
AssertionError: 0 != 1
```
</details>

The main difference is as of Python 3.11 bazel no longer appends the directory of the file to the sys.path. For example, compression test sys.path:

1. Contains `examples/python/compression/test` in 3.10
2. Doesn't contain `examples/python/compression/test` in 3.11

Closes #40273

PiperOrigin-RevId: 786308459
2025-07-23 09:21:50 -07:00
Marcel
ff7d726a91 [Python Bazel] Use PyInfo provider and py_* rules from rules_python (#37995)
The native one will be removed soon (Bazel 8?).

Closes #37995

PiperOrigin-RevId: 690671136
2024-10-28 10:47:26 -07:00
Xuan Wang
49d48c94ab [Python Example] Fix compression example (#34745)
Fix: https://github.com/grpc/grpc/issues/34690

Also added generated proto files to example folder so we can run example
directly.
<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-10-20 09:10:14 -07:00
Sergii Tkachenko
de6ed9ba9f [Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00
Lidi Zheng
2231c2ba77 Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
Esun Kim
e0ebe46ad3 Fix python examples 2020-12-23 09:23:32 -08:00
Esun Kim
e7afadc255 Refactor examples BUILD files 2020-12-23 09:23:32 -08:00
Esun Kim
7be821ac47 Yapf all target python sources 2020-01-03 09:25:10 -08:00
Lidi Zheng
c34bef4ca4 Format all Bazel files with buildifier 2019-11-05 14:03:52 -08:00
Richard Belleville
1444cd1dd3 Revert "Revert "Merge pull request #20097 from gnossen/dual_version_python_tests""
This reverts commit 24c562dbaa.
2019-08-29 10:51:36 -07:00
Richard Belleville
24c562dbaa Revert "Merge pull request #20097 from gnossen/dual_version_python_tests"
This reverts commit c9c847f334, reversing
changes made to 07ba4de392.
2019-08-29 10:47:44 -07:00
Richard Belleville
27990a5541 Explicitly add python versions to examples. 2019-08-27 17:47:52 -07:00
Richard Belleville
d932d2368e Merge pull request #19852 from gnossen/server_termination
Quit waiting for a day
2019-08-08 09:53:10 -07:00
Richard Belleville
e32980fba2 Quit waiting for a day 2019-08-05 16:19:49 -07:00
Richard Belleville
2b7ec3ad23 Fix up examples. 2019-08-01 16:18:45 -07:00
Richard Belleville
624839b704 Add example Python server using compression. 2019-06-21 13:59:01 -07:00