53 Commits

Author SHA1 Message Date
Sergii Tkachenko
6c7e2a94f9 [Python] Fix bazel tests compatibility with python 3.11 (#40273)
Fix python import paths in bazel tests:
- `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
- `//examples/python/cancellation:test/_cancellation_example_test`
- `//examples/python/compression:test/compression_example_test`

Previously paths were set incorrectly. With bazel + python 3.11, this resulted in the following errors:

<details>
<summary>error examples for posterity</summary>

#### `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
```
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 141, in _test_syntax_errors
    protos = protoc._protos("flawed.proto")
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/protoc.py", line 110, in _protos
    module = importlib.import_module(module_name)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "", line 1204, in _gcd_import
  File "", line 1176, in _find_and_load
  File "", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'flawed_pb2'

Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 168, in test_combined_import
    _run_in_subprocess(_test_combined_import)
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 48, in _run_in_subprocess
    raise error_queue.get()
ModuleNotFoundError: No module named 'simple_pb2'
```

#### `//examples/python/cancellation:test/_cancellation_example_test`
```
test_graceful_sigint (__main__.CancellationExampleTest.test_graceful_sigint) ... Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3027/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/cancellation/test/_cancellation_example_test.runfiles/com_github_grpc_grpc/examples/python/cancellation/server.py", line 26, in
    import search
ModuleNotFoundError: No module named 'search'
```

#### `//examples/python/compression:test/compression_example_test`
```
test_compression_example (__main__.CompressionExampleTest.test_compression_example) ... Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/client.py", line 24, in
    import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/server.py", line 26, in
    import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
FAIL

======================================================================
FAIL: test_compression_example (__main__.CompressionExampleTest.test_compression_example)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/test/compression_example_test.py", line 68, in test_compression_example
    self.assertEqual(0, client_return_code)
AssertionError: 0 != 1
```
</details>

The main difference is as of Python 3.11 bazel no longer appends the directory of the file to the sys.path. For example, compression test sys.path:

1. Contains `examples/python/compression/test` in 3.10
2. Doesn't contain `examples/python/compression/test` in 3.11

Closes #40273

PiperOrigin-RevId: 786308459
2025-07-23 09:21:50 -07:00
Nathan Baulch
8342a109ae Fix python typos (#38029)
Continuation of #37541 but focused on Python.

Top typos are:
* resolveable (5)
* dedecated (4)
* everyting (4)
* programatically (3)
* supercede (3)
* wil (3)
* acknowledgment (2)
* corutine (2)
* coversion (2)
* operaton (2)
* wich (2)

Closes #38029

PiperOrigin-RevId: 695759207
2024-11-12 09:15:38 -08:00
Marcel
ff7d726a91 [Python Bazel] Use PyInfo provider and py_* rules from rules_python (#37995)
The native one will be removed soon (Bazel 8?).

Closes #37995

PiperOrigin-RevId: 690671136
2024-10-28 10:47:26 -07:00
Sergii Tkachenko
de6ed9ba9f [Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00
Xuan Wang
149f5615ba Remove enum and future (#31381)
* [Python][Remove enum34] Remove enum34

* [Python][Remove futures] Remove futures

* Remove empty deps from .bazel files
2022-10-18 14:58:30 -07:00
Xuan Wang
7aca6d4007 [Remove Six] Remove dependency on six (#31340)
* [Python][Remove Six] Remove Six from requirements
2022-10-17 09:34:45 -07:00
Richard Belleville
468768865b Make Buildifier Sanity Test Strict (#27807)
* Fix all lint errors in repo.

* Use strict buildifier by default

* Whoops. That file does not exist

* Attempt fix to buildifier invocation

* Add missing copyright
2021-11-03 14:57:04 -07:00
Richard Belleville
a48067c4c8 Replace StopIteration with return (#26861) 2021-08-03 11:08:31 -07:00
Lidi Zheng
2231c2ba77 Introduce Python import sorting to our sanity test suite (#26768)
* Add isort_code.sh to sanity tests

* Run tools/distrib/isort_code.sh

* Fine tune the import order for relative imports

* Make pylint and project generation happy

* Fix a few corner cases

* Use --check instead of --diff

* The import order impacts test result somehow

* Make isort print diff and check output at the same time

* Let tools/run_tests/python_utils be firstparty library

* Run isort against latest HEAD
2021-07-26 12:31:21 -07:00
Yannic Bonenberger
06ecc87290 [bazel] Add fixes for --incompatible_load_proto_rules_from_bzl 2020-02-05 19:58:30 +01:00
Jan Tattermusch
27cdec0041 buildifier format 2020-01-15 04:43:59 -05:00
Richard Belleville
69f990673f Conform to protobuf's usage of six 2020-01-15 04:43:59 -05:00
Esun Kim
7be821ac47 Yapf all target python sources 2020-01-03 09:25:10 -08:00
Lidi Zheng
c34bef4ca4 Format all Bazel files with buildifier 2019-11-05 14:03:52 -08:00
chentanjun
4ecc1fe6a4 fix the wrong word 2019-10-01 05:13:33 +08:00
Richard Belleville
1444cd1dd3 Revert "Revert "Merge pull request #20097 from gnossen/dual_version_python_tests""
This reverts commit 24c562dbaa.
2019-08-29 10:51:36 -07:00
Richard Belleville
24c562dbaa Revert "Merge pull request #20097 from gnossen/dual_version_python_tests"
This reverts commit c9c847f334, reversing
changes made to 07ba4de392.
2019-08-29 10:47:44 -07:00
Richard Belleville
27990a5541 Explicitly add python versions to examples. 2019-08-27 17:47:52 -07:00
Richard Belleville
ba04bafede And the examples directory 2019-08-21 14:39:54 -07:00
Richard Belleville
d932d2368e Merge pull request #19852 from gnossen/server_termination
Quit waiting for a day
2019-08-08 09:53:10 -07:00
Richard Belleville
e32980fba2 Quit waiting for a day 2019-08-05 16:19:49 -07:00
Richard Belleville
66114201b4 Modernize cancellation example 2019-08-02 15:11:06 -07:00
Richard Belleville
805afe647d Copyright typo 2019-07-08 11:42:21 -07:00
Richard Belleville
79e78d16f7 Pylint 2019-07-08 11:09:44 -07:00
Richard Belleville
ba39c9255c Adopt reviewer's comments 2019-07-08 10:00:36 -07:00
Richard Belleville
915e97b115 Fix main thread starvation issues 2019-06-26 13:16:02 -07:00
Richard Belleville
1db141accc Change section title 2019-06-26 09:51:44 -07:00
Richard Belleville
25f3439c91 Make whole package testonly 2019-06-26 09:45:47 -07:00
Richard Belleville
7486026eb9 Annotate the proto file 2019-06-26 09:42:55 -07:00
Richard Belleville
4100084c78 Use six for compatibility in client 2019-06-26 09:37:33 -07:00
Richard Belleville
7fa7f932e3 Pull search algorithm out into another module 2019-06-26 09:32:58 -07:00
Richard Belleville
42b2fe154a Simplify search implementation 2019-06-26 09:21:29 -07:00
Richard Belleville
2bf4d502c1 Factor out simpler generator 2019-06-26 09:06:07 -07:00
Richard Belleville
fed1c629e0 Make compatible with Python 3 2019-06-25 16:07:49 -07:00
Richard Belleville
81f42031c6 Pylint 2019-06-25 15:58:06 -07:00
Richard Belleville
edbddf25ab Typos 2019-06-25 15:49:31 -07:00
Richard Belleville
786a3acab0 Add test 2019-06-25 14:20:07 -07:00
Richard Belleville
93d6344ac6 Add todo 2019-06-25 13:25:54 -07:00
Richard Belleville
dc8dba8afe Add docstrings 2019-06-25 13:19:02 -07:00
Richard Belleville
8f1bfdab55 Yapf 2019-06-25 13:01:22 -07:00
Richard Belleville
b12299701d Clean up logging 2019-06-25 13:00:09 -07:00
Richard Belleville
4c852bf25f Cancel RPCs after a hash limit has been reached 2019-06-25 12:50:10 -07:00
Richard Belleville
b9cc2c210f Explain how we take care of servicer threads 2019-06-25 11:15:43 -07:00
Richard Belleville
cdae8ca6ad Add intro about algorithm 2019-06-25 11:11:03 -07:00
Richard Belleville
82aa4068c7 Elaborate on cancelling streaming RPCs 2019-06-25 11:02:10 -07:00
Richard Belleville
4ee154dd53 Elaborate on unary cancellation 2019-06-25 10:50:04 -07:00
Richard Belleville
c9e83db6bc Implement streaming on the client side 2019-06-25 10:34:48 -07:00
Richard Belleville
b6a5e94f71 Respond to ctrl+c on client side 2019-06-25 09:36:49 -07:00
Richard Belleville
244279cb36 Add client CLI 2019-06-25 09:07:46 -07:00
Richard Belleville
b31431aea3 Switch over to a generator 2019-06-24 16:49:35 -07:00