gRPC is currently getting formatted with two different clang-format implementations, and due to some weirdness they have different include file orderings. This change introduces clang-format configuration to ensure that the two systems align - it's *highly* expected that this will need some maintenance going forward as the two systems evolve.
Closes#40905
PiperOrigin-RevId: 819606209
### Description
## Part 10 of Introducing Ruff
* In this PR - the suppression for `RET505` has been removed on the root `ruff.toml`
## Related:
* Next: #40186
* Prev: #40184
* b/423755915
Closes#40185
PiperOrigin-RevId: 810694059
Fix python import paths in bazel tests:
- `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
- `//examples/python/cancellation:test/_cancellation_example_test`
- `//examples/python/compression:test/compression_example_test`
Previously paths were set incorrectly. With bazel + python 3.11, this resulted in the following errors:
<details>
<summary>error examples for posterity</summary>
#### `//tools/distrib/python/grpcio_tools/grpc_tools/test:protoc_test`
```
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 141, in _test_syntax_errors
protos = protoc._protos("flawed.proto")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/protoc.py", line 110, in _protos
module = importlib.import_module(module_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "", line 1204, in _gcd_import
File "", line 1176, in _find_and_load
File "", line 1140, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'flawed_pb2'
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 168, in test_combined_import
_run_in_subprocess(_test_combined_import)
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3073/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.runfiles/com_github_grpc_grpc/tools/distrib/python/grpcio_tools/grpc_tools/test/protoc_test.py", line 48, in _run_in_subprocess
raise error_queue.get()
ModuleNotFoundError: No module named 'simple_pb2'
```
#### `//examples/python/cancellation:test/_cancellation_example_test`
```
test_graceful_sigint (__main__.CancellationExampleTest.test_graceful_sigint) ... Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/3027/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/cancellation/test/_cancellation_example_test.runfiles/com_github_grpc_grpc/examples/python/cancellation/server.py", line 26, in
import search
ModuleNotFoundError: No module named 'search'
```
#### `//examples/python/compression:test/compression_example_test`
```
test_compression_example (__main__.CompressionExampleTest.test_compression_example) ... Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/client.py", line 24, in
import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/server.py", line 26, in
import helloworld_pb2
ModuleNotFoundError: No module named 'helloworld_pb2'
FAIL
======================================================================
FAIL: test_compression_example (__main__.CompressionExampleTest.test_compression_example)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/.cache/bazel/_bazel_root/954bb7512d44d20015390af6e76121c6/sandbox/processwrapper-sandbox/2906/execroot/com_github_grpc_grpc/bazel-out/k8-fastbuild/bin/examples/python/compression/test/compression_example_test.runfiles/com_github_grpc_grpc/examples/python/compression/test/compression_example_test.py", line 68, in test_compression_example
self.assertEqual(0, client_return_code)
AssertionError: 0 != 1
```
</details>
The main difference is as of Python 3.11 bazel no longer appends the directory of the file to the sys.path. For example, compression test sys.path:
1. Contains `examples/python/compression/test` in 3.10
2. Doesn't contain `examples/python/compression/test` in 3.11
Closes#40273
PiperOrigin-RevId: 786308459
In preparation for migrating it from python3.7 to python3.11.
Pylint version had to be upgraded because pylint==2.2.2 doesn't support python 3.11.
The image will be updated separately to remove compiling python3.7 from source in #40139.
Pylint v2.17.7 docs: https://pylint.readthedocs.io/en/v2.17.7Closes#40169
PiperOrigin-RevId: 783026287
## Description
### Part 1 of Introducing Ruff
* This is Part 1 of introducing ruff series.
* The following PRs will remove some of the supressed rules introduced in this PR.
* This PR has all the folders which PyLint has currently.
* In the final part of these PRs, PyLint will be removed compeletly.
## Testing
CI
Closes#40160
PiperOrigin-RevId: 781990339
See https://github.com/grpc/grpc/issues/38210#issuecomment-2783948947 for context
Now that Event Engine is enabled, we no longer need the channel polling thread, so we can remove it.
Fork support (with event engine left enabled), is still broken on this PR as it is on master, but it will get cleaned up after https://github.com/grpc/grpc/pull/38980 (tested by manually merging and running fork tests)
While we're here:
- Regenerate greeter example protos
- Remove `channel_state_test.rb` in favor of `connectivity_watch_interrupted_test.rb`. The former was skipped due to https://bugs.ruby-lang.org/issues/15499, the new test changes things so that we no longer hit that bug, but still exercise the no_gvl connectivity state watch machinery.
- tweaks to fork tests (they are not running continuously but we should be able to re-enable them again after https://github.com/grpc/grpc/pull/38980 lands
Closes#39409
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39409 from apolcyn:fix_bugs 3e3ae7d81fcbd13b1d0eed6b13c7c39fe1bd1030
PiperOrigin-RevId: 766790187
[cpp/route_guide] Fix JSON minification removing spaces inside string values (#38954)
The existing JSON minification logic removes all whitespace characters using 'std::remove_if(db_.begin(), db_.end(), isspace);'. This approach inadvertently strips spaces inside JSON string values, leading to incorrect data parsing.
This commit introduces a new JSON minification function that:
- Removes spaces **outside** JSON string values.
- Preserves spaces **inside** JSON string values (`"..."`).
- Correctly handles escaped quotes (`\"`).
Fixes#38954
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
## Verification
### Before Fix
```bash
> ./cmake/build/route_guide_client
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1741496886.397564 341105 helper.cc:201] DB parsed, loaded 100 features.
-------------- GetFeature --------------
Found feature called BerkshireValleyManagementAreaTrail,Jefferson,NJ,USA at 40.9146, -74.6189
Found no feature at 0, 0
-------------- ListFeatures --------------
Looking for features between 40, -75 and 42, -73
Found feature called PatriotsPath,Mendham,NJ07945,USA at 40.7838, -74.6144
Found feature called 101NewJersey10,Whippany,NJ07981,USA at 40.8123, -74.3999
Found feature called U.S.6,Shohola,PA18458,USA at 41.3628, -74.9016
Found feature called 5ConnersRoad,Kingston,NY12401,USA at 42, -74.0371
Found feature called MidHudsonPsychiatricCenter,NewHampton,NY10958,USA at 41.4008, -74.3951
Found feature called 287FlugertownRoad,LivingstonManor,NY12758,USA at 41.9611, -74.6525
Found feature called 4001TremleyPointRoad,Linden,NJ07036,USA at 40.611, -74.2187
Found feature called 352SouthMountainRoad,Wallkill,NY12589,USA at 41.6802, -74.237
Found feature called BaileyTurnRoad,Harriman,NY10926,USA at 41.295, -74.1077
Found feature called 193-199WawayandaRoad,Hewitt,NJ07421,USA at 41.2145, -74.395
Found feature called 406-496WardAvenue,PineBush,NY12566,USA at 41.5737, -74.2848
...
```
### After Fix
```bash
> ./cmake/build/route_guide_client
WARNING: All log messages before absl::InitializeLog() is called are written to STDERR
I0000 00:00:1741496644.493263 337350 helper.cc:203] DB parsed, loaded 100 features.
-------------- GetFeature --------------
Found feature called Berkshire Valley Management Area Trail, Jefferson, NJ, USA at 40.9146, -74.6189
Found no feature at 0, 0
-------------- ListFeatures --------------
Looking for features between 40, -75 and 42, -73
Found feature called Patriots Path, Mendham, NJ 07945, USA at 40.7838, -74.6144
Found feature called 101 New Jersey 10, Whippany, NJ 07981, USA at 40.8123, -74.3999
Found feature called U.S. 6, Shohola, PA 18458, USA at 41.3628, -74.9016
Found feature called 5 Conners Road, Kingston, NY 12401, USA at 42, -74.0371
Found feature called Mid Hudson Psychiatric Center, New Hampton, NY 10958, USA at 41.4008, -74.3951
Found feature called 287 Flugertown Road, Livingston Manor, NY 12758, USA at 41.9611, -74.6525
Found feature called 4001 Tremley Point Road, Linden, NJ 07036, USA at 40.611, -74.2187
Found feature called 352 South Mountain Road, Wallkill, NY 12589, USA at 41.6802, -74.237
Found feature called Bailey Turn Road, Harriman, NY 10926, USA at 41.295, -74.1077
Found feature called 193-199 Wawayanda Road, Hewitt, NJ 07421, USA at 41.2145, -74.395
Found feature called 406-496 Ward Avenue, Pine Bush, NY 12566, USA at 41.5737, -74.2848
...
```
Closes#38955
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38955 from jiangqucheng:master 54089ec53b547f9c90e672104a668c12930f43b7
PiperOrigin-RevId: 744962894
route_guide_callback_client only works if started from the grpc/ directory. If started from other directories, it fails with the following error:
```
grpc/examples/cpp/route_guide
› ../../. /bazel-bin/examples/cpp/route_guide/route_guide_callback_client -db_path ./route_guide_db.json WARNING: All log messages before absl:: InitializeLog() is called are written to STDERR E0000 00:00:1740838254.946166 3533952 helper.cc:49] Failed to open examples/cpp/route_guide/route_guide_db.json
[1] 21202 abort ../../../bazel-bin/examples/cpp/route_guide/route_guide_callback_client
```
Add 'absl::ParseCommandLine(argc, argv)' to route_guide_callback_client so that the db_path argument is parsed.
Closes#38891
PiperOrigin-RevId: 733357751
Two changes are made to accept upcoming Protobuf v30
- Protobuf v30 will introduce a change that removes Bazel+MSVC support unless the `protobuf_allow_msvc option` is explicitly enabled. Therefore, to maintain compatibility, CI needs to define this option.
- CMake file for `protobuf/upb` now depends on the `protobuf_VERSION` variable, which is set in the main protobuf CMake file. A specific `cmake_externalproject` test includes `protobuf/upb` directly and therefore misses the opportunity to define this variable. To address this, the test now defines `protobuf_VERSION` directly.
Closes#38884
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38884 from veblush:protobuf-v30-misc fd0154452db4e21a3df0c47763529272e181bec1
PiperOrigin-RevId: 732251635
Starting with Protobuf v30, the `DebugString` output has become "keep-changing" causing our example tests, which depend on its exact output string, to fail. To ensure consistent test output, I've replaced the use of `DebugString` with direct member access & manual formatting.
Closes#38873
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38873 from veblush:protobuf-example ccda8eb9fb0c9b6e7c9ec9570e21c8bf61edb244
PiperOrigin-RevId: 731834283
- Updated Buildtools to 8.0.0
- Replaced `native.cc_proto_library` with the ones from `com_google_protobuf`. (Automatically fixed by the buildifier)
Closes#38446
PiperOrigin-RevId: 716355135
CMake needs following two changes which were missed part from https://github.com/grpc/grpc/pull/37919 to make CMake use C++17.
- **Explicitly require C++17 for gRPC targets using `cxx_std_17`**: This ensures that gRPC build targets are built with the correct C++17 support.
- **Enforce C++17 for all CMake invocations using `CMAKE_CXX_STANDARD=17`**: This is necessary due to Abseil's API changes depending on the C++ standard used. (e.g. Abseil's `absl::string_view` is implemented differently depending on the C++ standard. Before C++17, it's a distinct type. From C++17 onwards, it's simply an alias for `std::string_view`.) To maintain consistency and avoid build errors, all CMake builds (except particular build tests targetting C++20 or C++23) are now configured to use C++17.
The requirement of C++17 for building gRPC should be documented in the release notes and build instructions.
Closes#38335
PiperOrigin-RevId: 712926964
The gRPC basic turorial says to call the following command to run the server `./route_guide_server --db_path=path/to/route_guide_db.json`
But the server and client do not parse command line arguments and they always use default value `route_guide_db.json`. Thus the example only works if we run it from `examples/cpp/route_guide` dir. Otherwise we get the following error: `helper.cc:49] Failed to open route_guide_db.json
Aborted (core dumped)`
Add
`absl::ParseCommandLine(argc, argv);`
to both server and client `main` similar to `route_guide_callback_server.cc`
Closes#37857
PiperOrigin-RevId: 709154601
This bumps the minimum version of C++ to 17 from 14 per https://github.com/grpc/proposal/blob/master/L120-requiring-cpp17.md
Changes:
- Used C++17 instead of C++14 when building gRPC.
- Disabled `modernize-unary-static-assert` clang-tidy rule for the smooth transition. This rule will be re-enabled in a future update, along with a fix to address any identified issues.
- Added a post-install script for XCode/Cocoapod examples to enforce the use of C++17 across all projects, preventing build errors. (I'd like to have a better solution here but couldn't find it)
Closes#37919
PiperOrigin-RevId: 709073401
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
We already have the option to use Xds credentials based on `secure_mode`
flag, this PR add the ability to create Xds server based on the same
flag.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
To prepare for the upcoming upgrade to C++17, the following changes were made:
Increased minimum supported operating system versions:
- iOS: 11 (previously 10)
- macOS: 10.14 (previously 10.12)
- tvOS: 13.0 (previously 12.0)
In addition to this, version requirements across different projects were updated to use these for consistency.
Closes#37931
PiperOrigin-RevId: 686519641
Corrects bazel run command in README.
It does not match the bazel command in the BUILD file.
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#37664
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37664 from nolasconapoleao:fix_interceptor_readme 42d828f2ec1423b8ced0b000532efabbc1af7fde
PiperOrigin-RevId: 686223940
Continuation of #37541 but focused on C++.
Closes#37755
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37755 from NathanBaulch:typos-cpp 9a7fd9d65fa97a689d529772534df1a501031e9f
PiperOrigin-RevId: 684600898
With the CL-first approach, the docker test configs for Binder need to be deleted before the Binder code and tests themselves can be deleted in the next step. Sanity checks fail otherwise.
Closes#37862
PiperOrigin-RevId: 683691175
Fix: https://github.com/grpc/grpc/issues/37610
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
We need to create userguide examples for Xds not involving Observability and currently only the csm observability example has a docker image.
Closes#37778
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37778 from kannanjgithub:docker_xds_example_cpp bc91c8d5da04b1f6e9eef6e85c2f9a7d25fb716e
PiperOrigin-RevId: 678362755
The following files have been moved:
- src/core/lib/avl/*
- src/core/lib/backoff/*
- src/core/lib/debug/event_log*
- src/core/lib/iomgr/gethostname*
- src/core/lib/iomgr/grpc_if_nametoindex*
- src/core/lib/matchers/*
- src/core/lib/uri/* (renamed from uri_parser.* to uri.*)
- src/core/lib/gprpp/* (existing src/core/util/time.cc was renamed to gpr_time.cc to avoid conflict)
Closes#36792
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36792 from markdroth:reorg_util d4e8996f481c611ffbb06a8b04924ff81bc1bc2b
PiperOrigin-RevId: 676947640
