6837 Commits

Author SHA1 Message Date
Sergii Tkachenko
0a3681052b gRPC test DNS server: Synchronize shutdown to prevent data races
To avoid data races and use-after-free on shutdown in Python 3.13.
It seems to have worked before due to unintended guarantees of the
Python interpreter that are gone with the new version.

Uses `threading.Event` and `threading.Lock` to coordinate between the main thread handling signals and the `flush_stdout_loop` thread. The `_quit_on_signal` handler now signals the flush thread to exit, and waits for the flush thread to complete its current sleep cycle before calling `sys.exit()`. This prevents a use-after-free that shows up as a data race.

Ref b/474047558

PiperOrigin-RevId: 859182150
2026-01-21 11:15:46 -08:00
Tanvi Jagtap
01f57739a6 [PH2] Enable server_interceptors_end2end_test for PH2
PiperOrigin-RevId: 856145116
2026-01-14 04:05:09 -08:00
Tanvi Jagtap
b4ca1c97d8 [PH2] Enable client_interceptors_end2end_test for PH2
Adding PH2 experiment to the client_interceptors_end2end_test suite.

Also adding the Channel Arg wherever we missed it in the earlier PR : https://github.com/grpc/grpc/pull/41399/files

PiperOrigin-RevId: 856102643
2026-01-14 01:43:49 -08:00
Tanvi Jagtap
8a09cb0efa [PH2][Tests] Disabling Retry in multiple cpp end2end tests
PiperOrigin-RevId: 856040712
2026-01-13 22:32:34 -08:00
Michael Lumish
088436d01c [xds] Implement gRFC A101 (#41051)
This includes two major changes:

 1. An additional credentials option `sni_override` with the type `optional<string>`. If `nullopt`, it has no effect, and if set to the empty string it disables sending SNI entirely. Otherwise, the specified string will be sent.
 2. The implementation of [gRFC A101](https://github.com/grpc/proposal/blob/master/A101-SNI-setting-and-SNI-SAN-validation.md) using that new option. This includes options to set SNI and to validate SAN values against the set SNI value.

Closes #41051

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41051 from murgatroid99:xds_sni_support 6a1f8667dedc19947532720495b2932889236a12
PiperOrigin-RevId: 855765736
2026-01-13 09:48:39 -08:00
Tanvi Jagtap
1c2c9de77f [PH2][Tests] Enable time_change_test and mock_test
PiperOrigin-RevId: 855088648
2026-01-12 00:17:06 -08:00
Craig Tiller
a8b31fab81 Refactor CallTracer API for SendInitialMetadata.
This change introduces a new experiment `call_tracer_send_initial_metadata_is_an_annotation`. When enabled, the `CallTracer::RecordSendInitialMetadata` method will now record a `SendInitialMetadataAnnotation` and call a new `MutateSendInitialMetadata` method on the underlying `CallTracerInterface`.

The `CallTracerInterface` and its implementations (including XorMetrics, OpenCensus, OpenTelemetry, and test fakes) have been updated to include the new `MutateSendInitialMetadata` virtual method. The existing `RecordSendInitialMetadata` implementations are modified to check the experiment flag and delegate to `MutateSendInitialMetadata` if the experiment is active.

A new `SendInitialMetadataAnnotation` class is added, which inherits from `CallTracerAnnotationInterface::Annotation`. This annotation type is used to capture the state of the initial metadata for immutable tracing purposes.

Additionally, `ForEachKeyValue` methods are added to `MetadataInfo` and `HttpAnnotation` to facilitate iterating over metadata key-value pairs for annotation recording. The experiment configuration files are updated to include the new experiment.

PiperOrigin-RevId: 854227812
2026-01-09 09:47:03 -08:00
Tanvi Jagtap
33588a85e2 [PH2][Tests] Enabling generic_end2end_test, end2end_test
Bypassing 1 test in end2end_test because we want earlier validation of the flakes.
Fixing the bug may take about 2-4 working days. We need to check for the stability of the suite and so we need these to run on Kokoro.

Disabled retries in async_end2end_test , but not enabled the suite because of some timeout failure.

Also, refactored the code to add a new function ApplyCommonChannelArguments

PiperOrigin-RevId: 854186178
2026-01-09 07:29:45 -08:00
Tanvi Jagtap - Google LLC
1170ac2ad6 [PH2][BUILD][Trivial] (#41374)
[PH2][BUILD][Trivial]

Closes #41374

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41374 from tanvi-jagtap:2026_01_09_temp 03501d9d22d6a96d1b3e3941986239f81d6f432d
PiperOrigin-RevId: 854080754
2026-01-09 01:23:49 -08:00
Tanvi Jagtap
7f02b665ce [PH2][Test] Enable hybrid_end2end_test and raw_end2end_test
streaming_throughput_test is edited to disabled retry interceptor. streaming_throughput_test has NOT been enabled. Have to debug a flake.

PiperOrigin-RevId: 853963781
2026-01-08 18:43:07 -08:00
yuanweiz
20d4f1b233 [bzlmod] Avoid hard-coding workspace name so C++ DNS tests can build. (#41355)
Adapted from PR #40420

The original author left team and the PR couldn't be merged due to CLA check.
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #41355

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41355 from yuanweiz:workspace adb60674979a5d722f45e13e7c75291f22722f41
PiperOrigin-RevId: 853745450
2026-01-08 08:07:55 -08:00
Tanvi Jagtap
60094bd25f [PH2] Enabling client_callback_end2end_test
The full suite of 43 tests is passing with Retries disabled and PH2 on.
This is ready to be enabled on kokoro.

PiperOrigin-RevId: 853666979
2026-01-08 04:01:33 -08:00
Tanvi Jagtap
e1e3ebebf9 [PH2][Bug][Test][Tests] Adding peer_string_
This was missing in the transport.
Reference from CHTTP2 :
1. ab1d8f3c2f/src/core/ext/transport/chttp2/transport/chttp2_transport.cc (L2272)
2. ab1d8f3c2f/src/core/ext/transport/chttp2/transport/internal.h (L311)

Found while running cpp end to end test : function : CheckIsLocalhost
file : https://github.com/grpc/grpc/blob/master/test/cpp/end2end/end2end_test.cc
PiperOrigin-RevId: 853600618
2026-01-08 00:38:18 -08:00
Tanvi Jagtap - Google LLC
518f4a5ef0 [PH2][Trivial][BUILD] Adding a util file to cpp end2end folder (#41361)
[PH2][Trivial][BUILD] Adding a util file to cpp end2end folder

Adding some temporary helper functions for that.

Closes #41361

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41361 from tanvi-jagtap:2026_01_07_util_file 0702cb4958f80117f44c8bd796f58fa59ceba0c1
PiperOrigin-RevId: 853164713
2026-01-07 02:40:00 -08:00
Tanvi Jagtap
580a28f890 [PH2][AI] Adding a Gemini.md file for the cpp end to end test folder
PiperOrigin-RevId: 853114764
2026-01-07 00:13:36 -08:00
Tanvi Jagtap
1b9025f71d [PH2][Tests] Adding a few logs to help with future debugging
PiperOrigin-RevId: 853069945
2026-01-06 21:41:31 -08:00
Jordan Rupprecht
41780cddd1 Avoid multimap::find unspecified behavior (#41279)
When a `std::multimap` has multiple entries with the same key, calling `m.find(key)` returns an unspecified element.

Historically, this returns the first matching element. However, this is not guaranteed, and recent libc++ changes make this return an arbitrary element.

Using `m.equal_range(key)` is a replacement that will preserve the current behavior. The behavior of this is guaranteed to return a range of all matching elements in insertion order, and the beginning of the range is the same element as what's normally returned by `m.find(key)`.

Closes #41279

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41279 from rupprecht:multimap-find 70b116441d03eff80523e010b25336f5a75c70c2
PiperOrigin-RevId: 852844558
2026-01-06 10:44:32 -08:00
Craig Tiller
f045c84737 Rename EXPECT_DEATH tests to use a DeathTest suffix.
This change renames test suites containing EXPECT_DEATH assertions to include "DeathTest" in their name, following GoogleTest best practices for handling death tests.

PiperOrigin-RevId: 852840775
2026-01-06 10:39:03 -08:00
Tanvi Jagtap - Google LLC
a9ceb676ed [PH2][Tests] Enable cpp end2end tests that are working (#41339)
[PH2][Tests] Enable cpp end2end tests that are working

Closes #41339

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41339 from tanvi-jagtap:2026_01_04_bulk_enable_cpp d501d8c35778de3adf59b7e68e876caa7ffd5b45
PiperOrigin-RevId: 852185367
2026-01-05 01:30:13 -08:00
Tanvi Jagtap - Google LLC
5dd931da69 [PH2][Tests] Cpp end to end enabling multiple tests (#41332)
[PH2][Tests] Cpp end to end enabling multiple tests

Closes #41332

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41332 from tanvi-jagtap:2026_01_02_bulk_enable_passing_tests 4976d566fbb0942f1f03202b6f45c5c216846dba
PiperOrigin-RevId: 851511191
2026-01-02 19:20:52 -08:00
Tanvi Jagtap - Google LLC
4905ef5d27 [PH2][Tests] Cpp end to end enabling few tests (#41327)
[PH2][Tests] Cpp end to end enabling few tests

Closes #41327

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41327 from tanvi-jagtap:2025_12_30_thread_stress_test 5dd5e21feb7425436bd479bf3ca0d62f0a6b63c5
PiperOrigin-RevId: 850655928
2025-12-31 00:37:07 -08:00
Rishesh Agarwal
e96e1d0ce8 [BUILD] revert layering_check in test/cpp/end2end (#41326)
Closes #41326

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41326 from rishesh007:remove_parse_headers_2 9886d17265086ff3ec1979337250b66a6b4ba692
PiperOrigin-RevId: 850632406
2025-12-30 23:02:50 -08:00
Rishesh Agarwal
0d3ceaf900 [BUILD] revert layering_check in xds end2end test build file (#41302)
This reverts partial changes from  https://github.com/grpc/grpc/pull/41240.

As we don't want to maintain a separate copy of BUILD targets for all of the xDS protos, because
(a) this will be a maintenance burden
(b) it could lead to ODR violations if another target depends on both copies of the xDS proto targets.

Closes #41302

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41302 from rishesh007:remove_parse_headers 42134419d292d0b18ca1c1d71e7c3d1611c5115e
PiperOrigin-RevId: 850383567
2025-12-30 05:52:43 -08:00
Vignesh Babu
7d2ad5df80 [Build Fix] grpc/core/master/linux/grpc_flaky_network (#41272)
Should fix build failure: https://btx.cloud.google.com/invocations/0a1f31fb-d340-4ff5-af2c-25543f19068e/log

Closes #41272

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41272 from Vignesh2208:cbf 28a57fb83eebedce3353f59a8a0c528b58c8c24e
PiperOrigin-RevId: 846857224
2025-12-19 14:00:55 -08:00
Rishesh Agarwal
c705f05d29 Adding layering_check and parse_headers in end2end test build file (#41240)
Closes #41240

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41240 from rishesh007:layering_test_2 d9daaf9bb00fb3d6b09a12b8acca79b6e45c1873
PiperOrigin-RevId: 846532341
2025-12-18 20:52:18 -08:00
Rishesh Agarwal
09218ae889 Adding layering_check and parse_headers in test files (#41226)
Closes #41226

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41226 from rishesh007:layering_test 02e735767f22e934850d5e150c5e3d577bdfbecb
PiperOrigin-RevId: 844606259
2025-12-14 23:05:28 -08:00
Alisha Nanda
377b6759ba Implement StreamingFromClient and StreamingFromServer for callback API in gRPC benchmarks.
PiperOrigin-RevId: 843726088
2025-12-12 09:44:31 -08:00
Gregory Cooke
b87331376a [Testing] Fix spiffe portability (#41205)
Fix a few issues when build with OpenSSL versions

OpenSSL1.0.2 - copied some CRL related test code that was not valid assumptions for these tests.
OpenSSL1.1.1 - The regex is too sensitive, only do the regex check for BoringSSL
OpenSSL3 - We though the Invalid UTF8-SAN behavior should cause handshake failures for OpenSSL3 here and included different behavior, but that is still what is breaking. Let's revert that change.

Closes #41205

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41205 from gtcooke94:fix_spiffe_portability 8818df50053944444c1093bdf500944b690422d3
PiperOrigin-RevId: 842526173
2025-12-09 21:25:19 -08:00
Mark D. Roth
df27c82ab5 [LB] remove SubchannelCallTrackerInterface::Start() method (#41099)
This is needed for gRFC A105 (https://github.com/grpc/proposal/pull/516).  Specifically, see the "Interaction with xDS Circuit Breaking" section.

It's possible for an LB pick to be happening at the same time as the subchannel sees its underlying connection fail.  In this case, the picker can return a subchannel, but when the channel tries to start a call on the subchannel, the call creation fails, because there is no underlying connection.  In that case, the channel will queue the pick, on the assumption that the LB policy will soon notice that the subchannel has been disconnected and return a new picker, at which point the queued pick will be re-attempted with that new picker.

When the picker returns a complete pick, it can optionally return a `SubchannelCallTracker` object that allows it to see when the subchannel call starts and ends.  In the current API, when the channel successfully creates a call on the subchannel, it will immediately call `Start()`, and then when the subchannel call later ends, it will call `Finish()`.  However, when the race condition described above occurs, the `SubchannelCallTracker` object will be destroyed without `Start()` or `Finish()` ever having been called.  This API allows us to handle call counter incrementing and decrementing for things like xDS circuit breaking: we check the counter in the picker to see that it's currently below the limit, we increment the counter in `Start()`, and decrement it in `Finish()`.  If the subchannel call never starts, then the counter never gets incremented.

With the introduction of connection scaling functionality in the subchannel, this approach will no longer work, because the call may be queued inside of the subchannel rather than being immediately started on a connection, and the channel can't tell if that is going to happen.  In other words, there's no longer any benefit to the `Start()` method, because it will no longer actually indicate that the call is actually being started on a connection.  As a result, I am removing that method from the API.

For xDS circuit breaking in the xds_cluster_impl LB policy, we are now incrementing the call counter in the picker, and the `SubchannelCallTracker` object will decrement it when either `Finish()` is called or when the object is destroyed, whichever comes first.

For grpclb, the `Start()` method was used in an ugly hack to handle ownership of the client stats object between the grpclb policy and the client load reporting filter.  The LB policy passes a pointer to this object down to the filter via client initial metadata, which contains a raw pointer and does not hold a ref.  To handle ownership, the LB policy returns a `SubchannelCallTracker` that holds a ref to the client stats object, but when `Start()` is called, it releases that ref, on the assumption that the client load reporting filter will subsequently take ownership.  I've replaced this with a slightly cleaner approach whereby the call tracker always holds a ref to the client stats object, thus guaranteeing that the client stats object exists when the client load reporting filter sees it, and the client load reporting filter takes its own ref when it runs.  (An even cleaner approach would be to instead pass the client stats object to the filter via a call attribute, similar to how we pass the xDS cluster name from the ConfigSelector to the LB policy tree, but it doesn't seem worth putting that much effort into grpclb at this point.)

Closes #41099

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41099 from markdroth:xds_circuit_breaking_counter_change eaa06bbdf1688c31c0d1e3b3cabe6a7d015fc075
PiperOrigin-RevId: 842261731
2025-12-09 09:02:30 -08:00
Yousuk Seung
0a34a8cbc8 Skip sleuth.so and sleuth tests on Windows
PiperOrigin-RevId: 841929574
2025-12-08 15:06:45 -08:00
Chris Kennelly
b9d6903f1e Automated Code Change
PiperOrigin-RevId: 839268035
2025-12-02 07:56:20 -08:00
Alisha Nanda
8766eb4ae0 Update Sleuth version.
PiperOrigin-RevId: 838844425
2025-12-01 10:52:51 -08:00
Pawan Bhardwaj
76c81c27b0 [Cleanup] Remove workaround Apple CFStream bug from e2e tests (#41121)
<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #41121

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41121 from pawbhard:temp_check 185a3d8cc4f617d6df66d4f1adc738ef9a4b13f6
PiperOrigin-RevId: 838668301
2025-12-01 02:05:23 -08:00
Aananth V
931df0c444 Add support to export Instrument -> OpenTelemetry UpDownCounters.
PiperOrigin-RevId: 836631446
2025-11-25 05:51:46 -08:00
Yousuk Seung
8adedd8f2f Keep sleuth binary target name
PiperOrigin-RevId: 833982212
2025-11-18 14:59:13 -08:00
Akshit Patel
8e05cf46f8 [benchmark]
PiperOrigin-RevId: 833652476
2025-11-17 22:28:15 -08:00
Atan Bhardwaj
8070efca37 Refactor proto_reflection_descriptor_database in util to use absl::flat_hash_map and absl::flat_hash_set instead of std::unordered_map and std::unordered_set for potential performance improvements. This also involves including the necessary absl headers and updating the BUILD file.
Replace `.insert()` with `.emplace()` for `missing_symbols_` in
`proto_reflection_descriptor_database.cc.`

PiperOrigin-RevId: 831644658
2025-11-12 19:35:44 -08:00
Rishesh Agarwal
c411f81d38 adding address_sorting dep in naming test build (#41045)
Closes #41045

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41045 from rishesh007:add_dep_generate_resolver_component_tests 941f43f91d10ef599f16a82dfec1b3bb3f2dbe0a
PiperOrigin-RevId: 830835034
2025-11-11 03:27:04 -08:00
Rishesh Agarwal
2da858bbae Adding layering_check and parse_headers in each bazel build file
PiperOrigin-RevId: 830781168
2025-11-11 00:23:03 -08:00
Siddharth Nohria
f4742792e1 [Part 1] Resource Quota Write memory tracking.
Pass a MemoryAllocator to the serialize function, so that the write memory can be allocated towards Resource Quota accounting. Add templates for SerializationTraits, to allow implementations to continue using an implementation of Serialize, which does not take the allocator as a parameter. This change is a no-op for now, because all the callers of Serialize pass nullptr for the allocator.

PiperOrigin-RevId: 826159627
2025-10-30 13:21:27 -07:00
Rishesh Agarwal
23b094aa2b Enable layering_check and parse_headers in test/cpp/naming/BUILD.
PiperOrigin-RevId: 825919046
2025-10-30 01:33:27 -07:00
Craig Tiller
0a272b8d83 Export gRPC counters --> OTEL
This CL starts the work to export the new instrument domains system to OTEL.
Only counters are attempted at this point, the work is behind an experiment, and that experiment is disabled.

PiperOrigin-RevId: 825842970
2025-10-29 21:11:46 -07:00
Yousuk Seung
96353e6c0b Always link the sleuth library (#40970)
Closes #40970

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40970 from yousukseung:alwayslink a84e7152b7c32550ffe83fcb88cfd2d8e7f58d0f
PiperOrigin-RevId: 825579850
2025-10-29 09:26:33 -07:00
Yousuk Seung
7c1a546992 Add sleuth.so target (#40965)
This is a shared library target for gRPC Sleuth Python extension.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #40965

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40965 from yousukseung:sleuth-so b6f99b496b46e8dd599b08662370d4fdf4050b9c
PiperOrigin-RevId: 825233373
2025-10-28 15:38:22 -07:00
Rishesh Agarwal
3b48111cdc Enable layering_check and parse_headers for gRPC test targets.
PiperOrigin-RevId: 824369369
2025-10-26 23:30:20 -07:00
Yousuk Seung
3563928385 Use real Envoy service protos
PiperOrigin-RevId: 823718483
2025-10-24 17:16:51 -07:00
Yousuk Seung
67171d5e61 Move tool dependencies to the library.
PiperOrigin-RevId: 823564058
2025-10-24 09:44:54 -07:00
Yousuk Seung
1ab62ba748 Sleuth Tool Refactoring for Cython Export
- Use a print function instead of stdout/stderr
- Made existing main() into a library accessible via an API.
- Enabled multiple executions of the current main().
- Removed absl flags for better flag management when using the API.
- Consolidated SLEUTH_TOOL and absl flags (only behavioral change).

PiperOrigin-RevId: 823083768
2025-10-23 10:05:09 -07:00
Vignesh Babu
01915a021f Enhance LatentSee to support extra event data.
This change introduces the ability for LatentSee events to carry additional, type-specific data. This data is stored in a variable-sized manner within the LatentSee bins and can be serialized to JSON as part of the event's arguments. The ZTrace collector is updated to log its events using this new LatentSee extra event mechanism.

PiperOrigin-RevId: 822286249
2025-10-21 14:51:47 -07:00
Gregory Cooke
59293cb539 [Testing] Fix portability breakage due to sensitive error regex in SPIFFE tests. (#40902)
Skip the regex that is breaking in ossl3 builds

Fix invalid UTF-8 SAN test for OpenSSL 3.0

This change updates the InvalidUtf8San test case in spiffe_ssl_transport_security_test.cc to account for different behavior in OpenSSL 3.0. OpenSSL 3.0 is stricter about invalid UTF-8 in certificate Subject Alternative Names (SANs), causing the handshake to fail. The expectations for server and client success are adjusted accordingly for OpenSSL 3.0+. Additionally, EXPECT_EQ is changed to ASSERT_EQ when extracting the peer to prevent crashes if extraction fails.

Closes #40902

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40902 from gtcooke94:test_fixes e5e88926e2554716b621adaeb7e2af8cf17eea3c
PiperOrigin-RevId: 821934652
2025-10-20 21:17:42 -07:00