247 Commits

Author SHA1 Message Date
Rishesh Agarwal
09218ae889 Adding layering_check and parse_headers in test files (#41226)
Closes #41226

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/41226 from rishesh007:layering_test 02e735767f22e934850d5e150c5e3d577bdfbecb
PiperOrigin-RevId: 844606259
2025-12-14 23:05:28 -08:00
Aananth V
931df0c444 Add support to export Instrument -> OpenTelemetry UpDownCounters.
PiperOrigin-RevId: 836631446
2025-11-25 05:51:46 -08:00
Craig Tiller
0a272b8d83 Export gRPC counters --> OTEL
This CL starts the work to export the new instrument domains system to OTEL.
Only counters are attempted at this point, the work is behind an experiment, and that experiment is disabled.

PiperOrigin-RevId: 825842970
2025-10-29 21:11:46 -07:00
Craig Tiller
339906443b [clang-format] Match include file ordering to internal clang-format (#40905)
gRPC is currently getting formatted with two different clang-format implementations, and due to some weirdness they have different include file orderings. This change introduces clang-format configuration to ensure that the two systems align - it's *highly* expected that this will need some maintenance going forward as the two systems evolve.

Closes #40905

PiperOrigin-RevId: 819606209
2025-10-15 00:24:11 -07:00
Rishesh Agarwal
9d212b68c1 Update cpp/common, ext/otel, cpp/interop deps for layering_check and parse_headers
PiperOrigin-RevId: 802415268
2025-09-02 23:07:41 -07:00
Craig Tiller
d30abad3ed Introduce concrete wrapper types for call tracer interfaces.
This change adds new concrete classes (`CallSpan`, `CallTracer`, `ClientCallTracer`, `CallAttemptTracer`, `ServerCallTracer`) that wrap the existing `*Interface` types. These concrete types mirror the inheritance hierarchy of the interfaces and forward all method calls to the underlying interface pointer. This is the first step in refactoring the telemetry system to use concrete types instead of interfaces directly in the context.

PiperOrigin-RevId: 800743798
2025-08-28 22:08:47 -07:00
Mark D. Roth
5f4dec08d6 [metrics] add grpc.lb.backend_service label (#40486)
Implements [gRFC A89](https://github.com/grpc/proposal/blob/master/A89-backend-service-metric-label.md).

Closes #40486

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40486 from markdroth:metrics_backend_service_label b7486e23402d26707329db7e7f58de32d0e3bb4e
PiperOrigin-RevId: 797493482
2025-08-20 15:46:47 -07:00
Rishesh Agarwal
5453c37519 [build] Add Missing Dependencies for test/cpp/ext/filters, test/cpp/ext/gcp, test/cpp/ext/otel in Preparation for Enabling layering_check
PiperOrigin-RevId: 797264548
2025-08-20 05:01:42 -07:00
Yousuk Seung
34ebb0745d [OTel C++] Add a null-check to the text map propagator.
PiperOrigin-RevId: 795145776
2025-08-14 12:21:34 -07:00
Eugene Ostroukhov
5388bb68c1 [posix_engine] Remove PosixPollerManager (#40356)
PosixEngine now directly owns the poller, which will streamline change for poller enablement experiment.

Closes #40356

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40356 from eugeneo:no-poller-manager d067930a572330d47c0c09282629e2aff089d525
PiperOrigin-RevId: 795119886
2025-08-14 11:20:13 -07:00
Craig Tiller
806e186735 [telemetry] Prepare to introduce Concrete Types For Call Tracers
This change renames the existing interface types to have `Interface` in the name, and prepares the ground for concrete types for the call tracers.

PiperOrigin-RevId: 790173028
2025-08-02 11:42:16 -07:00
Yash Tibrewal
4c528497e9 [OTel C++] Implement retry metrics (#39195)
Implements OTel retry metrics as per https://github.com/grpc/proposal/pull/488

Closes #39195

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39195 from yashykt:AddOTelRetryMetrics b7e8b773839f6685564bf45c5c4c5eca630b8127
PiperOrigin-RevId: 787268504
2025-07-25 15:58:11 -07:00
Yash Tibrewal
6af1994cbb [OTel C++, Posix EE] Plumb TCP write timestamps and metrics to OTel tracers (#39946)
Changes -
* Amends Posix EventEngine to support `EventEngine::Endpoint::WriteEventSink`, removing support for the old and currently unused void* method of propagating timestamps.
* Modernize TracedBufferList to use C++ style
* Add tests to make sure OTel tracers receive TCP events.

The following kinds of annotations are seen -
```
E0622 20:55:48.864268  169397 otel_client_call_tracer.cc:366] 2025-06-22T20:55:48.864224659+00:00 TCP: SENDMSG byte_offset=311 delivery_rate=1170285714, is_delivery_rate_app_limited=1, packet_retx=0, packet_spurious_retx=0, packet_sent=2, data_retx=0, data_sent=91, data_notsent=0, pacing_rate=15697245508, min_rtt=21, congestion_window=10, reordering=3, recurring_retrans=0, busy_usec=0, rwnd_limited_usec=0, sndbuf_limited_usec=0
E0622 20:55:48.864639  169405 otel_client_call_tracer.cc:366] 2025-06-22T20:55:48.864397339+00:00 TCP: SCHEDULED byte_offset=311 delivery_rate=1170285714, is_delivery_rate_app_limited=1, packet_retx=0, packet_spurious_retx=0, packet_sent=3, packet_delivered=3, packet_delivered_ce=0, data_retx=0, data_sent=415, data_notsent=324, pacing_rate=15697245508, min_rtt=21, srtt=41, congestion_window=10, snd_ssthresh=2147483647, reordering=3, recurring_retrans=0, busy_usec=0, rwnd_limited_usec=0, sndbuf_limited_usec=0
E0622 20:55:48.864828  169405 otel_client_call_tracer.cc:366] 2025-06-22T20:55:48.86440637+00:00 TCP: SENT byte_offset=311 delivery_rate=1170285714, is_delivery_rate_app_limited=1, packet_retx=0, packet_spurious_retx=0, packet_sent=3, packet_delivered=3, packet_delivered_ce=0, data_retx=0, data_sent=415, data_notsent=324, pacing_rate=15697245508, min_rtt=21, srtt=41, congestion_window=10, snd_ssthresh=2147483647, reordering=3, recurring_retrans=0, busy_usec=0, rwnd_limited_usec=0, sndbuf_limited_usec=0
E0622 20:55:48.865587  169397 otel_client_call_tracer.cc:366] 2025-06-22T20:55:48.86543777+00:00 TCP: ACKED byte_offset=311 delivery_rate=1170285714, is_delivery_rate_app_limited=1, packet_retx=0, packet_spurious_retx=0, packet_sent=3, packet_delivered=4, packet_delivered_ce=0, data_retx=0, data_sent=415, data_notsent=0, pacing_rate=15697245508, min_rtt=21, srtt=41, congestion_window=10, snd_ssthresh=2147483647, reordering=3, recurring_retrans=0, busy_usec=0, rwnd_limited_usec=0, sndbuf_limited_usec=0
```

Closes #39946

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39946 from yashykt:EEPosixWritSink 96ea734c290e81a88d7d0c81c767ff354d1b9f7f
PiperOrigin-RevId: 775515293
2025-06-24 22:05:47 -07:00
Yash Tibrewal
0591b63d39 [CSM] Potentially help with flakiness (#39887)
Might help with flakes like -
https://btx.cloud.google.com/invocations/05b89b5c-a41a-471a-ada1-0402308a3e3e/targets/%2F%2Ftest%2Fcpp%2Fext%2Fcsm:metadata_exchange_test;config=a0aefead4af167db70a834e9bb987d3d114cfb740cb1aa85e472d8c51f5a69a6/tests

```
[test/cpp/ext/csm/metadata_exchange_test](http://test/cpp/ext/csm/metadata_exchange_test).cc:316
Expected equality of these values:
  data[kMetricName].size()
    Which is: 0
  1
```

Closes #39887

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39887 from yashykt:CsmPotentialFix 19493ec95b8974078cac00a3e23048fdc8590d20
PiperOrigin-RevId: 772201958
2025-06-16 15:52:44 -07:00
Yash Tibrewal
255938a7dd [OTel C++] Add unimplemented tracing methods (#39630)
Closes #39630

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39630 from yashykt:OTelTracer 4e8c7eb78f7036e62d161498169c3ffe38a07144
PiperOrigin-RevId: 762115094
2025-05-22 14:01:24 -07:00
Yash Tibrewal
66ef4ca71d Stabilize call tracer transport fix experiment (#39465)
Closes #39465

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39465 from yashykt:RemoveCallTracerTransportFix 2f338a53b0296208faeb3a31fb2efc5f494a6b0f
PiperOrigin-RevId: 755058234
2025-05-05 13:46:44 -07:00
Yash Tibrewal
649c4abfec [opencensus] Reduce flakiness in tracing tests (#39393)
Sample flakiness - https://btx.cloud.google.com/invocations/d8b31680-e8ad-490d-937d-d95d88f9b0cd/targets/%2F%2Ftest%2Fcpp%2Fext%2Ffilters%2Fcensus:grpc_opencensus_plugin_test;config=c765e2a5ffb8f9c148ecf742800b5eab4fa68cc283a63446e7cbde8ba0f03a13/log

Closes #39393

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39393 from yashykt:OpenCensusTestFlakiness 525cc846f710723aa75521ac73cf78af09f29703
PiperOrigin-RevId: 752361789
2025-04-28 10:36:55 -07:00
Craig Tiller
82bdf31cfa Switch chaotic-good to new style endpoint-transport creation
Leverage the work done in https://github.com/grpc/grpc/pull/39294 and https://github.com/grpc/grpc/pull/39252 to allow chaotic-good to be instantiated with regular credentials.

PiperOrigin-RevId: 750635528
2025-04-23 10:15:10 -07:00
Craig Tiller
2e10452ffb Split CoreConfiguration builders into two buckets: ephemeral and persistent.
We use builders for a variety of tasks - sometimes for testing, sometimes for adapting gRPC to novel environments, sometimes for once off special tasks early in a process lifetime.

Treating these as the same thing has started leading to some problems, so here I introduce a reset-proof registration for a subset of these tasks.

I'll be using this to register chaotic-good into the EndpointTransport registry so that it doesn't need to join the usual core configuration chain (and accidentally pull in protobuf to core).

PiperOrigin-RevId: 750048742
2025-04-21 22:38:04 -07:00
Yash Tibrewal
43272155f2 [OTel C++] Improve logging for tests (#39246)
Closes #39246

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39246 from yashykt:OTelTestFormat 3055cffd2fbbec5adca5004114752d25b3a71359
PiperOrigin-RevId: 748106709
2025-04-15 19:07:50 -07:00
Yash Tibrewal
2ec4d61d36 [OTel C++] Reorganize tests (#39215)
Closes #39215

PiperOrigin-RevId: 746275445
2025-04-10 20:18:02 -07:00
Mark D. Roth
0bf60a425e [metrics] pass StatsPluginGroup down via channel args (#39130)
Closes #39130

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/39130 from markdroth:stats_plugin_group_propagation 82b43c6adc1f007fe213cdf3d15b7647a3e39383
PiperOrigin-RevId: 743240083
2025-04-02 12:39:19 -07:00
Yash Tibrewal
751b39e1ac [otel] Remove broken internally tag on otel_tracing_test
PiperOrigin-RevId: 741547293
2025-03-28 09:20:22 -07:00
Yash Tibrewal
12278ba190 [OTel/CSM] Tags cleanup
PiperOrigin-RevId: 741436567
2025-03-28 01:25:26 -07:00
Yash Tibrewal
07602cfb72 [otel] Status check in otel_plugin_test
PiperOrigin-RevId: 741331459
2025-03-27 17:12:57 -07:00
Mark D. Roth
32881b9652 [reorg] move metadata, message, and interception_chain to src/core/call (#38972)
Closes #38972

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38972 from markdroth:reorg_call2 86686b70242c935076359c64123c47e707073aca
PiperOrigin-RevId: 737014454
2025-03-14 16:04:43 -07:00
Craig Tiller
48274038e2 Use tagging to flag broken internal libraries
PiperOrigin-RevId: 733145986
2025-03-03 19:34:07 -08:00
Yash Tibrewal
ac0eed3604 [OTel C++] Reland OTel C++ Tracing (#38852)
Reland https://github.com/grpc/grpc/pull/38437. Contains fix for the TransparentRetry flaky test. Verified with 10000 runs.

Closes #38852

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38852 from yashykt:RelandOTelTracing 4935227a4086bac22d3ca3f1ced4e309b3153792
PiperOrigin-RevId: 732160697
2025-02-28 09:19:18 -08:00
Craig Tiller
3e5854120d Tag some internal build properties
PiperOrigin-RevId: 731528957
2025-02-26 18:28:53 -08:00
Craig Tiller
a0ff460d2d Tag some targets that have internal build issues
PiperOrigin-RevId: 731520925
2025-02-26 17:56:54 -08:00
Craig Tiller
684662351a Tag some internal build state
PiperOrigin-RevId: 731469422
2025-02-26 15:08:31 -08:00
Craig Tiller
c35ad2f301 Tag some internal build state
PiperOrigin-RevId: 731468925
2025-02-26 15:07:09 -08:00
Yash Tibrewal
ca0d85cea5 Revert "[OTel C++] Add tracing support (#38437)" (#38821)
This reverts commit a6eb776b48.

Looks like the TransparentRetries test is flaky

Closes #38821

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38821 from yashykt:RevertOTelTracing 74abeb889bbf2fa3d0a4782ada1d19e7255e37e1
PiperOrigin-RevId: 730800878
2025-02-25 02:45:37 -08:00
Yash Tibrewal
a6eb776b48 [OTel C++] Add tracing support (#38437)
As per [gRFC A72](https://github.com/grpc/proposal/blob/master/A72-open-telemetry-tracing.md).

Closes #38437

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38437 from yashykt:OpenTelemetryTracing 30df7daae338d93f975d28931c0a005fae74d762
PiperOrigin-RevId: 730366527
2025-02-24 02:42:49 -08:00
Yash Tibrewal
211f9ecc04 [CallAttemptTracer] Fix call attempt tracer lifetimes for retries (#38729)
Fix #38728 heap-use-after-free. Details in the issue.

Also, fix a bug in chttp2 where we are using the parent call tracer instead of the call attempt tracer to record annotations for a stream.

Test is being added in https://github.com/grpc/grpc/pull/38437

Closes #38729

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38729 from yashykt:FixCallAttemptTracer cb09add0b2f257143cfd60efb82663d59d92835a
PiperOrigin-RevId: 729307540
2025-02-20 17:16:16 -08:00
Vignesh Babu
dbb582a7e0 [Experiments] Remove unused experiments and experiments which are aleady default set to true
PiperOrigin-RevId: 728780597
2025-02-19 12:54:42 -08:00
Yash Tibrewal
c637de558a [Cleanup] Avoid std::make_pair (#38636)
Maybe in some of these cases, `{}` would work instead of `std::pair`, but I'm just doing a simple find and replace here.

Closes #38636

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38636 from yashykt:NoMakePair d819f6a74be2fb2859083436297f79ed3139a1b7
PiperOrigin-RevId: 722772621
2025-02-03 13:19:08 -08:00
Craig Tiller
725d1eea73 [build] Remove language tag from BUILD files (#38501)
This is only ever set to C++, so removing it from bazel.

There are remnants left in cmakefile generation that I'm not planning on cleaning up (looked like some of the yaml/python pieces still use this)

Closes #38501

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38501 from ctiller:language c8aee06a01a449483b7585a367ff793fde47f68a
PiperOrigin-RevId: 718198572
2025-01-21 21:13:44 -08:00
AJ Heller
478b4c0fba [C++17] Replace absl::optional with std::optional (#38420)
Closes #38420

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38420 from drfloob:cpp17-optional 7f276aaa3c493a2ea7f4f1766da975e083914a5b
PiperOrigin-RevId: 715081837
2025-01-13 13:13:38 -08:00
AJ Heller
bd13a27610 [C++17] Replace absl::variant with std::variant (#38419)
absl::variant is a C++11 compatible shim for C++17's std::variant. See 2f016c4575/absl/types/variant.h (L38-L40).

Closes #38419

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38419 from drfloob:cpp17-variant 914ffd48413138458097b4726f89a58a48717154
PiperOrigin-RevId: 713817791
2025-01-09 15:38:32 -08:00
Esun Kim
8bf7149277 [Test] Fix logging test (#38232)
Number 25 doesn't have a corresponding enum code which can cause a problem when trying to use `absl::Status` for `grpc::Status`. Updated this test code to use 11 (kOutOfRange) instead.

Closes #38232

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38232 from veblush:status-logging c72414105392cde7d09d9045d23516b2b41aeab4
PiperOrigin-RevId: 703181134
2024-12-05 11:16:42 -08:00
Vignesh Babu
cccd74cf4e [StatsPlugingTest] Fix flakiness by increasing sleep duration in test (#38190)
Fixes flakiness: https://btx.cloud.google.com/invocations/593adc43-f0fc-4c3a-8a1b-3be4e47f439d/targets/%2F%2Ftest%2Fcpp%2Fext%2Ffilters%2Fcensus:grpc_opencensus_plugin_test@poller%3Depoll1;config=73d114f34497fe815dc9359619f4998ad393f9a101551e59d2a81873570c0afd/log

Closes #38190

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38190 from Vignesh2208:flaky-stats-plugin f3c5c22e994da660c2225287ed77756286d13f84
PiperOrigin-RevId: 700086889
2024-11-25 13:45:11 -08:00
Mark D. Roth
7570d8b545 [reorg] move src/core/lib/config -> src/core/config (#37847)
Closes #37847

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37847 from markdroth:reorg_config 191c5460bc2b9c974bf76a3eedc876963d296f47
PiperOrigin-RevId: 698957093
2024-11-22 05:46:04 +00:00
Mark D. Roth
55b1ae9700 [build] replace grpc_proto_library rules with separate proto_library/cc_proto_library/cc_grpc_library rules (#37943)
Closes #37943

PiperOrigin-RevId: 691538475
2024-10-30 13:45:41 -07:00
Yash Tibrewal
3d63a7b106 [CSM O11y] Change how mesh_id is populated (#37801)
Instead of getting value of `csm_mesh_id` from the bootstrap file, get it from the env var `CSM_MESH_ID`

Closes #37801

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37801 from yashykt:CsmMeshIdChange d0f149e02363350a3c0a43ddaf5cc872638b9067
PiperOrigin-RevId: 685864223
2024-10-14 16:01:07 -07:00
Yash Tibrewal
e35831a5a8 [logging_test] Fix timeout (#37874)
Closes #37874

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37874 from yashykt:FixLoggingTest fa5d67230cafa480eadc0d7a6e71d7dbb62ebd63
PiperOrigin-RevId: 684201591
2024-10-09 16:01:42 -07:00
Craig Tiller
dbb5164ac7 [clang-format] Remove custom clang-format rules for include ordering (#37820)
Closes #37820

PiperOrigin-RevId: 682352913
2024-10-04 09:44:20 -07:00
Yijie Ma
c8765eb9c4 [OTel plugin] Do not crash when the gauge in AddCallback is different from the gauge in Report (#37764)
Log error message instead of crashing for this API misuse.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #37764

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37764 from yijiem:report-different-gauge-wont-crash 1b6e912bfc65c44b754e9fa143f8c22e15ba49ef
PiperOrigin-RevId: 677944595
2024-09-23 14:42:51 -07:00
Mark D. Roth
f6c57b6384 [reorg] move a bunch of stuff to src/core/util (#36792)
The following files have been moved:
- src/core/lib/avl/*
- src/core/lib/backoff/*
- src/core/lib/debug/event_log*
- src/core/lib/iomgr/gethostname*
- src/core/lib/iomgr/grpc_if_nametoindex*
- src/core/lib/matchers/*
- src/core/lib/uri/* (renamed from uri_parser.* to uri.*)
- src/core/lib/gprpp/* (existing src/core/util/time.cc was renamed to gpr_time.cc to avoid conflict)

Closes #36792

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36792 from markdroth:reorg_util d4e8996f481c611ffbb06a8b04924ff81bc1bc2b
PiperOrigin-RevId: 676947640
2024-09-20 13:19:02 -07:00
Yash Tibrewal
605a15e7eb [OTel C++] Fix race when adding and removing callbacks (#37485)
Split off from https://github.com/grpc/grpc/pull/37425

We are adding and removing callbacks on the OpenTelemetry Async Instruments without synchronization. This opens us to races where we have an AddCallback and RemoveCallback operation happening at the same time. The correct result after these operations is to still have a callback registered with OpenTelemetry at the end, but the two operations could race and we could just decide to remove the OpenTelemetry callback.

The fix delays removing OpenTelemetry callbacks to plugin destruction time.

Closes #37485

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37485 from yashykt:FixRaceOTelGauge 016b0a41b54102c69e6828425c014ada2b066af2
PiperOrigin-RevId: 663492598
2024-08-15 16:22:51 -07:00