[Gpr_To_Absl_Logging] Remove gpr_log_severity_string
This function is not getting used anymore gpr_log_severity_string
Closes#37013
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/37013 from tanvi-jagtap:gpr_log_severity_string_remove 3346face4c1b5262e3af8f394f3c75729391ad6b
PiperOrigin-RevId: 645796077
As title
`_pb.rb` protos regenerated using `grpc-tools` 1.64, which has a more recent `protoc`. (otherwise tests fail b/c we can't load the older generated code with google-protobuf 4.x - also see https://github.com/grpc/grpc/pull/36900#issuecomment-2163871696
cc @dazuma @alto-ruby
Closes#36982
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36982 from apolcyn:update_dep 84de585b421c9be5ff9f1f736278d2f82eb9c764
PiperOrigin-RevId: 645189362
This commit implements `logger=` for GRPC module. It will allow us to inject a custom logger on the fly easily. Using `self.extend` inside this setter will help us deal with backward compatible for some custom Logger Module.
<!--
Your pull request will be routed to the following person by default for triaging.
If you know who should review your pull request, please remove the mentioning below.
-->
Fixed#24084
cc @sampajano , @apolcyn and @alto-ruby TIA
re: https://github.com/grpc/grpc/pull/24072Closes#36989
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36989 from nvh0412:chore/add-logger-setter-for-GRPC-ruby-2 f5a91225e7e1b08cfd7b99e2ed8231f0cf3d75a7
PiperOrigin-RevId: 645189146
### Problem 1
Context :
gpr_log() internally calls gpr_log_message() .
gpr_log_message() may either call gpr_default_log() or a user provided logging function.
gpr_default_log() uses absl LOG. This is the default. Most users will log this way.
For the small percentage of users who have customized the logging function, gpr_log will log to custom this function.
Problem :
We have converted half the instances of gpr_log to absl LOG().
For users who use the defaults - there will be no issue.
For the users who use a customized logging function
1. All the absl LOGs will log to the absl log sink.
2. All the gpr_log statements will log via this user provided function.
This is in-consistent behaviour and will cause confusion and difficulty in debugging.
Solution:
All logs should go to the same sink.
So we decided to make gpr_set_log_function a no op in this release.
The function will be deleted in the next release.
https://github.com/grpc/proposal/pull/425
### Problem 2
Context :
gpr_should_log is used to avoid computing expensive stuff for logging if the log is not going to be visible.
Problem :
gpr_should_log was referencing the GRPC_VERBOSITY flag and values set by gpr_set_log_verbosity .
However, actual logging happens based on the absl settings.
This is incorrect. Because if the old settings are not honoured, they should not be checked and no decision in code should be made based on settings which are not going to get used.
Solution :
Given the above changes in Problem 1, since all custom logging is disabled, all logging from gRPC with honour the absl LOG settings. Hence we modified the gpr_should_log function to refer to absl settings.
### Problem 3
We still have the issue of php using a custom log sink. We will address this in a separate PR.
### Problem 4
Tests related to test/core/end2end/tests/no_logging.cc are broken . These will be fixed in another PR.
Closes#36961
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36961 from tanvi-jagtap:fix_gpr_should_log 70c3224af16ba14ce9f1b58fa659d69fa8278eb3
PiperOrigin-RevId: 645096418
Fixing ruby build.
GPR_ASSERT function has been deleted.
Closes#36935
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36935 from tanvi-jagtap:fix_ruby_build e82f026dde5e138c6bb5043b4bd148f3c0530fd6
PiperOrigin-RevId: 643618811
Fixes the CBF of `src/ruby/end2end/killed_client_thread_test.rb` (failure mode is a hang of the child process that receives the SIGTERM) that has been happening since https://github.com/grpc/grpc/pull/36724
So far grpc-ruby CQ pluck operations have so far used a 20ms-interval busy poll to check interrupts in case we've received a signal, handle process shutdown, etc. This means ongoing RPCs will not terminate their CQ operations if we need to terminate the process (the loop simply exits without waiting for the CQ op to finish), causing a leak. Those RPCs can leave refs over their corresponding channels preventing [this](8564f72e8e/src/ruby/ext/grpc/rb_channel.c (L653)) from terminating (the channels don't reach state SHUTDOWN after being destroyed).
Fix is to unblock CQ pluck operations by cancelling calls, and thus allowing the CQ pluck to actually complete its operation. For server listening CQ operations, we unblock them by shutting down the server.
A side win here is to remove the [20ms-interval busy poll](8564f72e8e/src/ruby/ext/grpc/rb_completion_queue.c (L44)) on CQ operations, which was only needed to handle shutdown.
Closes#36903
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36903 from apolcyn:fix_ruby_interrupt bed1ee2feba4f7e2977cc29f921c7b37a84d3407
PiperOrigin-RevId: 643046465
Change was created by the release automation script. See go/grpc-release.
Closes#36892
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36892 from XuanWang-Amos:bump_dev_version_202406112301 9b2898d716af0d8dcd90aab23e35959cd6c38855
PiperOrigin-RevId: 643027546
[grpc][Gpr_To_Absl_Logging] Migrating from gpr to absl logging GPR_ASSERT
Replacing GPR_ASSERT with c assert.
These changes have been made using string replacement and regex.
Closes#36817
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36817 from tanvi-jagtap:gpr_assert_removal_in_php 2222f9541314679e5a73f301bfafd5a70f280471
PiperOrigin-RevId: 642855768
[Gpr_To_Absl_Logging] Move function to test header form log.h
This is not really needed in log.h
Closes#36860
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36860 from tanvi-jagtap:move_function_to_test_header e6494bd06f2e4a08c91eb41f420607f9568b22ac
PiperOrigin-RevId: 642080756
Change was created by the release automation script. See go/grpc-release.
Closes#36544
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36544 from yashykt:bump_dev_version_202405061956 69ee5c869efa544c0ce768792750720df9d11f07
PiperOrigin-RevId: 631187829
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#36333
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36333 from yijiem:bump_dev_version_202404101721 0a7cc5b4b740688b81417ef2da8f93b4c027f3ed
PiperOrigin-RevId: 623878150
Ref: https://bugs.ruby-lang.org/issues/20311
C global variable that contain references to Ruby object MUST be declared to the Ruby GC to prevent these objects from being collected or moved.
There are a few exceptions to that, such as classes defined using the C APIs such as `rb_define_class` etc.
Up to Ruby 3.4 however, there was a bug that caused classes created from Ruby with the `Struct.new("Name")` API to also be marked as immortal and immovable.
GRPC has been relying on this bug, which I fixed in Ruby 3.4, and now GRPC is crashing when `Struct::Status` is moved around by the GC.
```
-- C level backtrace information -------------------------------------------
ruby(rb_print_backtrace+0x14) [0x5577db219d41] ruby-3.4.0/vm_dump.c:820
ruby(rb_vm_bugreport) ruby-3.4.0/vm_dump.c:1151
ruby(rb_bug_for_fatal_signal+0xfc) [0x5577db3cc60c] ruby-3.4.0/error.c:1066
ruby(sigsegv+0x4d) [0x5577db16358d] ruby-3.4.0/signal.c:926
libc.so.6(0x7f5bacd32520) [0x7f5bacd32520]
ruby(rb_class_superclass+0x32) [0x5577db0a9152] ruby-3.4.0/object.c:2239
ruby(struct_ivar_get+0x2a) [0x5577db194e02] ruby-3.4.0/struct.c:49
ruby(struct_ivar_get) ruby-3.4.0/struct.c:40
ruby(num_members) ruby-3.4.0/struct.c:705
ruby(rb_struct_new+0x56) [0x5577db19a9d6] ruby-3.4.0/struct.c:848
lib/grpc/grpc_c.so(grpc_run_batch_stack_build_result+0xe6) [0x7f5b84bb6b96] /tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/ext/grpc/rb_call.c:780
lib/grpc/grpc_c.so(grpc_rb_call_run_batch_try) /tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/ext/grpc/rb_call.c:839
ruby(rb_ensure+0x10c) [0x5577db007d3c] ruby-3.4.0/eval.c:1000
/tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/lib/grpc/grpc_c.so(grpc_rb_call_run_batch+0xca) [0x7f5b84bb595a] /tmp/bundle/ruby/3.4.0+0/bundler/gems/grpc-5ed33ee673e3/src/ruby/ext/grpc/rb_call.c:893
ruby(vm_call_cfunc_with_frame_+0x117) [0x5577db1f4c77] ruby-3.4.0/vm_insnhelper.c:3524
```
cc @apolcyn @peterzhu2118 @k0kubun
Closes#36125
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36125 from Shopify:fix-ruby-3.4-compat 7a127599c8385a1c3794c7788e034c41710a8f9b
PiperOrigin-RevId: 616152904
In grpc v1.46.2 and later versions, #29155 caused the Ruby client to return `GRPC::Core::CallError`, a non-standard error, with a message: `grpc_call_start_batch failed with outstanding read or write present (code=8)`. However, the actual error should have been `GRPC::DeadlineExceeded`. This occurred because when `GRPC::DeadlineExceeded` is raised, the `@metadata_received` flag doesn't get flipped. When `receive_and_check_status` runs to determine
the error, the `RECV_INITIAL_METADATA` is erroneously sent again.
To avoid this, flip the flag in an `ensure` block whenever the `RECV_INITIAL_METADATA` op is set.
Closes#33283Closes#33565
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/33565 from stanhu:sh-ruby-fix-issue-33283 850889558cf070e5e7b6eb471d71a61300b3442f
PiperOrigin-RevId: 614088694
gRPC has been having the `wrap_memcpy` workaround to handle the breaking change of `memcpy` on glibc 2.14 ([ref1](https://www.win.tue.nl/~aeb/linux/misc/gcc-semibug.html), [ref2](https://stackoverflow.com/questions/35656696/explanation-of-memcpy-memmove-glibc-2-14-2-2-5)). This was necessary to build more portable artifacts which are expected to run on most linux distributions and we manged to have https://github.com/grpc/grpc/pull/5007 years ago for this.
Since we started to use manylinux2010-based docker images for artifacts, however, this became unnecessary since CentOS 6 has glibc 2.12 which is older than 2.14 so it doesn't have memcpy issue, which is another benefit of using manylinux2010.
Superseding https://github.com/grpc/grpc/pull/23385
Changes after the original PR was made; Ruby is using manylinux images (manylinux2014) enabling this change, no more old docker images in our test env.
Closes#35826
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35826 from veblush:del-wrap-memcpy-2 71e6718356b2b6684bb7e977fe66ef39561c05a7
PiperOrigin-RevId: 607499060
Change was created by the release automation script. See go/grpc-release.
Closes#35899
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35899 from eugeneo:bump_dev_version_202402131133 73950425c9f00d56ce8d6b9f2f36f69b6dce4a40
PiperOrigin-RevId: 606777850
As announced in https://discourse.llvm.org/t/clang-16-notice-of-potentially-breaking-changes/65562, clang 16 defaults `-Wincompatible-function-pointer-types` to be on. This causes a number of hard errors due to implicit conversions between `void *` and `void`, such as:
```
../../../../src/ruby/ext/grpc/rb_channel.c:770:47: error: incompatible function pointer types passing 'VALUE (VALUE)' (aka 'unsigned long (unsigned long)') to parameter of type 'VALUE (*)(void *)' (aka 'unsigned long (*)(void *)') [-Wincompatible-function-pointer-types]
g_channel_polling_thread = rb_thread_create(run_poll_channels_loop, NULL);
^~~~~~~~~~~~~~~~~~~~~~
/root/.rbenv/versions/3.1.4/include/ruby-3.1.0/ruby/internal/intern/thread.h:190:32: note: passing argument to parameter 'f' here
VALUE rb_thread_create(VALUE (*f)(void *g), void *g);
^
../../../../src/ruby/ext/grpc/rb_channel.c:780:41: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes]
void grpc_rb_channel_polling_thread_stop() {
^
void
../../../../src/ruby/ext/grpc/rb_channel.c:786:30: error: incompatible function pointer types passing 'void (void *)' to parameter of type 'void *(*)(void *)' [-Wincompatible-function-pointer-types]
rb_thread_call_without_gvl(run_poll_channels_loop_unblocking_func, NULL, NULL,
```
This commit fixes these pointer types using wrapper functions where necessary.
This issue was also raised on the FreeBSD port of the grpc gem: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=271540
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#34481
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34481 from stanhu:sh-ruby-fix-clang-16 63c7424ad0c93f7a7aa1ce0e84ea7916c4cbdc5f
PiperOrigin-RevId: 600548467
`GPR_BACKWARDS_COMPATIBILITY_MODE` was devised to support old Linux kernel with old glibc but gRPC is now expected to support glibc 2.17 and later (2.17 is derived from the oldest but supported Linux distro, CentOS 7). Effectively, gRPC doesn't need `GPR_BACKWARDS_COMPATIBILITY_MODE` anymore. Hence, let's remove it.
gRPC Python and Ruby that distribute binary artifacts already began to use `manylinux` environment dealing with this issue in a better way so they don't need this flag anymore.
Closes#35602
PiperOrigin-RevId: 599895270
The `grpc_channel_args` is retained on the Ruby object and used for recreating the channel after forking in
grpc_rb_channel_maybe_recreate_channel_after_fork(). Previously, the key for each argument was taken from a Ruby string directly, which could be invalidated if the Ruby string is modified or moved by the GC. Duplicate the string for the key instead, so we own it.
Reproducer in https://github.com/grpc/grpc/issues/35489
<!--
If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the appropriate
lang label.
-->
Closes#35488
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35488 from Shopify:key-uaf c1813cee0184a5003f01dd1086df4f322917281b
PiperOrigin-RevId: 599304551
Change was created by the release automation script. See go/grpc-release.
Closes#35580
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/35580 from stanley-cheung:bump_dev_version_202401171835 b363888ca5d88c11e67237ca4bc2c5c3cf1fcbdf
PiperOrigin-RevId: 599267642
Address #28382. This is a recreation of #31368 except e2e tests are not handled here (yet).
Closes#34861
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/34861 from rockspore:tls_version f9a1215ac1291722dba4f22cedc128c241aa3cab
PiperOrigin-RevId: 589847110
This commit upgrades gRPC to protobuf v25.0 and makes some fixes to
account for upb changes. One major change is that upb has been merged
into the protobuf repo, so we can now drop the separate `@upb`
dependency. Another is that `.upb.c` files no longer exist and there are
new `.upb_minitable.h` and `.upb_minitable.c` files. The longer
filenames exceeded a Windows restriction, so to work around that I
renamed the `upb-generated` directory to just `upb-gen`, and likewise
for `upbdefs-generated`.
Currently it is very easy to use the `TlsCredentialsOptions` in such a
way that it produces a memory leak. For example, the code block
```
{
TlsCredentialsOptions options;
}
```
produces a memory leak. This PR fixes up the ownership bugs in this
class and its `grpc_tls_credentials_options`, the C-core analogue.
Introduce `grpc-native-debug` gems containing debug symbol packages that
complement the shared libraries shipped in pre-compiled binary gems.
src/ruby/nativedebug/README.md has details on usage.
<!--
If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.
If your pull request is for a specific language, please add the
appropriate
lang label.
-->
https://github.com/grpc/grpc/pull/33538 added `-weak_framework
CoreFoundation` in `DLDFLAGS` for only `arm64-darwin` builds, but the
issue reported in https://github.com/grpc/grpc/issues/33483 can also
happen on `x86-darwin` builds. This can happen if:
1. The Ruby interpreter is compiled without
`-Wl,-undefined,dynamic_lookup`.
2. This happens if the Ruby interpreter is built with XCode 14.0 to 14.2
(https://bugs.ruby-lang.org/issues/19005).
Simplify the logic and always include `-weak_framework CoreFoundation`
for macOS builds.
grpc 1.57.0 crashes win ruby and alpine due to no `strdup` in musl libc.
This diff replace `strdup` with `grp_strdup`
```
Thread 1 "ruby" received signal SIGSEGV, Segmentation fault.
0x00000000000a4596 in ?? ()
(gdb) bt
#0 0x00000000000a4596 in ?? ()
#1 0x00007ffff14e298c in grpc_rb_channel_create_in_process_add_args_hash_cb (key=<optimized out>, val=<optimized out>, args_obj=<optimized out>) at rb_channel_args.c:84
#2 0x00007ffff7c2b9ea in hash_ar_foreach_iter (error=0, argp=140737488344784, value=<optimized out>, key=<optimized out>) at hash.c:1341
```
fixes#34044closes#27995
This was done manually due to a problem with
`tools/distrib/python/make_grpcio_tools.py`. ~I fixed it in this PR
(depends on cl/547979185), so there is a fair chance this upgrade will
work normally for the next release.~ The fix may be problematic for
upgrading protobuf on older release branches, so the improvement will be
worked on separately. CC @jtattermusch
This also updates the UPB dep to the latest commit on the 23.x branch.
So far, we've structured global C-core init/shutdown as follows:
1) every grpc-ruby object calls `grpc_init` when allocated, and
`grpc_shutdown` when finalized
2) grpc-ruby background threads are each wrapped in a
`grpc_init/shutdown` pair - for example see
b32d94de05/src/ruby/ext/grpc/rb_event_thread.c (L122)
and
b32d94de05/src/ruby/ext/grpc/rb_event_thread.c (L136)
But because ruby doesn't join ruby threads when the process is
terminating, the `init/shutdown` pairs in 2) are always left open. I.e.,
thus far we have never been invoking the final call to `grpc_shutdown`
which actually does C-core global shutdown. Thus our calls to
`grpc_shutdown` are useless.
So we might as well keep things simple and not even attempt to call
`grpc_shutdown`. Now we just call `grpc_init` before using C-core, and
then never even attempt global shutdown (in non-forking situations).
As a bonus, this fixes the issue with the event thread's
`grpc_ruby_init` racing with `prefork` that is mentioned in
https://github.com/grpc/grpc/pull/33666
Adds experimental fork support to gRPC/Ruby
Works towards https://github.com/grpc/grpc/issues/8798 (see caveats for why this wasn't marked fixed yet)
Works towards https://github.com/grpc/grpc/issues/33578 (see caveats for why this wasn't marked fixed yet)
This leverages existing `pthread_atfork` based C-core support for
forking that python/php use, but there's a bit extra involved mainly
because gRPC/Ruby has additional background threads.
New tests under `src/ruby/end2end` show example usage.
Based on https://github.com/grpc/grpc/pull/33495
Caveats:
- Bidi streams are not yet supported (bidi streams spawn background
threads which are not yet fork safe)
- Servers not supported
- Only linux supported
Remove `src/ruby/ext/grpc/objs/` and strip `src/ruby/ext/grpc/libs/` in
ruby build dir to save some space.
This can reduce the grpc gem size from more than 1G to about 70~80M.
This only affects opt build, dbg build remains the same.
A proper fix maybe use the same makefile to build c-core and the
extension rather than building c-core when generating the extension
makefile.
fixes: https://github.com/grpc/grpc/issues/33412
