Noise is currently higher than our detection threshold.
Closes#38674
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38674 from ctiller:limits 9d3f4901e4a3d261fb121f5f22ecf60431e29cca
PiperOrigin-RevId: 722842062
We used to use a thresholded sum of differences to determine significance of benchmark results. This is significantly flawed, especially when adding new checks - the significance threshold must also be updated, but it's unclear exactly how.
This new approach relies on counting the number of significant increases/decreases across the benchmarks, and then reporting based on how we should react: increases are biased towards (let's not increase memory blindly!), then decreases (huzzah, go collect your bonus!), then neutral changes.
Closes#38447
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/38447 from ctiller:diffy 7c846eaf55cdbd23aafac75d6e2e169f846557ce
PiperOrigin-RevId: 715951309
Built on #36862 which should be merged first.
Closes#36863
COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36863 from ctiller:cgmem 9d45ced0b6f6b23ccb38974e139f6ba4315651a0
PiperOrigin-RevId: 644219306
Relands #34785, which was reverted in #34818.
The first commit is the revert. The second commit removes the gtest
dependency from the xds_server library, which should address the
testonly problem internally.
- Switched from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies
Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.
To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
* Added new files for channel client/server
* Committing to switch branch
* Rebasing branch
* Switching branch
* Server process getting called
* Still working
* RPC received success, with sleep
* gRPC Receive success, grpc timeout
* Earlier but Clang tidy
* Fix timeout issue, remove some logs
* Added signint handler, test passing but flaky
* added sleep to reduce flakiness, removed some dependencies, changed LOG to gpr_log
* Changed benchmark_name default back to call
* remove deleted files
* grpc shutdown timeout
* trying to add shutdown
* Some changes
* Removed shutdown
* Automated change: Fix sanity tests
* Changes for review comments
* Changed comments
* Changed benchmark driver defaults so that CI testing would happen for all benchmarks
* Automated change: Fix sanity tests
Co-authored-by: nancylucy01 <nancylucy01@users.noreply.github.com>
* Changed benchmark driver's flags and updated calling python script. Untested
* Changes made for debugging/testing
* Switching branch
* Removed debug statements and changed file permissions for memory_diff
* Minor changes for review
* Changed secure description
* Changed flag configuration for clarity
* Added dep to BUILD file
* Changed return value
* Edits for review, added comments
* Changed a print statement
