40 Commits

Author SHA1 Message Date
Sreenithi Sridharan
ce9f0496e7 [Fix][Python] Pin pip version to 25.2 in Python tests (#40959)
Pip released [v25.3](https://pip.pypa.io/en/stable/news/#v25-3) a few days back which doesn't support `setup.py` builds anymore, while we are yet to migrate to use the `pyproject.toml` build system in #40833.

Hence some of our Python tests using the most recent versions of pip for the build have started to fail with errors like:
```
  ERROR: Failed building wheel for grpcio
Failed to build grpcio
error: failed-wheel-build-for-install

× Failed to build installable wheels for some pyproject.toml based projects
```

This PR hence pins the pip version to 25.2 which still supports setup.py build until #40833 is submitted.

Closes #40959

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/40959 from sreenithi:pin_pip_version d72e0563ce0f00a962bd03ddd279cf587494fd3e
PiperOrigin-RevId: 824584176
2025-10-27 10:46:00 -07:00
Xuan Wang
7d04e020df [Python Dist] Fix grpc_distribtests_python (#36352)
`setuptools` made a decision to change the artifact name it builds (from `grpcio-health-checking` to `grpcio_health_checking`) in their latest release (https://github.com/pypa/setuptools/issues/4300).
As a result, we need broaden our regex so that our tests can pickup the correct files.

### Note
* Using `[_-]*` instead of `[_-]?` to match one character since `bash` uses a different flavor of regular expressions called basic regular expressions (BREs) which do not support the optional quantifier `?`.

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #36352

COPYBARA_INTEGRATE_REVIEW=https://github.com/grpc/grpc/pull/36352 from XuanWang-Amos:fix_distribution_test 8dfcc4e9a3b3513b8c8d0a5321d689592e1fe7da
PiperOrigin-RevId: 625083784
2024-04-15 14:29:41 -07:00
Xuan Wang
9b424b5dd4 [Python o11y] Fix Python O11Y artifacts name (#35965)
Update observability to PYPI failed because the artifact name is not correct.

This PR:
* Fix the artifacts name.
* Add step to test observability artifacts in `test_packages.sh`.
* Added `-fno-ipa-cp` compile flag.
  * We're seeing `inlining failed in call to always_inline 'vsnprintf': function body can be overwritten at link time` errors when building from source using musl libc.
  * Based on [investigation](https://gitlab.alpinelinux.org/alpine/aports/-/issues/8626), it's because we're using `-flto` flag.
  * One solution is to [disable fortify by adding this flag](https://gitlab.alpinelinux.org/alpine/aports/-/issues/8626).

After this PR, the observability artifacts have the correct name:
* [Distribution Tests Python Linux](https://pantheon.corp.google.com/storage/browser/grpc-testing-kokoro-prod/test_result_public/prod/grpc/core/pull_request/linux/grpc_distribtests_python/28687/20240223-105306/github/grpc/artifacts;tab=objects?pageState=(%22StorageObjectListTable%22:(%22f%22:%22%255B%255D%22))&e=13802955&mods=-logs_tg_prod&prefix=&forceOnObjectsSortingFiltering=false)

Also tested that the artifacts build in this PR can be successfully uploaded to testpypi:
* https://test.pypi.org/project/grpcio-observability/1.63.0.dev0

<!--

If you know who should review your pull request, please assign it to that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the appropriate
lang label.

-->

Closes #35965

PiperOrigin-RevId: 611268154
2024-02-28 16:33:28 -08:00
Xuan Wang
c7a1336566 [Python 3.12] Deprecate distutil (#34186)
### Background

* `distutils` is deprecated with removal planned for Python 3.12
([pep-0632](https://peps.python.org/pep-0632/)), thus we're trying to
replace all distutils usage with setuptools.
* Please note that user still have access to `distutils` if setuptools
is installed and `SETUPTOOLS_USE_DISTUTILS` is set to `local` (The
default in setuptools, more details can be found [in this
discussion](https://github.com/pypa/setuptools/issues/2806#issuecomment-1193336591)).

### How we decide the replacement

* We're following setuptools [Porting from Distutils
guide](https://setuptools.pypa.io/en/latest/deprecated/distutils-legacy.html#porting-from-distutils)
when deciding the replacement.

#### Replacement not mentioned in the guide

* Replaced `distutils.utils.get_platform()` with
`sysconfig.get_platform()`.
* Based on the [answer
here](https://stackoverflow.com/questions/71664875/what-is-the-replacement-for-distutils-util-get-platform),
and also checked the document that `sysconfig.get_platform()` is good
enough for our use cases.
* Replaced `DistutilsOptionError` with `OptionError`.
* `setuptools.error` is exporting it as `OptionError` [in the
code](https://github.com/pypa/setuptools/blob/v59.6.0/setuptools/errors.py).
* Upgrade `setuptools` in `test_packages.sh` and changed the version
ping to `59.6.0` in `build_artifact_python.bat`.
* `distutils.errors.*` is not fully re-exported until `59.0.0` (See
[this issue](https://github.com/pypa/setuptools/issues/2698) for more
details).

### Changes not included in this PR

* We're patching some compiler related functions provided by distutils
in our code
([example](ee4efc31c1/src/python/grpcio/_spawn_patch.py (L30))),
but since `setuptools` doesn't have similar interface (See [this issue
for more details](https://github.com/pypa/setuptools/issues/2806)), we
don't have a clear path to replace them yet.


<!--

If you know who should review your pull request, please assign it to
that
person, otherwise the pull request would get assigned randomly.

If your pull request is for a specific language, please add the
appropriate
lang label.

-->
2023-09-06 14:32:30 -07:00
Sergii Tkachenko
de6ed9ba9f [Python] Migrate from yapf to black (#33138)
- Switched  from yapf to black
- Reconfigure isort for black
- Resolve black/pylint idiosyncrasies 

Note: I used `--experimental-string-processing` because black was
producing "implicit string concatenation", similar to what described
here: https://github.com/psf/black/issues/1837. While currently this
feature is experimental, it will be enabled by default:
https://github.com/psf/black/issues/2188. After running black with the
new string processing so that the generated code merges these `"hello" "
world"` strings concatenations, then I removed
`--experimental-string-processing` for stability, and regenerated the
code again.

To the reviewer: don't even try to open "Files Changed" tab 😄 It's
better to review commit-by-commit, and ignore `run black and isort`.
2023-06-09 15:08:55 -07:00
Lidi Zheng
cd4261b946 Support musllinux binary wheels on x64 and x86 (#28092)
* Support musllinux binary wheels

* Skip aarach64 for now
* Consolidate the difference of mktemp
* Extend linux artifact building time && install bash for distribtest
* Stop using grpc.tools, use grpc_tools
* Update the README to use grpc_tools
* Force static link libc++ for alpine binaries
* Rebase recent build script changes

* Install ccache for musllinux distribtest images

* Revert timeout change to grpc_build_artifacts
2022-02-10 14:14:56 -08:00
Lidi Zheng
5820e152cd Remove Python 2.7 binary wheel generations (#26691)
* Remove Python 2.7 binary wheels

* Make sure pip won't pick-up newer versions

* Attempt to fix the distribtest

* Fallback to virtualenv==16.7.9 for 34 and 35

* Remove 34 from binary wheel distribtests
2021-07-23 08:20:39 -07:00
Jan Tattermusch
d61db49b48 Add python3.8 aarch64 wheel distribtest (#26136)
* add python3.8 aarch64 distribtest

* register qemu emulator for linux distribtests
2021-05-03 08:16:11 +02:00
Lidi Zheng
f42ad52c44 Update --dest-dir to -d 2019-03-13 12:40:10 -07:00
Lidi Zheng
64b29fba5d An attempt to fix distrib test 2019-03-13 09:58:54 -07:00
Lidi Zheng
fa7f010795 Fix the hash checking mechanism 2019-03-08 10:47:04 -08:00
Lidi Zheng
127a6c1d9e Revert "Revert "Roll foward "Strip Python wheel binary"""
This reverts commit 98fc902200.
2019-03-08 10:29:00 -08:00
Lidi Zheng
98fc902200 Revert "Roll foward "Strip Python wheel binary"" 2019-03-07 17:34:45 -08:00
Lidi Zheng
50a1ddab5c Revert "Revert "Strip Python wheel binary""
This reverts commit 04609b1ea5.
2019-03-06 07:57:21 -08:00
Jan Tattermusch
04609b1ea5 Revert "Strip Python wheel binary" 2019-03-06 11:00:13 +01:00
Lidi Zheng
168df1cb5f Check hashes for pip wheels before installing 2019-03-04 16:01:45 -08:00
Mehrdad Afshari
2cb912c716 Add grpcio-testing to Python distribtests 2018-06-20 10:06:32 -07:00
Mehrdad Afshari
1a92cd0b52 More consistent source/binary Python distribtests 2018-06-20 10:06:32 -07:00
Mehrdad Afshari
f4bea02a72 Fix the Python source distribtest script 2018-04-27 06:59:51 -07:00
Mehrdad Afshari
b189923376 Split binary and source distribtests for Python 2018-04-11 14:48:48 -07:00
Mehrdad Afshari
0a400c6c0a Fix python/run_distrib_test.sh to pass shellcheck 2018-02-07 13:48:14 -08:00
Jan Tattermusch
7897ae9308 auto-fix most of licenses 2017-06-08 11:22:41 +02:00
Nathaniel Manista
5450f05e0c Migrate distrib, interop, and stress to GA API 2016-08-11 14:07:37 +00:00
Masood Malekghassemi
302bfd1a10 Test Python source distribution installation 2016-07-15 23:48:25 -07:00
Jan Tattermusch
4e2f7727a9 rename grpc.protoc.compiler to grpc.tools.protoc 2016-05-06 12:28:58 -07:00
Jan Tattermusch
6f0fe0f327 fix python distribtests with docker on overlay 2016-05-06 12:16:28 -07:00
Jan Tattermusch
4ead664c33 small addendum for python distribtests 2016-05-05 08:01:40 -07:00
Masood Malekghassemi
916c960e59 Fix Python distrib test 2016-05-04 15:30:30 -07:00
Masood Malekghassemi
e9407c2e74 Test binary Python distributions 2016-05-04 15:30:30 -07:00
Craig Tiller
6169d5f7b0 Update copyrights 2016-03-31 07:46:18 -07:00
Craig Tiller
c0473cc823 Merge github.com:grpc/grpc into compress 2016-02-11 08:51:16 -08:00
Jan Tattermusch
0474a1c393 python fixes 2016-02-09 08:34:02 -08:00
Jan Tattermusch
6677a86283 do not hardcode the python package version 2016-02-09 08:00:41 -08:00
Craig Tiller
55e813f946 Merge pull request #5088 from murgatroid99/core_long_int64_t
Replace 'long' with 'int64_t' in public core headers
2016-02-08 10:22:42 -08:00
Craig Tiller
6b5555ead1 Fix docker files, distrib test script 2016-02-05 16:07:18 -08:00
Craig Tiller
5c04760d46 Change default on Python install option 2016-02-05 14:25:56 -08:00
murgatroid99
7a55684a13 Clang format and fix copyrights 2016-02-05 11:33:30 -08:00
Craig Tiller
f3f8fdbc54 Fix copyright 2016-02-05 10:46:43 -08:00
Jan Tattermusch
eae8fc4a8b python distrib smoketest 2016-02-04 14:29:50 -08:00
Craig Tiller
b2afc1f680 Starting Python 2016-02-02 12:30:08 -08:00