[pull] master from ray-project:master - #1193
Merged
Merged
Conversation
…ies (#65036) TorchSquashedGaussian.logp() recovered the pre-squash value via atanh(clamp(action)), which caps the recovered value at ~7.25 and corrupts logp (large negative) once the policy mean saturates tanh. This breaks SAC/TQC alpha tuning and the critic entropy target on envs where the policy saturates (e.g. Humanoid). This is also part of the picture why TQC can't solve humanoid. The solution involves some magic math ( see https://github.com/openai/spinningup/blob/master/spinup/algos/pytorch/sac/core.py#L59-L60 ) Adds a regression unit test. --------- Signed-off-by: Artur Niederfahrenhorst <artur@anyscale.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…#64799) ## Description `ray::StatusOr<T>` keeps its value in a union member `data_` that is only a live object when `ok()`; in the error state `data_` is left unconstructed. Two paths ignored that and touched the unconstructed storage: - `swap()` swapped `data_` unconditionally, move-constructing from / assigning into unconstructed storage when either operand was an error. - `operator=(const&)` and `operator=(&&)` set `status_` to OK before calling `AssignValue`, which destroys the old value only when `ok()`. With the status flipped first, it ran `data_.~T()` on unconstructed storage whenever the destination was previously an error. Both assignment operators now branch on the current state: when this already holds a value, assign through `T`'s own `operator=` to reuse its resources and stay exception-safe (a throwing assignment leaves the object intact instead of status-OK-with-no-value); otherwise placement-new the value and only then set `status_` to OK. `swap` is specialized for the four `ok()`/error combinations: swap the underlying values (or statuses) directly when both sides match, and move the lone value across when only one side holds it — O(1) for swappable `T`, with no touch of unconstructed storage. Added tests covering assignment and swap across the error/value combinations using a type with a live-instance counter, so a stray or missing ctor/dtor is caught in a normal test run without relying on a sanitizer. Each branch was verified to fail against the pre-fix code. ## Related issues Fixes #64798 ## Additional information `bazel test //src/ray/common/tests:status_or_test` passes, including under `--config=asan`. Verified the new tests fail when the fix is reverted. --------- Signed-off-by: yangjie01 <yangjie01@baidu.com> Co-authored-by: Chi-Sheng Liu <chishengliu@chishengliu.com>
## Root cause Verl SGLang crashes on Intel GPU because Ray sets ONEAPI_DEVICE_SELECTOR=level_zero:N in workers, but SGLang suppresses Ray's env var via RAY_EXPERIMENTAL_NOSET_ONEAPI_DEVICE_SELECTOR and sets its own bare IDs (e.g. ONEAPI_DEVICE_SELECTOR=0,1). That format is invalid — the level_zero: prefix is required — causing an immediate torch import crash in SGLang subprocesses. ## Changes **python/ray/_private/accelerators/intel_gpu.py** - ZE_AFFINITY_MASK is now the primary env var (bare IDs, consistent with CUDA_VISIBLE_DEVICES) - get_current_process_visible_accelerator_ids() reads ZE_AFFINITY_MASK first, falls back to ONEAPI_DEVICE_SELECTOR for backward compat (handles both level_zero:0,1 and bare 0,1 formats) - set_current_process_visible_accelerator_ids() writes both ZE_AFFINITY_MASK (physical bare IDs) and ONEAPI_DEVICE_SELECTOR (level_zero: prefixed re-indexed sequential IDs) - Added RAY_EXPERIMENTAL_NOSET_ZE_AFFINITY_MASK flag; existing RAY_EXPERIMENTAL_NOSET_ONEAPI_DEVICE_SELECTOR still works independently **Bug found and fixed:** initial implementation wrote ONEAPI_DEVICE_SELECTOR=level_zero:1 for physical GPU 1, but ZE_AFFINITY_MASK re-indexes devices before ONEAPI_DEVICE_SELECTOR applies — intersection was empty, worker saw 0 GPUs. Fixed: ONEAPI_DEVICE_SELECTOR always uses sequential re-indexed IDs (level_zero:0, level_zero:0,1) regardless of physical IDs. Hardware-confirmed on 2x Intel GPU: GPU 1 before fix: ZE=1 + ONEAPI=level_zero:1 → 0 devices visible GPU 1 after fix: ZE=1 + ONEAPI=level_zero:0 → 1 device visible ✓ **doc/source/ray-core/scheduling/accelerators.rst** - Updated tip and code examples to use ZE_AFFINITY_MASK - Added backward compat note for ONEAPI_DEVICE_SELECTOR users **doc/source/serve/llm/user-guides/sglang.md** - Added Intel GPU prerequisite entry: RAY_EXPERIMENTAL_NOSET_ZE_AFFINITY_MASK=1 and RAY_EXPERIMENTAL_NOSET_ONEAPI_DEVICE_SELECTOR=1 (both needed since Ray now writes both vars) **python/ray/tests/accelerators/test_intel_gpu.py** - Added parametrize cases for non-zero physical GPU IDs (["1"], ["1","3"]) — these are the cases that catch the re-indexing bug - Replaced manual del os.environ cleanup with clean_accelerator_env fixture that restores env vars even on test failure - Split noset flag tests into two independent functions **python/ray/tests/accelerators/test_intel_gpu_e2e.py** - Updated gpu_task and cluster_probe_task to return ze_affinity_mask - Updated _validate_gpu_binding_common to check ZE_AFFINITY_MASK for physical GPU IDs (ONEAPI_DEVICE_SELECTOR now carries re-indexed IDs so physical ID comparison against it would be wrong) ## Test plan - `python -m pytest python/ray/tests/accelerators/test_intel_gpu.py -v` No GPU needed, passes on any machine - `RAY_PYTEST_USE_GPU=1 python -m pytest python/ray/tests/accelerators/test_intel_gpu_e2e.py -v` Requires Intel GPU + dpctl. Single-node 2x GPU tests pass. Multi-node test (test_scale_out_task_distribution) is implemented but not yet validated — no multi-node Intel GPU cluster available. --------- Signed-off-by: Kah Lun teoh <kah.lun.teoh@intel.com> Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
…65182) ## Description Fix the test_tpu.py on windows ``` [2026-08-03T18:43:13Z] ================================== FAILURES =================================== -- | [2026-08-03T18:43:13Z] _________________ test_autodetect_num_tpus_vfio_mixed_groups __________________ | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] path = '/sys/kernel/iommu_groups/10/devices\\0000:01:00.0\\vendor', args = () | [2026-08-03T18:43:13Z] kwargs = {'encoding': 'ascii'} | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] def fake_open(path, *args, **kwargs): | [2026-08-03T18:43:13Z] try: | [2026-08-03T18:43:13Z] > return mock.mock_open(read_data=vendor_results[path])() | [2026-08-03T18:43:13Z] E KeyError: '/sys/kernel/iommu_groups/10/devices\\0000:01:00.0\\vendor' | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] python\ray\tests\accelerators\test_tpu.py:120: KeyError | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] During handling of the above exception, another exception occurred: | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] mock_list = <MagicMock name='listdir' id='2259299081216'> | [2026-08-03T18:43:13Z] mock_glob = <MagicMock name='glob' id='2259298850976'> | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] @patch("glob.glob") | [2026-08-03T18:43:13Z] @patch("os.listdir") | [2026-08-03T18:43:13Z] def test_autodetect_num_tpus_vfio_mixed_groups(mock_list, mock_glob): | [2026-08-03T18:43:13Z] # Two VFIO groups: one is a Google TPU (vendor 0x1ae0), the other is the | [2026-08-03T18:43:13Z] # BlueField-3 SoC (vendor 0x15b3). Only the TPU-backed group is counted. | [2026-08-03T18:43:13Z] mock_glob.return_value = [] | [2026-08-03T18:43:13Z] listdir_results = { | [2026-08-03T18:43:13Z] "/dev/vfio": ["vfio", "10", "96"], | [2026-08-03T18:43:13Z] "/sys/kernel/iommu_groups/10/devices": ["0000:01:00.0"], | [2026-08-03T18:43:13Z] "/sys/kernel/iommu_groups/96/devices": ["0016:03:00.2"], | [2026-08-03T18:43:13Z] } | [2026-08-03T18:43:13Z] vendor_results = { | [2026-08-03T18:43:13Z] "/sys/kernel/iommu_groups/10/devices/0000:01:00.0/vendor": "0x1ae0\n", | [2026-08-03T18:43:13Z] "/sys/kernel/iommu_groups/96/devices/0016:03:00.2/vendor": "0x15b3\n", | [2026-08-03T18:43:13Z] } | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] def fake_listdir(path): | [2026-08-03T18:43:13Z] try: | [2026-08-03T18:43:13Z] return listdir_results[path] | [2026-08-03T18:43:13Z] except KeyError: | [2026-08-03T18:43:13Z] raise AssertionError(f"unexpected listdir: {path}") | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] def fake_open(path, *args, **kwargs): | [2026-08-03T18:43:13Z] try: | [2026-08-03T18:43:13Z] return mock.mock_open(read_data=vendor_results[path])() | [2026-08-03T18:43:13Z] except KeyError: | [2026-08-03T18:43:13Z] raise AssertionError(f"unexpected open: {path}") | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] mock_list.side_effect = fake_listdir | [2026-08-03T18:43:13Z] with patch("builtins.open", side_effect=fake_open): | [2026-08-03T18:43:13Z] TPUAcceleratorManager.get_current_node_num_accelerators.cache_clear() | [2026-08-03T18:43:13Z] > assert TPUAcceleratorManager.get_current_node_num_accelerators() == 1 | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] python\ray\tests\accelerators\test_tpu.py:127: | [2026-08-03T18:43:13Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | [2026-08-03T18:43:13Z] C:\rayci\python\ray\_private\accelerators\tpu.py:678: in get_current_node_num_accelerators | [2026-08-03T18:43:13Z] if _is_vfio_group_a_tpu(group): | [2026-08-03T18:43:13Z] C:\rayci\python\ray\_private\accelerators\tpu.py:601: in _is_vfio_group_a_tpu | [2026-08-03T18:43:13Z] with open(vendor_path, encoding="ascii") as f: | [2026-08-03T18:43:13Z] C:\Miniconda3\lib\unittest\mock.py:1114: in __call__ | [2026-08-03T18:43:13Z] return self._mock_call(*args, **kwargs) | [2026-08-03T18:43:13Z] C:\Miniconda3\lib\unittest\mock.py:1118: in _mock_call | [2026-08-03T18:43:13Z] return self._execute_mock_call(*args, **kwargs) | [2026-08-03T18:43:13Z] C:\Miniconda3\lib\unittest\mock.py:1179: in _execute_mock_call | [2026-08-03T18:43:13Z] result = effect(*args, **kwargs) | [2026-08-03T18:43:13Z] _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] path = '/sys/kernel/iommu_groups/10/devices\\0000:01:00.0\\vendor', args = () | [2026-08-03T18:43:13Z] kwargs = {'encoding': 'ascii'} | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] def fake_open(path, *args, **kwargs): | [2026-08-03T18:43:13Z] try: | [2026-08-03T18:43:13Z] return mock.mock_open(read_data=vendor_results[path])() | [2026-08-03T18:43:13Z] except KeyError: | [2026-08-03T18:43:13Z] > raise AssertionError(f"unexpected open: {path}") | [2026-08-03T18:43:13Z] E AssertionError: unexpected open: /sys/kernel/iommu_groups/10/devices\0000:01:00.0\vendor | [2026-08-03T18:43:13Z] | [2026-08-03T18:43:13Z] python\ray\tests\accelerators\test_tpu.py:122: AssertionError ``` Signed-off-by: Rueian Huang <rueiancsie@gmail.com>
## Description Adds a collapsed Serve LLM grafana row for native vLLM KV offload/reload observability: GPU ↔ CPU throughput, operation rate, transfer bandwidth, CPU transfer pressure, external prefix-cache hit rate, and P90 lookup delay. Panels join vLLM worker metrics with serve deployment/replica labels. <img width="3417" height="953" alt="Screenshot 2026-07-29 at 5 29 57 PM" src="https://github.com/user-attachments/assets/313975d2-a1cd-4834-8d6c-bb5ccefcb62a" /> ## Related issues > Link related issues: "Fixes #1234", "Closes #1234", or "Related to #1234". ## Additional information > Optional: Add implementation details, API changes, usage examples, screenshots, etc. --------- Signed-off-by: Jeffrey Wang <jeffreywang@anyscale.com>
…5131) ## Why are these changes needed? Fixes #64273 The Java dependency `com.google.code.gson:gson` is currently pinned to version `2.9.1`, which is outdated. This PR upgrades it to `2.11.0`, which includes bug fixes, performance improvements, and security enhancements over the previous version. Key changes in gson 2.10.0–2.11.0: - Improved `TypeToken` API safety - Fixed edge cases in JSON parsing - Performance improvements in serialization/deserialization ## Changes - Updated `java/dependencies.bzl`: bumped `com.google.code.gson:gson` from `2.9.1` to `2.11.0` - No lock file update required (Ray uses dynamic Maven resolution via `rules_jvm_external`, no `maven_install.json` pins gson) ## Checks - [ ] I've run `scripts/format.sh` to lint the changes in this PR. - [ ] I've made sure the tests are passing. ## Testing Dependency version bump only. No API-level changes to Ray source code. Gson 2.11.0 is backward compatible with 2.9.1 for all usages present in this codebase. Signed-off-by: tqKhanh1712 <tonhannhan223@gmail.com>
…#65187) ## Why are these changes needed? Follow-up to #64765 / #63017. `ArrowBlockAccessor.to_pandas()` maps Arrow types to `pd.ArrowDtype` via a `_types_mapper`, with carve-outs for extension and dictionary types. Arrow's `null` type needs the same carve-out. `null[pyarrow]` is unusable from pandas: the type carries no type information, so pandas cannot box a non-null value into such a column. `fillna` and masked assignment raise `ArrowInvalid`, and in some pyarrow/pandas combinations the failure comes out of Arrow C++ and aborts the worker process rather than raising. This is easy to hit in practice because the type is assigned per block, not per dataset. A column that has values overall can still be entirely null within one block — which is common with small blocks — so a pandas UDF like `map_batches(lambda df: df.fillna(...), batch_format="pandas")` fails on whichever block happens to hold only nulls: ```python ds = ray.data.from_items( [{"a": 1.0, "b": 2.0}, {"a": 3.0, "b": None}, {"a": None, "b": 4.0}], override_num_blocks=3, ) ds.map_batches(lambda df: df.fillna({"a": 0.0, "b": 0.0}), batch_format="pandas").take_all() # ArrowInvalid on the block where "a" (or "b") is all-null ``` The fix returns `None` from `_types_mapper` for null-typed columns, falling back to pandas' default conversion. The Arrow round-trip is unchanged: `PandasBlockAccessor.to_arrow()` already coerces all-null columns back to `pa.null()`. The one behavioral consequence, noted in a comment: a column that is all-null in *every* block stays null-typed instead of being promoted — same as before this change. ## Related issue number Follow-up to #64765. Two tests added: - `test_arrow_block_to_pandas_null_type_is_not_arrow_backed` — a `pa.null()` column is not `pd.ArrowDtype`, still round-trips back to `pa.null()` untouched, and `fillna` now works and adopts the fill value's type. - `test_pandas_udf_can_fill_per_block_null_columns` — end-to-end regression test for the `map_batches` + `fillna` failure above. Signed-off-by: Goutam <goutam@anyscale.com> Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
## Motivation Batch inference via `map_batches` with a GPU actor has a lot of the same boilerplate code: move data to and from the device, convert data into tensors that can be processed by the GPU, convert the resultant output tensors back to a format that Ray Data can process. A lot of this is duplicated code, and similar to how we have sensible defaults in other parts of our API (such as `DefaultCollateFn` for `iter_torch_batches`), we should provide a short-hand framework for batch inference on `map_batches`. ## API The user has to subclass this class: ``` @publicapi(stability="alpha") class TorchInference: def initialize(self, *args: Any, **kwargs: Any) -> None: pass def get_device(self) -> "torch.device": ... def collate( self, input_batch: "DataBatch" ) -> Union["TensorBatchType", Tuple["TensorBatchType", Any]]: ... def process_on_device( self, input_batch: "DataBatch", collated_tensors: "TensorBatchType", collated_other: Any, ) -> Union["TensorBatchType", Tuple["TensorBatchType", Any]]: raise NotImplementedError() def finalize( self, input_batch: "DataBatch", output_tensors: "TensorBatchType", output_other: Any, ) -> "DataBatch": ... ``` and mandatorily implement `process_on_device` but can also optionally implement the other 4 methods. On the Ray Data side, we manage moving data to and from device and also providing sensible defaults for `get_device`, `collate` and `finalize`. ### The Defaults - `get_device`: `torch.device("cuda")` - `collate`: take a numpy batch, convert and concatenate tensors as necessary. - `fianlize`: recursively take every Tensor in the batch and convert them to numpy via `.numpy()`. ## What Happens Underneath? After initialization, for each batch, the following occurs: 1. The `DataBatch` is fed into `collate` 2. The tensor output of `collate` is moved to device. 3. `process_on_device` is called on GPU device. 4. The tensor output of `process_on_device` is moved back to CPU. 5. `finalize` is called on CPU tensor. Each stage may need to reference 1) additional information created in the steps that are not tensors and 2) some other columns in the original batch. Therefore, for each function invocation, we provide the original batch as `input_batch` and any other miscellaneous output as `output_other`. --------- Signed-off-by: Daniel Shin <kyuds@anyscale.com> Signed-off-by: Daniel Shin <88547237+kyuds@users.noreply.github.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
8 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )