🔄 daily merge: master → main 2025-10-31

[antfin-oss]

This Pull Request was created automatically to merge the latest changes from master into main branch.

📅 Created: 2025-10-31
🔀 Merge direction: master → main
🤖 Triggered by: Scheduled

Please review and merge if everything looks good.

[sourcery-ai]

The pull request #666 has too many files changed.

The GitHub API will only let us fetch up to 300 changed files, and this pull request has 5083.

[gemini-code-assist]

Summary of Changes

Hello @antfin-oss, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request integrates the latest changes from the master branch into main, encompassing a broad range of updates to the project's build system, CI/CD pipelines, and development tooling. The changes aim to enhance build reliability, improve developer experience through updated linting and dependency management, and ensure compatibility across various platforms and Python versions. The documentation infrastructure has also been significantly revised to align with new standards and tools.

Highlights

• Bazel Configuration Updates: The .bazelrc file has been updated to make --incompatible_strict_action_env a global default, add workspace_status_command for Linux, include /utf-8 cxxopt for Windows, and ignore warnings for third-party files. It also introduces test_env=PATH for CI Python tests and allows custom user options via .user.bazelrc. The BUILD.bazel file now uses pkg_files and pkg_zip for packaging, and introduces Python binaries for generating packages. The WORKSPACE file has been updated to io_ray, includes rules_java, and adds uv and platform-specific redis binaries. Various bazel/*.bzl files have been updated for consistency with the new workspace name, updated dependencies, and new Python macros.
• CI/CD Pipeline Refinements (Buildkite): Significant changes have been made to the Buildkite pipeline configurations. A new _images.rayci.yml file defines various image build steps, including extra variants. Python 3.10 has been added to several matrix builds, and cibase tags are now widely used. Dedicated build steps for ray-core, ray-dashboard, and ray-java have been introduced. Test configurations have been streamlined, with cgraph python tests and workflow tests removed, and dask & modin tests split. The lint.rayci.yml now includes semgrep_lint and adjusts docbuild environments. Automated dependency compilation is now handled by a new dependencies.rayci.yml file and raydepsets tool. macOS builds have been consolidated to macos-arm64, and RAYCI_DISABLE_TEST_DB is now used in release automation. Dockerfiles for forge and manylinux have been updated to use miniforge3 and uv, and include Node.js and Azure CLI installations.
• Codebase and Documentation Updates: The .github/CODEOWNERS file has been simplified and updated to reflect new team responsibilities and file structures. The .github/PULL_REQUEST_TEMPLATE.md has been revised for clarity. .gitignore now includes new entries for build artifacts and user configurations. The .pre-commit-config.yaml has been updated with new pydoclint and semgrep hooks, and expanded buildifier and cpplint coverage. The .rayciversion has been incremented. The .readthedocs.yaml now points to a specific lock file for documentation dependencies. .vale configuration files have been updated with new rules and vocabulary. C++ code has seen updates to actor_creator.h, actor_task_caller.h, metric.h, ray_runtime.h, task_caller.h, config_internal.cc, abstract_ray_runtime.cc, metric.cc, native_ray_runtime.cc, local_mode_object_store.cc, runtime_env.cc, local_mode_task_submitter.cc, native_task_submitter.cc, task_executor.cc, api_test.cc, cluster_mode_test.cc, counter.cc, simple_kv_store.cc, process_helper.cc, process_helper.h, util.cc, and util.h for consistency, bug fixes, and improved network utility. The cpp/test_submit_cpp_job.py has been updated to remove _node_ip_address from _ray_start.

Ignored Files

• Ignored by pattern: .gemini/** (1)
  • .gemini/config.yaml
• Ignored by pattern: .github/workflows/** (1)
  • .github/workflows/stale_pull_request.yaml

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request is a large automated merge that includes a significant amount of refactoring across the build system, CI configuration, and dependency management. Key changes include a major overhaul of the Bazel build system, including changing the workspace name to io_ray, breaking down the monolithic root BUILD.bazel file, and modernizing packaging rules. It also features a comprehensive refactoring of the Buildkite CI pipelines, a switch from miniconda to miniforge, and the introduction of a new dependency management tool raydepsets. Additionally, there's an expansion of aarch64 support and standardization on arm64 for macOS CI, along with numerous cleanups and improvements to tests, linting, and documentation. Overall, these changes are a massive step forward for the project's infrastructure. My review focuses on a few specific areas that could be improved or have significant impact.

[gemini-code-assist]

The workspace name has been changed from com_github_ray_project_ray to io_ray. This is a significant breaking change for any downstream projects that depend on Ray via Bazel. While this is likely intentional as part of the build system overhaul, it's a major change that should be clearly communicated to the community.

[gemini-code-assist]

The Metric::Record function has been changed to construct a std::vector<std::pair<std::string_view, std::string>> from the input std::unordered_map. This involves iterating over the map and creating a new vector for every call. Given that metrics can be recorded frequently in performance-sensitive code, this conversion could introduce a non-trivial overhead.

If possible, consider changing the underlying stats::Metric::Record to accept const std::unordered_map<std::string, std::string> & directly to avoid this conversion. If that's not feasible, this change is acceptable, but the performance implication is worth noting.

[gemini-code-assist]

The script has been updated to use ray symmetric-run, which is a great simplification. However, the method for determining the head node IP address has changed. The new version uses the hostname directly (ip_head=$head_node:$port), which assumes that the hostname is resolvable to the correct IP address from all worker nodes. This might not be true in all network configurations.

The previous version was more robust as it used hostname --ip-address to explicitly get the IP address. To avoid potential issues in different environments, I recommend reverting to a more explicit IP discovery method.

Suggested change

	head_node=${nodes_array[0]}
	head_node_ip=$(srun --nodes=1 --ntasks=1 -w "$head_node" hostname --ip-address)
	# if we detect a space character in the head node IP, we'll
	# convert it to an ipv4 address. This step is optional.
	if [[ "$head_node_ip" == *" "* ]]; then
	IFS=' ' read -ra ADDR <<<"$head_node_ip"
	if [[ ${#ADDR[0]} -gt 16 ]]; then
	head_node_ip=${ADDR[1]}
	else
	head_node_ip=${ADDR[0]}
	fi
	echo "IPV6 address detected. We split the IPV4 address as $head_node_ip"
	fi
	# __doc_head_address_end__
	# __doc_head_ray_start__
	port=6379
	ip_head=$head_node_ip:$port
	ip_head=$head_node:$port
	export ip_head
	echo "IP Head: $ip_head"
	# __doc_head_address_end__
	head_node=${nodes_array[0]}
	head_node_ip=$(srun --nodes=1 --ntasks=1 -w "$head_node" hostname --ip-address | awk '{print $1}')
	port=6379
	ip_head=${head_node_ip}:${port}
	export ip_head
	echo "IP Head: $ip_head"
	# __doc_head_address_end__

[github-actions]

This pull request has been automatically marked as stale because it has not had
any activity for 14 days. It will be closed in another 14 days if no further activity occurs.
Thank you for your contributions.

You can always ask for help on our discussion forum or Ray's public slack channel.

If you'd like to keep this open, just leave any comment, and the stale label will be removed.