mirror of
https://github.com/openai/codex.git
synced 2026-04-28 02:11:08 +03:00
343d1af3da
## Why This PR is the current, consolidated follow-up to the earlier Windows Bazel attempt in #11229. The goal is no longer just to get a tiny Windows smoke job limping along: it is to make the ordinary Bazel CI path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with the same broad `//...` test shape that macOS and Linux already use. The earlier smoke-list version of this work was useful as a foothold, but it was not a good long-term landing point. Windows Bazel kept surfacing real issues outside that allowlist: - GitHub's Windows runner exposed runfiles-manifest bugs such as `FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even when the manifest file existed. - `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed `windows-gnullvm`-specific fixes for our hermetic toolchain. - the V8 path needed more work than just turning the Windows matrix entry back on: `rusty_v8` does not ship Windows GNU artifacts in the same shape we need, and Bazel's in-tree V8 build needed a set of Windows GNU portability fixes. Windows performance pressure also pushed this toward a full solution instead of a permanent smoke suite. During this investigation we hit targets such as `//codex-rs/shell-command:shell-command-unit-tests` that were much more expensive on Windows because they repeatedly spawn real PowerShell parsers (see #16057 for one concrete example of that pressure). That made it much more valuable to get the real Windows Bazel path working than to keep iterating on a narrowly curated subset. The net result is that this PR now aims for the same CI contract on Windows that we already expect elsewhere: keep standalone `//third_party/v8:all` out of the ordinary Bazel lane, but allow V8 consumers under `//codex-rs/...` to build and test transitively through `//...`. ## What Changed ### CI and workflow wiring - re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel matrix entry in `.github/workflows/bazel.yml` - move the Windows Bazel output root to `D:\b` and enable `git config --global core.longpaths true` in `.github/actions/setup-bazel-ci/action.yml` - keep the ordinary Bazel target set on Windows aligned with macOS and Linux by running `//...` while excluding only standalone `//third_party/v8:all` targets from the normal lane ### Toolchain and module support for `windows-gnullvm` - patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows exec/toolchain platform instead of collapsing into the generic Windows shape - patch `rules_rust` build-script environment handling so llvm-mingw build-script probes do not inherit unsupported `-fstack-protector*` flags - patch the LLVM module archive so it extracts cleanly on Windows and provides the MinGW libraries this toolchain needs - patch Abseil so its thread-local identity path matches the hermetic `windows-gnullvm` toolchain instead of taking an incompatible MinGW pthread path - keep both MSVC and GNU Windows targets in the generated Cargo metadata because the current V8 release-asset story still uses MSVC-shaped names in some places while the Bazel build targets the GNU ABI ### Windows test-launch and binary-behavior fixes - update `workspace_root_test_launcher.bat.tpl` to read the runfiles manifest directly instead of shelling out to `findstr`, which was the source of the `D:MANIFEST` failures on the GitHub Windows runner - thread a larger Windows GNU stack reserve through `defs.bzl` so Bazel-built binaries that pull in V8 behave correctly both under normal builds and under `bazel test` - remove the no-longer-needed Windows bootstrap sh-toolchain override from `.bazelrc` ### V8 / `rusty_v8` Windows GNU support - export and apply the new Windows GNU patch set from `patches/BUILD.bazel` / `MODULE.bazel` - patch the V8 module/rules/source layers so the in-tree V8 build can produce Windows GNU archives under Bazel - teach `third_party/v8/BUILD.bazel` to build Windows GNU static archives in-tree instead of aliasing them to the MSVC prebuilts - reuse the Linux release binding for the experimental Windows GNU path where `rusty_v8` does not currently publish a Windows GNU binding artifact ## Testing - the primary end-to-end validation for this work is the `Bazel` workflow plus `v8-canary`, since the hard parts are Windows-specific and depend on real GitHub runner behavior - before consolidation back onto this PR, the same net change passed the full Bazel matrix in [run 23675590471](https://github.com/openai/codex/actions/runs/23675590471) and passed `v8-canary` in [run 23675590453](https://github.com/openai/codex/actions/runs/23675590453) - those successful runs included the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path, not the earlier Windows smoke allowlist --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952). * #16067 * __->__ #15952
113 lines
4.7 KiB
Plaintext
113 lines
4.7 KiB
Plaintext
common --repo_env=BAZEL_DO_NOT_DETECT_CPP_TOOLCHAIN=1
|
|
common --repo_env=BAZEL_NO_APPLE_CPP_TOOLCHAIN=1
|
|
# Dummy xcode config so we don't need to build xcode_locator in repo rule.
|
|
common --xcode_version_config=//:disable_xcode
|
|
|
|
common --disk_cache=~/.cache/bazel-disk-cache
|
|
common --repo_contents_cache=~/.cache/bazel-repo-contents-cache
|
|
common --repository_cache=~/.cache/bazel-repo-cache
|
|
common --remote_cache_compression
|
|
startup --experimental_remote_repo_contents_cache
|
|
|
|
common --experimental_platform_in_output_dir
|
|
|
|
# Runfiles strategy rationale: codex-rs/utils/cargo-bin/README.md
|
|
common --noenable_runfiles
|
|
|
|
common --enable_platform_specific_config
|
|
common:linux --host_platform=//:local_linux
|
|
common:windows --host_platform=//:local_windows
|
|
common --@rules_cc//cc/toolchains/args/archiver_flags:use_libtool_on_macos=False
|
|
common --@llvm//config:experimental_stub_libgcc_s
|
|
|
|
# TODO(zbarsky): rules_rust doesn't implement this flag properly with remote exec...
|
|
# common --@rules_rust//rust/settings:pipelined_compilation
|
|
|
|
common --incompatible_strict_action_env
|
|
# Not ideal, but We need to allow dotslash to be found
|
|
common:linux --test_env=PATH=/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
|
|
common:macos --test_env=PATH=/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin
|
|
|
|
# Pass through some env vars Windows needs to use powershell?
|
|
common:windows --test_env=PATH
|
|
common:windows --test_env=SYSTEMROOT
|
|
common:windows --test_env=COMSPEC
|
|
common:windows --test_env=WINDIR
|
|
|
|
common --test_output=errors
|
|
common --bes_results_url=https://app.buildbuddy.io/invocation/
|
|
common --bes_backend=grpcs://remote.buildbuddy.io
|
|
common --remote_cache=grpcs://remote.buildbuddy.io
|
|
common --remote_download_toplevel
|
|
common --nobuild_runfile_links
|
|
common --remote_timeout=3600
|
|
common --noexperimental_throttle_remote_action_building
|
|
common --experimental_remote_execution_keepalive
|
|
common --grpc_keepalive_time=30s
|
|
|
|
# This limits both in-flight executions and concurrent downloads. Even with high number
|
|
# of jobs execution will still be limited by CPU cores, so this just pays a bit of
|
|
# memory in exchange for higher download concurrency.
|
|
common --jobs=30
|
|
|
|
common:remote --extra_execution_platforms=//:rbe
|
|
common:remote --remote_executor=grpcs://remote.buildbuddy.io
|
|
common:remote --jobs=800
|
|
# TODO(team): Evaluate if this actually helps, zbarsky is not sure, everything seems bottlenecked on `core` either way.
|
|
# Enable pipelined compilation since we are not bound by local CPU count.
|
|
#common:remote --@rules_rust//rust/settings:pipelined_compilation
|
|
|
|
# GitHub Actions CI configs.
|
|
common:ci --remote_download_minimal
|
|
common:ci --keep_going
|
|
common:ci --verbose_failures
|
|
common:ci --build_metadata=REPO_URL=https://github.com/openai/codex.git
|
|
common:ci --build_metadata=ROLE=CI
|
|
common:ci --build_metadata=VISIBILITY=PUBLIC
|
|
|
|
# Disable disk cache in CI since we have a remote one and aren't using persistent workers.
|
|
common:ci --disk_cache=
|
|
|
|
# Shared config for the main Bazel CI workflow.
|
|
common:ci-bazel --config=ci
|
|
common:ci-bazel --build_metadata=TAG_workflow=bazel
|
|
|
|
# Shared config for Bazel-backed Rust linting.
|
|
build:clippy --aspects=@rules_rust//rust:defs.bzl%rust_clippy_aspect
|
|
build:clippy --output_groups=+clippy_checks
|
|
build:clippy --@rules_rust//rust/settings:clippy.toml=//codex-rs:clippy.toml
|
|
|
|
# Rearrange caches on Windows so they're on the same volume as the checkout.
|
|
common:ci-windows --config=ci-bazel
|
|
common:ci-windows --build_metadata=TAG_os=windows
|
|
common:ci-windows --repo_contents_cache=D:/a/.cache/bazel-repo-contents-cache
|
|
common:ci-windows --repository_cache=D:/a/.cache/bazel-repo-cache
|
|
|
|
# We prefer to run the build actions entirely remotely so we can dial up the concurrency.
|
|
# We have platform-specific tests, so we want to execute the tests on all platforms using the strongest sandboxing available on each platform.
|
|
|
|
# On linux, we can do a full remote build/test, by targeting the right (x86/arm) runners, so we have coverage of both.
|
|
# Linux crossbuilds don't work until we untangle the libc constraint mess.
|
|
common:ci-linux --config=ci-bazel
|
|
common:ci-linux --build_metadata=TAG_os=linux
|
|
common:ci-linux --config=remote
|
|
common:ci-linux --strategy=remote
|
|
common:ci-linux --platforms=//:rbe
|
|
|
|
# On mac, we can run all the build actions remotely but test actions locally.
|
|
common:ci-macos --config=ci-bazel
|
|
common:ci-macos --build_metadata=TAG_os=macos
|
|
common:ci-macos --config=remote
|
|
common:ci-macos --strategy=remote
|
|
common:ci-macos --strategy=TestRunner=darwin-sandbox,local
|
|
|
|
# Linux-only V8 CI config.
|
|
common:ci-v8 --config=ci
|
|
common:ci-v8 --build_metadata=TAG_workflow=v8
|
|
common:ci-v8 --build_metadata=TAG_os=linux
|
|
common:ci-v8 --config=remote
|
|
common:ci-v8 --strategy=remote
|
|
|
|
# Optional per-user local overrides.
|
|
try-import %workspace%/user.bazelrc
|