mirror of
https://github.com/openai/codex.git
synced 2026-04-30 11:21:34 +03:00
343d1af3da
## Why This PR is the current, consolidated follow-up to the earlier Windows Bazel attempt in #11229. The goal is no longer just to get a tiny Windows smoke job limping along: it is to make the ordinary Bazel CI path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with the same broad `//...` test shape that macOS and Linux already use. The earlier smoke-list version of this work was useful as a foothold, but it was not a good long-term landing point. Windows Bazel kept surfacing real issues outside that allowlist: - GitHub's Windows runner exposed runfiles-manifest bugs such as `FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even when the manifest file existed. - `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed `windows-gnullvm`-specific fixes for our hermetic toolchain. - the V8 path needed more work than just turning the Windows matrix entry back on: `rusty_v8` does not ship Windows GNU artifacts in the same shape we need, and Bazel's in-tree V8 build needed a set of Windows GNU portability fixes. Windows performance pressure also pushed this toward a full solution instead of a permanent smoke suite. During this investigation we hit targets such as `//codex-rs/shell-command:shell-command-unit-tests` that were much more expensive on Windows because they repeatedly spawn real PowerShell parsers (see #16057 for one concrete example of that pressure). That made it much more valuable to get the real Windows Bazel path working than to keep iterating on a narrowly curated subset. The net result is that this PR now aims for the same CI contract on Windows that we already expect elsewhere: keep standalone `//third_party/v8:all` out of the ordinary Bazel lane, but allow V8 consumers under `//codex-rs/...` to build and test transitively through `//...`. ## What Changed ### CI and workflow wiring - re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel matrix entry in `.github/workflows/bazel.yml` - move the Windows Bazel output root to `D:\b` and enable `git config --global core.longpaths true` in `.github/actions/setup-bazel-ci/action.yml` - keep the ordinary Bazel target set on Windows aligned with macOS and Linux by running `//...` while excluding only standalone `//third_party/v8:all` targets from the normal lane ### Toolchain and module support for `windows-gnullvm` - patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows exec/toolchain platform instead of collapsing into the generic Windows shape - patch `rules_rust` build-script environment handling so llvm-mingw build-script probes do not inherit unsupported `-fstack-protector*` flags - patch the LLVM module archive so it extracts cleanly on Windows and provides the MinGW libraries this toolchain needs - patch Abseil so its thread-local identity path matches the hermetic `windows-gnullvm` toolchain instead of taking an incompatible MinGW pthread path - keep both MSVC and GNU Windows targets in the generated Cargo metadata because the current V8 release-asset story still uses MSVC-shaped names in some places while the Bazel build targets the GNU ABI ### Windows test-launch and binary-behavior fixes - update `workspace_root_test_launcher.bat.tpl` to read the runfiles manifest directly instead of shelling out to `findstr`, which was the source of the `D:MANIFEST` failures on the GitHub Windows runner - thread a larger Windows GNU stack reserve through `defs.bzl` so Bazel-built binaries that pull in V8 behave correctly both under normal builds and under `bazel test` - remove the no-longer-needed Windows bootstrap sh-toolchain override from `.bazelrc` ### V8 / `rusty_v8` Windows GNU support - export and apply the new Windows GNU patch set from `patches/BUILD.bazel` / `MODULE.bazel` - patch the V8 module/rules/source layers so the in-tree V8 build can produce Windows GNU archives under Bazel - teach `third_party/v8/BUILD.bazel` to build Windows GNU static archives in-tree instead of aliasing them to the MSVC prebuilts - reuse the Linux release binding for the experimental Windows GNU path where `rusty_v8` does not currently publish a Windows GNU binding artifact ## Testing - the primary end-to-end validation for this work is the `Bazel` workflow plus `v8-canary`, since the hard parts are Windows-specific and depend on real GitHub runner behavior - before consolidation back onto this PR, the same net change passed the full Bazel matrix in [run 23675590471](https://github.com/openai/codex/actions/runs/23675590471) and passed `v8-canary` in [run 23675590453](https://github.com/openai/codex/actions/runs/23675590453) - those successful runs included the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path, not the earlier Windows smoke allowlist --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952). * #16067 * __->__ #15952
276 lines
9.3 KiB
Diff
276 lines
9.3 KiB
Diff
# What: adapt upstream V8 Bazel rules to this workspace's hermetic toolchains
|
|
# and externally provided dependencies.
|
|
# Scope: Bazel BUILD/defs/BUILD.icu integration only, including dependency
|
|
# wiring, generated sources, and visibility; no standalone V8 source patching.
|
|
|
|
diff --git a/orig/v8-14.6.202.11/bazel/defs.bzl b/mod/v8-14.6.202.11/bazel/defs.bzl
|
|
index 9648e4a..88efd41 100644
|
|
--- a/orig/v8-14.6.202.11/bazel/defs.bzl
|
|
+++ b/mod/v8-14.6.202.11/bazel/defs.bzl
|
|
@@ -97,7 +97,7 @@ v8_config = rule(
|
|
|
|
def _default_args():
|
|
return struct(
|
|
- deps = [":define_flags", "@libcxx//:libc++"],
|
|
+ deps = [":define_flags"],
|
|
defines = select({
|
|
"@v8//bazel/config:is_windows": [
|
|
"UNICODE",
|
|
@@ -128,12 +128,6 @@ def _default_args():
|
|
],
|
|
"//conditions:default": [],
|
|
}) + select({
|
|
- "@v8//bazel/config:is_clang": [
|
|
- "-Wno-invalid-offsetof",
|
|
- "-Wno-deprecated-this-capture",
|
|
- "-Wno-deprecated-declarations",
|
|
- "-std=c++20",
|
|
- ],
|
|
"@v8//bazel/config:is_gcc": [
|
|
"-Wno-extra",
|
|
"-Wno-array-bounds",
|
|
@@ -155,7 +149,15 @@ def _default_args():
|
|
- "@v8//bazel/config:is_windows": [
|
|
- "/std:c++20",
|
|
- ],
|
|
- "//conditions:default": [],
|
|
+ "@v8//bazel/config:is_windows": [
|
|
+ "-Wno-invalid-offsetof",
|
|
+ "-Wno-deprecated-this-capture",
|
|
+ "-Wno-deprecated-declarations",
|
|
+ "-std=c++20",
|
|
+ ],
|
|
+ "//conditions:default": [
|
|
+ "-Wno-invalid-offsetof",
|
|
+ "-Wno-deprecated-this-capture",
|
|
+ "-Wno-deprecated-declarations",
|
|
+ "-std=c++20",
|
|
+ ],
|
|
}) + select({
|
|
"@v8//bazel/config:is_gcc_fastbuild": [
|
|
# Non-debug builds without optimizations fail because
|
|
@@ -180,10 +179,10 @@ def _default_args():
|
|
"@v8//bazel/config:is_windows": [
|
|
- "Winmm.lib",
|
|
- "DbgHelp.lib",
|
|
- "Advapi32.lib",
|
|
+ "-lwinmm",
|
|
+ "-ldbghelp",
|
|
+ "-ladvapi32",
|
|
],
|
|
"@v8//bazel/config:is_macos": ["-pthread"],
|
|
- "//conditions:default": ["-Wl,--no-as-needed -ldl -latomic -pthread"],
|
|
+ "//conditions:default": ["-Wl,--no-as-needed -ldl -pthread"],
|
|
}) + select({
|
|
":should_add_rdynamic": ["-rdynamic"],
|
|
"//conditions:default": [],
|
|
diff --git a/orig/v8-14.6.202.11/BUILD.bazel b/mod/v8-14.6.202.11/BUILD.bazel
|
|
index 85f31b7..7314584 100644
|
|
--- a/orig/v8-14.6.202.11/BUILD.bazel
|
|
+++ b/mod/v8-14.6.202.11/BUILD.bazel
|
|
@@ -303,7 +303,7 @@ v8_int(
|
|
# If no explicit value for v8_enable_pointer_compression, we set it to 'none'.
|
|
v8_string(
|
|
name = "v8_enable_pointer_compression",
|
|
- default = "none",
|
|
+ default = "False",
|
|
)
|
|
|
|
# Default setting for v8_enable_pointer_compression.
|
|
@@ -4077,28 +4077,14 @@ filegroup(
|
|
}),
|
|
)
|
|
|
|
-v8_library(
|
|
- name = "lib_dragonbox",
|
|
- srcs = ["third_party/dragonbox/src/include/dragonbox/dragonbox.h"],
|
|
- hdrs = [
|
|
- "third_party/dragonbox/src/include/dragonbox/dragonbox.h",
|
|
- ],
|
|
- includes = [
|
|
- "third_party/dragonbox/src/include",
|
|
- ],
|
|
+alias(
|
|
+ name = "lib_dragonbox",
|
|
+ actual = "@dragonbox//:dragonbox",
|
|
)
|
|
|
|
-v8_library(
|
|
- name = "lib_fp16",
|
|
- srcs = ["third_party/fp16/src/include/fp16.h"],
|
|
- hdrs = [
|
|
- "third_party/fp16/src/include/fp16/fp16.h",
|
|
- "third_party/fp16/src/include/fp16/bitcasts.h",
|
|
- "third_party/fp16/src/include/fp16/macros.h",
|
|
- ],
|
|
- includes = [
|
|
- "third_party/fp16/src/include",
|
|
- ],
|
|
+alias(
|
|
+ name = "lib_fp16",
|
|
+ actual = "@fp16//:fp16",
|
|
)
|
|
|
|
filegroup(
|
|
@@ -4405,6 +4391,20 @@ genrule(
|
|
srcs = [
|
|
"include/js_protocol.pdl",
|
|
"src/inspector/inspector_protocol_config.json",
|
|
+ "third_party/inspector_protocol/code_generator.py",
|
|
+ "third_party/inspector_protocol/pdl.py",
|
|
+ "third_party/inspector_protocol/lib/Forward_h.template",
|
|
+ "third_party/inspector_protocol/lib/Object_cpp.template",
|
|
+ "third_party/inspector_protocol/lib/Object_h.template",
|
|
+ "third_party/inspector_protocol/lib/Protocol_cpp.template",
|
|
+ "third_party/inspector_protocol/lib/ValueConversions_cpp.template",
|
|
+ "third_party/inspector_protocol/lib/ValueConversions_h.template",
|
|
+ "third_party/inspector_protocol/lib/Values_cpp.template",
|
|
+ "third_party/inspector_protocol/lib/Values_h.template",
|
|
+ "third_party/inspector_protocol/templates/Exported_h.template",
|
|
+ "third_party/inspector_protocol/templates/Imported_h.template",
|
|
+ "third_party/inspector_protocol/templates/TypeBuilder_cpp.template",
|
|
+ "third_party/inspector_protocol/templates/TypeBuilder_h.template",
|
|
],
|
|
outs = [
|
|
"include/inspector/Debugger.h",
|
|
@@ -4426,15 +4426,18 @@ genrule(
|
|
"src/inspector/protocol/Schema.cpp",
|
|
"src/inspector/protocol/Schema.h",
|
|
],
|
|
- cmd = "$(location :code_generator) --jinja_dir . \
|
|
- --inspector_protocol_dir third_party/inspector_protocol \
|
|
+ cmd = "INSPECTOR_PROTOCOL_DIR=$$(dirname $(execpath third_party/inspector_protocol/code_generator.py)); \
|
|
+ PYTHONPATH=$$INSPECTOR_PROTOCOL_DIR:external/rules_python++pip+v8_python_deps_311_jinja2/site-packages:external/rules_python++pip+v8_python_deps_311_markupsafe/site-packages:$${PYTHONPATH-} \
|
|
+ $(execpath @rules_python//python/bin:python) $(execpath third_party/inspector_protocol/code_generator.py) --jinja_dir . \
|
|
+ --inspector_protocol_dir $$INSPECTOR_PROTOCOL_DIR \
|
|
--config $(location :src/inspector/inspector_protocol_config.json) \
|
|
--config_value protocol.path=$(location :include/js_protocol.pdl) \
|
|
--output_base $(@D)/src/inspector",
|
|
- local = 1,
|
|
message = "Generating inspector files",
|
|
tools = [
|
|
- ":code_generator",
|
|
+ "@rules_python//python/bin:python",
|
|
+ requirement("jinja2"),
|
|
+ requirement("markupsafe"),
|
|
],
|
|
)
|
|
|
|
@@ -4448,6 +4451,15 @@ filegroup(
|
|
],
|
|
)
|
|
|
|
+cc_library(
|
|
+ name = "rusty_v8_internal_headers",
|
|
+ hdrs = [
|
|
+ "src/libplatform/default-platform.h",
|
|
+ ],
|
|
+ strip_include_prefix = "",
|
|
+ visibility = ["//visibility:public"],
|
|
+)
|
|
+
|
|
filegroup(
|
|
name = "d8_files",
|
|
srcs = [
|
|
@@ -4567,16 +4579,9 @@ cc_library(
|
|
],
|
|
)
|
|
|
|
-cc_library(
|
|
- name = "simdutf",
|
|
- srcs = ["third_party/simdutf/simdutf.cpp"],
|
|
- hdrs = ["third_party/simdutf/simdutf.h"],
|
|
- copts = select({
|
|
- "@v8//bazel/config:is_clang": ["-std=c++20"],
|
|
- "@v8//bazel/config:is_gcc": ["-std=gnu++2a"],
|
|
- "@v8//bazel/config:is_windows": ["/std:c++20"],
|
|
- "//conditions:default": [],
|
|
- }),
|
|
+alias(
|
|
+ name = "simdutf",
|
|
+ actual = "@simdutf//:simdutf",
|
|
)
|
|
|
|
v8_library(
|
|
@@ -4593,7 +4598,7 @@ v8_library(
|
|
copts = ["-Wno-implicit-fallthrough"],
|
|
icu_deps = [
|
|
":icu/generated_torque_definitions_headers",
|
|
- "//external:icu",
|
|
+ "@icu//:icu",
|
|
],
|
|
icu_srcs = [
|
|
":generated_regexp_special_case",
|
|
@@ -4608,7 +4613,7 @@ v8_library(
|
|
],
|
|
deps = [
|
|
":lib_dragonbox",
|
|
- "//third_party/fast_float/src:fast_float",
|
|
+ "@fast_float//:fast_float",
|
|
":lib_fp16",
|
|
":simdutf",
|
|
":v8_libbase",
|
|
@@ -4664,6 +4669,7 @@ alias(
|
|
alias(
|
|
name = "core_lib_icu",
|
|
actual = "icu/v8",
|
|
+ visibility = ["//visibility:public"],
|
|
)
|
|
|
|
v8_library(
|
|
@@ -4715,7 +4721,7 @@ v8_binary(
|
|
],
|
|
deps = [
|
|
":v8_libbase",
|
|
- "//external:icu",
|
|
+ "@icu//:icu",
|
|
],
|
|
)
|
|
|
|
diff --git a/orig/v8-14.6.202.11/bazel/BUILD.icu b/mod/v8-14.6.202.11/bazel/BUILD.icu
|
|
index 5fda2f4..381386c 100644
|
|
--- a/orig/v8-14.6.202.11/bazel/BUILD.icu
|
|
+++ b/mod/v8-14.6.202.11/bazel/BUILD.icu
|
|
@@ -1,3 +1,5 @@
|
|
+load("@rules_cc//cc:defs.bzl", "cc_library")
|
|
+
|
|
# Copyright 2021 the V8 project authors. All rights reserved.
|
|
# Use of this source code is governed by a BSD-style license that can be
|
|
# found in the LICENSE file.
|
|
@@ -16,10 +18,7 @@ cc_library(
|
|
]),
|
|
copts = select({
|
|
"@platforms//os:windows": [
|
|
- "/wd4005", # Macro redefinition.
|
|
- "/wd4068", # Unknown pragmas.
|
|
- "/wd4267", # Conversion from size_t on 64-bits.
|
|
- "/utf-8", # ICU source files are in UTF-8.
|
|
+ "-Wno-deprecated-declarations",
|
|
],
|
|
"//conditions:default": [
|
|
"-Wno-deprecated-declarations",
|
|
@@ -65,10 +64,7 @@ cc_library(
|
|
]),
|
|
copts = select({
|
|
"@platforms//os:windows": [
|
|
- "/wd4005", # Macro redefinition.
|
|
- "/wd4068", # Unknown pragmas.
|
|
- "/wd4267", # Conversion from size_t on 64-bits.
|
|
- "/utf-8", # ICU source files are in UTF-8.
|
|
+ "-Wno-deprecated-declarations",
|
|
],
|
|
"//conditions:default": [
|
|
"-Wno-deprecated-declarations",
|
|
@@ -93,10 +89,7 @@ cc_library(
|
|
]),
|
|
copts = select({
|
|
"@platforms//os:windows": [
|
|
- "/wd4005", # Macro redefinition.
|
|
- "/wd4068", # Unknown pragmas.
|
|
- "/wd4267", # Conversion from size_t on 64-bits.
|
|
- "/utf-8", # ICU source files are in UTF-8.
|
|
+ "-Wno-deprecated-declarations",
|
|
],
|
|
"//conditions:default": [],
|
|
}),
|