Compare commits

...

11 Commits

Author SHA1 Message Date
mikhail-oai
1f24a3e2c7 Remove redundant clone in auth storage test 2026-03-20 16:20:45 -04:00
mikhail-oai
15563e1e9d Use storage helpers in auth and OAuth keyring tests 2026-03-20 16:20:45 -04:00
mikhail-oai
e0f0c0ff1a Use platform spec based on type of storage to use 2026-03-20 16:20:45 -04:00
mikhail-oai
b3b7c81d21 cleaning up unnecessary code and exposed API 2026-03-20 16:20:45 -04:00
mikhail-oai
8446a7349e revision timestamp is not necessary 2026-03-20 16:20:45 -04:00
mikhail-oai
52a50a6ad5 windows gating 2026-03-20 16:20:45 -04:00
mikhail-oai
482e2d0287 scope json breakout to windows only 2026-03-20 16:20:45 -04:00
mikhail-oai
d2b2ee88fc generalize json serialization and unify MCP/Auth codepaths. 2026-03-20 16:20:45 -04:00
mikhail-oai
b6340cffa2 Use binary split-entry keyring storage for auth to avoid Windows credential size failures 2026-03-20 16:20:45 -04:00
Channing Conger
1350477150 Add v8-poc consumer of our new built v8 (#15203)
This adds a dummy v8-poc project that in Cargo links against our
prebuilt binaries and the ones provided by rusty_v8 for non musl
platforms. This demonstrates that we can successfully link and use v8 on
all platforms that we want to target.

In bazel things are slightly more complicated. Since the libraries as
published have libc++ linked in already we end up with a lot of double
linked symbols if we try to use them in bazel land. Instead we fall back
to building rusty_v8 and v8 from source (cached of course) on the
platforms we ship to.

There is likely some compatibility drift in the windows bazel builder
that we'll need to reconcile before we can re-enable them. I'm happy to
be on the hook to unwind that.
2026-03-20 12:08:25 -07:00
Channing Conger
a941d8439d Bump aws-lc-rs (#15337)
Bump our dep.

RUSTSEC-2026-0048
Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0048
2026-03-20 18:59:13 +00:00
23 changed files with 2224 additions and 261 deletions

View File

@@ -447,6 +447,24 @@ jobs:
echo "CFLAGS=${cflags}" >> "$GITHUB_ENV"
echo "CXXFLAGS=${cxxflags}" >> "$GITHUB_ENV"
- if: ${{ matrix.target == 'x86_64-unknown-linux-musl' || matrix.target == 'aarch64-unknown-linux-musl' }}
name: Configure musl rusty_v8 artifact overrides
env:
TARGET: ${{ matrix.target }}
shell: bash
run: |
set -euo pipefail
version="$(python3 "${GITHUB_WORKSPACE}/.github/scripts/rusty_v8_bazel.py" resolved-v8-crate-version)"
release_tag="rusty-v8-v${version}"
base_url="https://github.com/openai/codex/releases/download/${release_tag}"
archive="https://github.com/openai/codex/releases/download/rusty-v8-v${version}/librusty_v8_release_${TARGET}.a.gz"
binding_dir="${RUNNER_TEMP}/rusty_v8"
binding_path="${binding_dir}/src_binding_release_${TARGET}.rs"
mkdir -p "${binding_dir}"
curl -fsSL "${base_url}/src_binding_release_${TARGET}.rs" -o "${binding_path}"
echo "RUSTY_V8_ARCHIVE=${archive}" >> "$GITHUB_ENV"
echo "RUSTY_V8_SRC_BINDING_PATH=${binding_path}" >> "$GITHUB_ENV"
- name: Install cargo-chef
if: ${{ matrix.profile == 'release' }}
uses: taiki-e/install-action@44c6d64aa62cd779e873306675c7a58e86d6d532 # v2

View File

@@ -210,6 +210,24 @@ jobs:
echo "CFLAGS=${cflags}" >> "$GITHUB_ENV"
echo "CXXFLAGS=${cxxflags}" >> "$GITHUB_ENV"
- if: ${{ matrix.target == 'x86_64-unknown-linux-musl' || matrix.target == 'aarch64-unknown-linux-musl' }}
name: Configure musl rusty_v8 artifact overrides
env:
TARGET: ${{ matrix.target }}
shell: bash
run: |
set -euo pipefail
version="$(python3 "${GITHUB_WORKSPACE}/.github/scripts/rusty_v8_bazel.py" resolved-v8-crate-version)"
release_tag="rusty-v8-v${version}"
base_url="https://github.com/openai/codex/releases/download/${release_tag}"
archive="https://github.com/openai/codex/releases/download/rusty-v8-v${version}/librusty_v8_release_${TARGET}.a.gz"
binding_dir="${RUNNER_TEMP}/rusty_v8"
binding_path="${binding_dir}/src_binding_release_${TARGET}.rs"
mkdir -p "${binding_dir}"
curl -fsSL "${base_url}/src_binding_release_${TARGET}.rs" -o "${binding_path}"
echo "RUSTY_V8_ARCHIVE=${archive}" >> "$GITHUB_ENV"
echo "RUSTY_V8_SRC_BINDING_PATH=${binding_path}" >> "$GITHUB_ENV"
- name: Cargo build
shell: bash
run: |

View File

@@ -144,6 +144,33 @@ crate.annotation(
)
http_archive = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_file = use_repo_rule("@bazel_tools//tools/build_defs/repo:http.bzl", "http_file")
new_local_repository = use_repo_rule("@bazel_tools//tools/build_defs/repo:local.bzl", "new_local_repository")
new_local_repository(
name = "v8_targets",
build_file = "//third_party/v8:BUILD.bazel",
path = "third_party/v8",
)
crate.annotation(
build_script_data = [
"@v8_targets//:rusty_v8_archive_for_target",
"@v8_targets//:rusty_v8_binding_for_target",
],
build_script_env = {
"RUSTY_V8_ARCHIVE": "$(execpath @v8_targets//:rusty_v8_archive_for_target)",
"RUSTY_V8_SRC_BINDING_PATH": "$(execpath @v8_targets//:rusty_v8_binding_for_target)",
},
crate = "v8",
gen_build_script = "on",
patch_args = ["-p1"],
patches = [
"//patches:rusty_v8_prebuilt_out_dir.patch",
],
)
inject_repo(crate, "v8_targets")
llvm = use_extension("@llvm//extensions:llvm.bzl", "llvm")
use_repo(llvm, "llvm-project")
@@ -210,6 +237,86 @@ http_archive(
urls = ["https://static.crates.io/crates/v8/v8-146.4.0.crate"],
)
http_file(
name = "rusty_v8_146_4_0_aarch64_apple_darwin_archive",
downloaded_file_path = "librusty_v8_release_aarch64-apple-darwin.a.gz",
urls = [
"https://github.com/denoland/rusty_v8/releases/download/v146.4.0/librusty_v8_release_aarch64-apple-darwin.a.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_aarch64_unknown_linux_gnu_archive",
downloaded_file_path = "librusty_v8_release_aarch64-unknown-linux-gnu.a.gz",
urls = [
"https://github.com/denoland/rusty_v8/releases/download/v146.4.0/librusty_v8_release_aarch64-unknown-linux-gnu.a.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_aarch64_pc_windows_msvc_archive",
downloaded_file_path = "rusty_v8_release_aarch64-pc-windows-msvc.lib.gz",
urls = [
"https://github.com/denoland/rusty_v8/releases/download/v146.4.0/rusty_v8_release_aarch64-pc-windows-msvc.lib.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_x86_64_apple_darwin_archive",
downloaded_file_path = "librusty_v8_release_x86_64-apple-darwin.a.gz",
urls = [
"https://github.com/denoland/rusty_v8/releases/download/v146.4.0/librusty_v8_release_x86_64-apple-darwin.a.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_x86_64_unknown_linux_gnu_archive",
downloaded_file_path = "librusty_v8_release_x86_64-unknown-linux-gnu.a.gz",
urls = [
"https://github.com/denoland/rusty_v8/releases/download/v146.4.0/librusty_v8_release_x86_64-unknown-linux-gnu.a.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_x86_64_pc_windows_msvc_archive",
downloaded_file_path = "rusty_v8_release_x86_64-pc-windows-msvc.lib.gz",
urls = [
"https://github.com/denoland/rusty_v8/releases/download/v146.4.0/rusty_v8_release_x86_64-pc-windows-msvc.lib.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_aarch64_unknown_linux_musl_archive",
downloaded_file_path = "librusty_v8_release_aarch64-unknown-linux-musl.a.gz",
urls = [
"https://github.com/openai/codex/releases/download/rusty-v8-v146.4.0/librusty_v8_release_aarch64-unknown-linux-musl.a.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_aarch64_unknown_linux_musl_binding",
downloaded_file_path = "src_binding_release_aarch64-unknown-linux-musl.rs",
urls = [
"https://github.com/openai/codex/releases/download/rusty-v8-v146.4.0/src_binding_release_aarch64-unknown-linux-musl.rs",
],
)
http_file(
name = "rusty_v8_146_4_0_x86_64_unknown_linux_musl_archive",
downloaded_file_path = "librusty_v8_release_x86_64-unknown-linux-musl.a.gz",
urls = [
"https://github.com/openai/codex/releases/download/rusty-v8-v146.4.0/librusty_v8_release_x86_64-unknown-linux-musl.a.gz",
],
)
http_file(
name = "rusty_v8_146_4_0_x86_64_unknown_linux_musl_binding",
downloaded_file_path = "src_binding_release_x86_64-unknown-linux-musl.rs",
urls = [
"https://github.com/openai/codex/releases/download/rusty-v8-v146.4.0/src_binding_release_x86_64-unknown-linux-musl.rs",
],
)
use_repo(crate, "crates")
bazel_dep(name = "libcap", version = "2.27.bcr.1")

23
MODULE.bazel.lock generated

File diff suppressed because one or more lines are too long

249
codex-rs/Cargo.lock generated
View File

@@ -800,9 +800,9 @@ checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "aws-lc-rs"
version = "1.15.4"
version = "1.16.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7b7b6141e96a8c160799cc2d5adecd5cbbe5054cb8c7c4af53da0f83bb7ad256"
checksum = "a054912289d18629dc78375ba2c3726a3afe3ff71b4edba9dedfca0e3446d1fc"
dependencies = [
"aws-lc-sys",
"untrusted 0.7.1",
@@ -811,9 +811,9 @@ dependencies = [
[[package]]
name = "aws-lc-sys"
version = "0.37.0"
version = "0.39.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c34dda4df7017c8db52132f0f8a2e0f8161649d15723ed63fc00c82d0f2081a"
checksum = "1fa7e52a4c5c547c741610a2c6f123f3881e409b714cd27e6798ef020c514f0a"
dependencies = [
"cc",
"cmake",
@@ -949,6 +949,8 @@ dependencies = [
"cexpr",
"clang-sys",
"itertools 0.13.0",
"log",
"prettyplease",
"proc-macro2",
"quote",
"regex",
@@ -1152,6 +1154,16 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ade8366b8bd5ba243f0a58f036cc0ca8a2f069cff1a2351ef1cac6b083e16fc0"
[[package]]
name = "calendrical_calculations"
version = "0.2.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a0b39595c6ee54a8d0900204ba4c401d0ab4eb45adaf07178e8d017541529e7"
dependencies = [
"core_maths",
"displaydoc",
]
[[package]]
name = "cassowary"
version = "0.3.0"
@@ -1584,7 +1596,7 @@ dependencies = [
"thiserror 2.0.18",
"tokio",
"url",
"which",
"which 8.0.0",
"wiremock",
"zip",
]
@@ -1930,7 +1942,7 @@ dependencies = [
"url",
"uuid",
"walkdir",
"which",
"which 8.0.0",
"wildmatch",
"windows-sys 0.52.0",
"wiremock",
@@ -2146,6 +2158,9 @@ name = "codex-keyring-store"
version = "0.0.0"
dependencies = [
"keyring",
"pretty_assertions",
"serde",
"serde_json",
"tracing",
]
@@ -2179,7 +2194,7 @@ dependencies = [
"serde_json",
"tokio",
"tracing",
"which",
"which 8.0.0",
"wiremock",
]
@@ -2432,7 +2447,7 @@ dependencies = [
"tracing",
"urlencoding",
"webbrowser",
"which",
"which 8.0.0",
]
[[package]]
@@ -2472,7 +2487,7 @@ dependencies = [
"tree-sitter",
"tree-sitter-bash",
"url",
"which",
"which 8.0.0",
]
[[package]]
@@ -2644,7 +2659,7 @@ dependencies = [
"uuid",
"vt100",
"webbrowser",
"which",
"which 8.0.0",
"windows-sys 0.52.0",
"winsplit",
]
@@ -2736,7 +2751,7 @@ dependencies = [
"uuid",
"vt100",
"webbrowser",
"which",
"which 8.0.0",
"windows-sys 0.52.0",
"winsplit",
]
@@ -2907,6 +2922,14 @@ dependencies = [
"regex-lite",
]
[[package]]
name = "codex-v8-poc"
version = "0.0.0"
dependencies = [
"pretty_assertions",
"v8",
]
[[package]]
name = "codex-windows-sandbox"
version = "0.0.0"
@@ -3112,6 +3135,15 @@ version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "773648b94d0e5d620f64f280777445740e61fe701025087ec8b57f45c791888b"
[[package]]
name = "core_maths"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77745e017f5edba1a9c1d854f6f3a52dac8a12dd5af5d2f54aecf61e43d80d30"
dependencies = [
"libm",
]
[[package]]
name = "core_test_support"
version = "0.0.0"
@@ -3717,6 +3749,38 @@ dependencies = [
"subtle",
]
[[package]]
name = "diplomat"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9adb46b05e2f53dcf6a7dfc242e4ce9eb60c369b6b6eb10826a01e93167f59c6"
dependencies = [
"diplomat_core",
"proc-macro2",
"quote",
"syn 2.0.114",
]
[[package]]
name = "diplomat-runtime"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0569bd3caaf13829da7ee4e83dbf9197a0e1ecd72772da6d08f0b4c9285c8d29"
[[package]]
name = "diplomat_core"
version = "0.14.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "51731530ed7f2d4495019abc7df3744f53338e69e2863a6a64ae91821c763df1"
dependencies = [
"proc-macro2",
"quote",
"serde",
"smallvec",
"strck",
"syn 2.0.114",
]
[[package]]
name = "dirs"
version = "6.0.0"
@@ -4323,6 +4387,16 @@ dependencies = [
"libc",
]
[[package]]
name = "fslock"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "04412b8935272e3a9bae6f48c7bfff74c2911f60525404edfdd28e49884c3bfb"
dependencies = [
"libc",
"winapi",
]
[[package]]
name = "futures"
version = "0.3.31"
@@ -4551,6 +4625,15 @@ dependencies = [
"regex-syntax 0.8.8",
]
[[package]]
name = "gzip-header"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "95cc527b92e6029a62960ad99aa8a6660faa4555fe5f731aab13aa6a921795a2"
dependencies = [
"crc32fast",
]
[[package]]
name = "h2"
version = "0.4.13"
@@ -5008,6 +5091,28 @@ dependencies = [
"cc",
]
[[package]]
name = "icu_calendar"
version = "2.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f0e52e009b6b16ba9c0693578796f2dd4aaa59a7f8f920423706714a89ac4e"
dependencies = [
"calendrical_calculations",
"displaydoc",
"icu_calendar_data",
"icu_locale",
"icu_locale_core",
"icu_provider",
"tinystr",
"zerovec",
]
[[package]]
name = "icu_calendar_data"
version = "2.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "527f04223b17edfe0bd43baf14a0cb1b017830db65f3950dc00224860a9a446d"
[[package]]
name = "icu_collections"
version = "2.1.1"
@@ -5442,6 +5547,12 @@ version = "1.0.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "92ecc6618181def0457392ccd0ee51198e065e016d1d527a7ac1b6dc7c1f09d2"
[[package]]
name = "ixdtf"
version = "0.6.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "84de9d95a6d2547d9b77ee3f25fa0ee32e3c3a6484d47a55adebc0439c077992"
[[package]]
name = "jiff"
version = "0.2.18"
@@ -7167,6 +7278,16 @@ dependencies = [
"yansi",
]
[[package]]
name = "prettyplease"
version = "0.2.37"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "479ca8adacdd7ce8f1fb39ce9ecccbfe93a3f1344b3d0d97f20bc0196208f62b"
dependencies = [
"proc-macro2",
"syn 2.0.114",
]
[[package]]
name = "proc-macro-crate"
version = "3.4.0"
@@ -8000,6 +8121,16 @@ dependencies = [
"webpki-roots 1.0.5",
]
[[package]]
name = "resb"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6a067ab3b5ca3b4dc307d0de9cf75f9f5e6ca9717b192b2f28a36c83e5de9e76"
dependencies = [
"potential_utf",
"serde_core",
]
[[package]]
name = "resolv-conf"
version = "0.7.6"
@@ -9380,6 +9511,15 @@ dependencies = [
"serde_json",
]
[[package]]
name = "strck"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42316e70da376f3d113a68d138a60d8a9883c604fe97942721ec2068dab13a9f"
dependencies = [
"unicode-ident",
]
[[package]]
name = "streaming-iterator"
version = "0.1.9"
@@ -9624,6 +9764,39 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "temporal_capi"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a151e402c2bdb6a3a2a2f3f225eddaead2e7ce7dd5d3fa2090deb11b17aa4ed8"
dependencies = [
"diplomat",
"diplomat-runtime",
"icu_calendar",
"icu_locale",
"num-traits",
"temporal_rs",
"timezone_provider",
"writeable",
"zoneinfo64",
]
[[package]]
name = "temporal_rs"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "88afde3bd75d2fc68d77a914bece426aa08aa7649ffd0cdd4a11c3d4d33474d1"
dependencies = [
"core_maths",
"icu_calendar",
"icu_locale",
"ixdtf",
"num-traits",
"timezone_provider",
"tinystr",
"writeable",
]
[[package]]
name = "term"
version = "0.7.0"
@@ -9831,6 +10004,18 @@ dependencies = [
"time-core",
]
[[package]]
name = "timezone_provider"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df9ba0000e9e73862f3e7ca1ff159e2ddf915c9d8bb11e38a7874760f445d993"
dependencies = [
"tinystr",
"zerotrie",
"zerovec",
"zoneinfo64",
]
[[package]]
name = "tiny-keccak"
version = "2.0.2"
@@ -10630,6 +10815,23 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "v8"
version = "146.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d97bcac5cdc5a195a4813f1855a6bc658f240452aac36caa12fd6c6f16026ab1"
dependencies = [
"bindgen",
"bitflags 2.10.0",
"fslock",
"gzip-header",
"home",
"miniz_oxide",
"paste",
"temporal_capi",
"which 6.0.3",
]
[[package]]
name = "valuable"
version = "0.1.1"
@@ -10929,6 +11131,18 @@ version = "0.1.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a28ac98ddc8b9274cb41bb4d9d4d5c425b6020c50c46f25559911905610b4a88"
[[package]]
name = "which"
version = "6.0.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b4ee928febd44d98f2f459a4a79bd4d928591333a494a10a868418ac1b39cf1f"
dependencies = [
"either",
"home",
"rustix 0.38.44",
"winsafe",
]
[[package]]
name = "which"
version = "8.0.0"
@@ -11932,6 +12146,19 @@ version = "1.0.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3ff05f8caa9038894637571ae6b9e29466c1f4f829d26c9b28f869a29cbe3445"
[[package]]
name = "zoneinfo64"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bb2e5597efbe7c421da8a7fd396b20b571704e787c21a272eecf35dfe9d386f0"
dependencies = [
"calendrical_calculations",
"icu_locale_core",
"potential_utf",
"resb",
"serde",
]
[[package]]
name = "zopfli"
version = "0.8.3"

View File

@@ -45,6 +45,7 @@ members = [
"otel",
"tui",
"tui_app_server",
"v8-poc",
"utils/absolute-path",
"utils/cargo-bin",
"utils/git",
@@ -137,6 +138,7 @@ codex-test-macros = { path = "test-macros" }
codex-terminal-detection = { path = "terminal-detection" }
codex-tui = { path = "tui" }
codex-tui-app-server = { path = "tui_app_server" }
codex-v8-poc = { path = "v8-poc" }
codex-utils-absolute-path = { path = "utils/absolute-path" }
codex-utils-approval-presets = { path = "utils/approval-presets" }
codex-utils-cache = { path = "utils/cache" }
@@ -245,6 +247,7 @@ regex-lite = "0.1.8"
reqwest = "0.12"
rmcp = { version = "0.15.0", default-features = false }
runfiles = { git = "https://github.com/dzbarsky/rules_rust", rev = "b56cbaa8465e74127f1ea216f813cd377295ad81" }
v8 = "=146.4.0"
rustls = { version = "0.23", default-features = false, features = [
"ring",
"std",
@@ -370,7 +373,8 @@ ignored = [
"icu_provider",
"openssl-sys",
"codex-utils-readiness",
"codex-secrets"
"codex-secrets",
"codex-v8-poc"
]
[profile.release]

View File

@@ -9,8 +9,13 @@ workspace = true
[dependencies]
keyring = { workspace = true, features = ["crypto-rust"] }
serde = { workspace = true, features = ["derive"] }
serde_json = { workspace = true }
tracing = { workspace = true }
[dev-dependencies]
pretty_assertions = { workspace = true }
[target.'cfg(target_os = "linux")'.dependencies]
keyring = { workspace = true, features = ["linux-native-async-persistent"] }

View File

@@ -0,0 +1,192 @@
use crate::CredentialStoreError;
use crate::KeyringStore;
use serde_json::Value;
use std::fmt;
#[derive(Debug, Clone)]
pub struct FullJsonKeyringError {
message: String,
}
pub type JsonKeyringError = FullJsonKeyringError;
impl FullJsonKeyringError {
fn new(message: impl Into<String>) -> Self {
Self {
message: message.into(),
}
}
}
impl fmt::Display for FullJsonKeyringError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.message)
}
}
impl std::error::Error for FullJsonKeyringError {}
pub fn load_json_from_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
) -> Result<Option<Value>, JsonKeyringError> {
if let Some(bytes) = load_secret_from_keyring(keyring_store, service, base_key, "JSON record")?
{
let value = serde_json::from_slice(&bytes).map_err(|err| {
FullJsonKeyringError::new(format!(
"failed to deserialize JSON record from keyring secret: {err}"
))
})?;
return Ok(Some(value));
}
match keyring_store.load(service, base_key) {
Ok(Some(serialized)) => serde_json::from_str(&serialized).map(Some).map_err(|err| {
FullJsonKeyringError::new(format!(
"failed to deserialize JSON record from keyring password: {err}"
))
}),
Ok(None) => Ok(None),
Err(error) => Err(credential_store_error("load", "JSON record", error)),
}
}
pub fn save_json_to_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
value: &Value,
) -> Result<(), JsonKeyringError> {
let bytes = serde_json::to_vec(value).map_err(|err| {
FullJsonKeyringError::new(format!("failed to serialize JSON record: {err}"))
})?;
save_secret_to_keyring(keyring_store, service, base_key, &bytes, "JSON record")
}
pub fn delete_json_from_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
) -> Result<bool, JsonKeyringError> {
delete_keyring_entry(keyring_store, service, base_key, "JSON record")
}
fn load_secret_from_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
key: &str,
field: &str,
) -> Result<Option<Vec<u8>>, FullJsonKeyringError> {
keyring_store
.load_secret(service, key)
.map_err(|err| credential_store_error("load", field, err))
}
fn save_secret_to_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
key: &str,
value: &[u8],
field: &str,
) -> Result<(), FullJsonKeyringError> {
keyring_store
.save_secret(service, key, value)
.map_err(|err| credential_store_error("write", field, err))
}
fn delete_keyring_entry<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
key: &str,
field: &str,
) -> Result<bool, FullJsonKeyringError> {
keyring_store
.delete(service, key)
.map_err(|err| credential_store_error("delete", field, err))
}
fn credential_store_error(
action: &str,
field: &str,
error: CredentialStoreError,
) -> FullJsonKeyringError {
FullJsonKeyringError::new(format!(
"failed to {action} {field} in keyring: {}",
error.message()
))
}
#[cfg(test)]
mod tests {
use super::delete_json_from_keyring;
use super::load_json_from_keyring;
use super::save_json_to_keyring;
use crate::KeyringStore;
use crate::tests::MockKeyringStore;
use pretty_assertions::assert_eq;
use serde_json::json;
const SERVICE: &str = "Test Service";
const BASE_KEY: &str = "base";
#[test]
fn json_storage_round_trips_using_full_backend() {
let store = MockKeyringStore::default();
let expected = json!({
"token": "secret",
"nested": {"id": 7}
});
save_json_to_keyring(&store, SERVICE, BASE_KEY, &expected).expect("JSON should save");
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON should load")
.expect("JSON should exist");
assert_eq!(loaded, expected);
assert_eq!(
store.saved_secret(BASE_KEY),
Some(serde_json::to_vec(&expected).expect("JSON should serialize")),
);
}
#[test]
fn json_storage_loads_legacy_single_entry() {
let store = MockKeyringStore::default();
let expected = json!({
"token": "secret",
"nested": {"id": 9}
});
store
.save(
SERVICE,
BASE_KEY,
&serde_json::to_string(&expected).expect("JSON should serialize"),
)
.expect("legacy JSON should save");
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON should load")
.expect("JSON should exist");
assert_eq!(loaded, expected);
}
#[test]
fn json_storage_delete_removes_full_entry() {
let store = MockKeyringStore::default();
let expected = json!({"current": true});
save_json_to_keyring(&store, SERVICE, BASE_KEY, &expected).expect("JSON should save");
let removed = delete_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON delete should succeed");
assert!(removed);
assert!(
load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON load should succeed")
.is_none()
);
assert!(!store.contains(BASE_KEY));
}
}

View File

@@ -0,0 +1,736 @@
use crate::CredentialStoreError;
use crate::KeyringStore;
use serde::Deserialize;
use serde::Serialize;
use serde_json::Map;
use serde_json::Value;
use std::fmt;
use std::fmt::Write as _;
use tracing::warn;
const LAYOUT_VERSION: &str = "v1";
const MANIFEST_ENTRY: &str = "manifest";
const VALUE_ENTRY_PREFIX: &str = "value";
const ROOT_PATH_SENTINEL: &str = "root";
#[derive(Debug, Clone)]
pub struct SplitJsonKeyringError {
message: String,
}
pub type JsonKeyringError = SplitJsonKeyringError;
impl SplitJsonKeyringError {
fn new(message: impl Into<String>) -> Self {
Self {
message: message.into(),
}
}
}
impl fmt::Display for SplitJsonKeyringError {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "{}", self.message)
}
}
impl std::error::Error for SplitJsonKeyringError {}
#[derive(Clone, Copy, Debug, Deserialize, Serialize, PartialEq, Eq)]
#[serde(rename_all = "lowercase")]
enum JsonNodeKind {
Null,
Bool,
Number,
String,
Object,
Array,
}
impl JsonNodeKind {
fn from_value(value: &Value) -> Self {
match value {
Value::Null => Self::Null,
Value::Bool(_) => Self::Bool,
Value::Number(_) => Self::Number,
Value::String(_) => Self::String,
Value::Object(_) => Self::Object,
Value::Array(_) => Self::Array,
}
}
fn is_container(self) -> bool {
matches!(self, Self::Object | Self::Array)
}
fn empty_value(self) -> Option<Value> {
match self {
Self::Object => Some(Value::Object(Map::new())),
Self::Array => Some(Value::Array(Vec::new())),
Self::Null | Self::Bool | Self::Number | Self::String => None,
}
}
}
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq, Eq)]
struct SplitJsonNode {
path: String,
kind: JsonNodeKind,
}
#[derive(Clone, Debug, Deserialize, Serialize, PartialEq, Eq)]
struct SplitJsonManifest {
nodes: Vec<SplitJsonNode>,
}
type SplitJsonLeafValues = Vec<(String, Vec<u8>)>;
pub fn load_json_from_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
) -> Result<Option<Value>, JsonKeyringError> {
let Some(manifest) = load_manifest(keyring_store, service, base_key)? else {
return Ok(None);
};
inflate_split_json(keyring_store, service, base_key, &manifest).map(Some)
}
pub fn save_json_to_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
value: &Value,
) -> Result<(), JsonKeyringError> {
let previous_manifest = match load_manifest(keyring_store, service, base_key) {
Ok(manifest) => manifest,
Err(err) => {
warn!("failed to read previous split JSON manifest from keyring: {err}");
None
}
};
let (manifest, leaf_values) = flatten_split_json(value)?;
let current_scalar_paths = manifest
.nodes
.iter()
.filter(|node| !node.kind.is_container())
.map(|node| node.path.as_str())
.collect::<std::collections::HashSet<_>>();
for (path, bytes) in leaf_values {
let key = value_key(base_key, &path);
save_secret_to_keyring(
keyring_store,
service,
&key,
&bytes,
&format!("JSON value at {path}"),
)?;
}
let manifest_key = layout_key(base_key, MANIFEST_ENTRY);
let manifest_bytes = serde_json::to_vec(&manifest).map_err(|err| {
SplitJsonKeyringError::new(format!("failed to serialize JSON manifest: {err}"))
})?;
save_secret_to_keyring(
keyring_store,
service,
&manifest_key,
&manifest_bytes,
"JSON manifest",
)?;
if let Some(previous_manifest) = previous_manifest {
for node in previous_manifest.nodes {
if node.kind.is_container() || current_scalar_paths.contains(node.path.as_str()) {
continue;
}
let key = value_key(base_key, &node.path);
if let Err(err) = delete_keyring_entry(
keyring_store,
service,
&key,
&format!("stale JSON value at {}", node.path),
) {
warn!("failed to remove stale split JSON value from keyring: {err}");
}
}
}
Ok(())
}
pub fn delete_json_from_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
) -> Result<bool, JsonKeyringError> {
let Some(manifest) = load_manifest(keyring_store, service, base_key)? else {
return Ok(false);
};
let mut removed = false;
for node in manifest.nodes {
if node.kind.is_container() {
continue;
}
let key = value_key(base_key, &node.path);
removed |= delete_keyring_entry(
keyring_store,
service,
&key,
&format!("JSON value at {}", node.path),
)?;
}
let manifest_key = layout_key(base_key, MANIFEST_ENTRY);
removed |= delete_keyring_entry(keyring_store, service, &manifest_key, "JSON manifest")?;
Ok(removed)
}
fn flatten_split_json(
value: &Value,
) -> Result<(SplitJsonManifest, SplitJsonLeafValues), SplitJsonKeyringError> {
let mut nodes = Vec::new();
let mut leaf_values = Vec::new();
collect_nodes("", value, &mut nodes, &mut leaf_values)?;
nodes.sort_by(|left, right| {
path_depth(&left.path)
.cmp(&path_depth(&right.path))
.then_with(|| left.path.cmp(&right.path))
});
leaf_values.sort_by(|left, right| left.0.cmp(&right.0));
Ok((SplitJsonManifest { nodes }, leaf_values))
}
fn collect_nodes(
path: &str,
value: &Value,
nodes: &mut Vec<SplitJsonNode>,
leaf_values: &mut SplitJsonLeafValues,
) -> Result<(), SplitJsonKeyringError> {
let kind = JsonNodeKind::from_value(value);
nodes.push(SplitJsonNode {
path: path.to_string(),
kind,
});
match value {
Value::Object(map) => {
let mut keys = map.keys().cloned().collect::<Vec<_>>();
keys.sort();
for key in keys {
let child_path = append_json_pointer_token(path, &key);
let child_value = map.get(&key).ok_or_else(|| {
SplitJsonKeyringError::new(format!(
"missing object value for path {child_path}"
))
})?;
collect_nodes(&child_path, child_value, nodes, leaf_values)?;
}
}
Value::Array(items) => {
for (index, item) in items.iter().enumerate() {
let child_path = append_json_pointer_token(path, &index.to_string());
collect_nodes(&child_path, item, nodes, leaf_values)?;
}
}
Value::Null | Value::Bool(_) | Value::Number(_) | Value::String(_) => {
let bytes = serde_json::to_vec(value).map_err(|err| {
SplitJsonKeyringError::new(format!(
"failed to serialize JSON value at {path}: {err}"
))
})?;
leaf_values.push((path.to_string(), bytes));
}
}
Ok(())
}
fn inflate_split_json<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
manifest: &SplitJsonManifest,
) -> Result<Value, SplitJsonKeyringError> {
let root_node = manifest
.nodes
.iter()
.find(|node| node.path.is_empty())
.ok_or_else(|| SplitJsonKeyringError::new("missing root JSON node in keyring manifest"))?;
let mut result = if let Some(value) = root_node.kind.empty_value() {
value
} else {
load_value(keyring_store, service, base_key, "")?
};
let mut nodes = manifest.nodes.clone();
nodes.sort_by(|left, right| {
path_depth(&left.path)
.cmp(&path_depth(&right.path))
.then_with(|| left.path.cmp(&right.path))
});
for node in nodes.into_iter().filter(|node| !node.path.is_empty()) {
let value = if let Some(value) = node.kind.empty_value() {
value
} else {
load_value(keyring_store, service, base_key, &node.path)?
};
insert_value_at_pointer(&mut result, &node.path, value)?;
}
Ok(result)
}
fn load_value<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
path: &str,
) -> Result<Value, SplitJsonKeyringError> {
let key = value_key(base_key, path);
let bytes = load_secret_from_keyring(
keyring_store,
service,
&key,
&format!("JSON value at {path}"),
)?
.ok_or_else(|| {
SplitJsonKeyringError::new(format!("missing JSON value at {path} in keyring"))
})?;
serde_json::from_slice(&bytes).map_err(|err| {
SplitJsonKeyringError::new(format!("failed to deserialize JSON value at {path}: {err}"))
})
}
fn insert_value_at_pointer(
root: &mut Value,
pointer: &str,
value: Value,
) -> Result<(), SplitJsonKeyringError> {
if pointer.is_empty() {
*root = value;
return Ok(());
}
let tokens = decode_json_pointer(pointer)?;
let Some((last, parents)) = tokens.split_last() else {
return Err(SplitJsonKeyringError::new(
"missing JSON pointer path tokens",
));
};
let mut current = root;
for token in parents {
current = match current {
Value::Object(map) => map.get_mut(token).ok_or_else(|| {
SplitJsonKeyringError::new(format!(
"missing parent object entry for JSON pointer {pointer}"
))
})?,
Value::Array(items) => {
let index = parse_array_index(token, pointer)?;
items.get_mut(index).ok_or_else(|| {
SplitJsonKeyringError::new(format!(
"missing parent array entry for JSON pointer {pointer}"
))
})?
}
Value::Null | Value::Bool(_) | Value::Number(_) | Value::String(_) => {
return Err(SplitJsonKeyringError::new(format!(
"encountered scalar while walking JSON pointer {pointer}"
)));
}
};
}
match current {
Value::Object(map) => {
map.insert(last.to_string(), value);
Ok(())
}
Value::Array(items) => {
let index = parse_array_index(last, pointer)?;
if index >= items.len() {
items.resize(index + 1, Value::Null);
}
items[index] = value;
Ok(())
}
Value::Null | Value::Bool(_) | Value::Number(_) | Value::String(_) => {
Err(SplitJsonKeyringError::new(format!(
"encountered scalar while assigning JSON pointer {pointer}"
)))
}
}
}
fn load_manifest<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
base_key: &str,
) -> Result<Option<SplitJsonManifest>, SplitJsonKeyringError> {
let manifest_key = layout_key(base_key, MANIFEST_ENTRY);
let Some(bytes) =
load_secret_from_keyring(keyring_store, service, &manifest_key, "JSON manifest")?
else {
return Ok(None);
};
let manifest: SplitJsonManifest = serde_json::from_slice(&bytes).map_err(|err| {
SplitJsonKeyringError::new(format!("failed to deserialize JSON manifest: {err}"))
})?;
if manifest.nodes.is_empty() {
return Err(SplitJsonKeyringError::new("JSON manifest is empty"));
}
Ok(Some(manifest))
}
fn load_secret_from_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
key: &str,
field: &str,
) -> Result<Option<Vec<u8>>, SplitJsonKeyringError> {
keyring_store
.load_secret(service, key)
.map_err(|err| credential_store_error("load", field, err))
}
fn save_secret_to_keyring<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
key: &str,
value: &[u8],
field: &str,
) -> Result<(), SplitJsonKeyringError> {
keyring_store
.save_secret(service, key, value)
.map_err(|err| credential_store_error("write", field, err))
}
fn delete_keyring_entry<K: KeyringStore + ?Sized>(
keyring_store: &K,
service: &str,
key: &str,
field: &str,
) -> Result<bool, SplitJsonKeyringError> {
keyring_store
.delete(service, key)
.map_err(|err| credential_store_error("delete", field, err))
}
fn credential_store_error(
action: &str,
field: &str,
error: CredentialStoreError,
) -> SplitJsonKeyringError {
SplitJsonKeyringError::new(format!(
"failed to {action} {field} in keyring: {}",
error.message()
))
}
fn layout_key(base_key: &str, suffix: &str) -> String {
format!("{base_key}|{LAYOUT_VERSION}|{suffix}")
}
fn value_key(base_key: &str, path: &str) -> String {
let encoded_path = encode_path(path);
layout_key(base_key, &format!("{VALUE_ENTRY_PREFIX}|{encoded_path}"))
}
fn encode_path(path: &str) -> String {
if path.is_empty() {
return ROOT_PATH_SENTINEL.to_string();
}
let mut encoded = String::with_capacity(path.len() * 2);
for byte in path.as_bytes() {
let _ = write!(&mut encoded, "{byte:02x}");
}
encoded
}
fn append_json_pointer_token(path: &str, token: &str) -> String {
let escaped = token.replace('~', "~0").replace('/', "~1");
if path.is_empty() {
format!("/{escaped}")
} else {
format!("{path}/{escaped}")
}
}
fn decode_json_pointer(pointer: &str) -> Result<Vec<String>, SplitJsonKeyringError> {
if pointer.is_empty() {
return Ok(Vec::new());
}
if !pointer.starts_with('/') {
return Err(SplitJsonKeyringError::new(format!(
"invalid JSON pointer {pointer}: expected leading slash"
)));
}
pointer[1..]
.split('/')
.map(unescape_json_pointer_token)
.collect()
}
fn unescape_json_pointer_token(token: &str) -> Result<String, SplitJsonKeyringError> {
let mut result = String::with_capacity(token.len());
let mut chars = token.chars();
while let Some(ch) = chars.next() {
if ch != '~' {
result.push(ch);
continue;
}
match chars.next() {
Some('0') => result.push('~'),
Some('1') => result.push('/'),
Some(other) => {
return Err(SplitJsonKeyringError::new(format!(
"invalid JSON pointer escape sequence ~{other}"
)));
}
None => {
return Err(SplitJsonKeyringError::new(
"invalid JSON pointer escape sequence at end of token",
));
}
}
}
Ok(result)
}
fn parse_array_index(token: &str, pointer: &str) -> Result<usize, SplitJsonKeyringError> {
token.parse::<usize>().map_err(|err| {
SplitJsonKeyringError::new(format!(
"invalid array index '{token}' in JSON pointer {pointer}: {err}"
))
})
}
fn path_depth(path: &str) -> usize {
path.chars().filter(|ch| *ch == '/').count()
}
#[cfg(test)]
mod tests {
use super::LAYOUT_VERSION;
use super::MANIFEST_ENTRY;
use super::delete_json_from_keyring;
use super::layout_key;
use super::load_json_from_keyring;
use super::save_json_to_keyring;
use super::value_key;
use crate::KeyringStore;
use crate::tests::MockKeyringStore;
use pretty_assertions::assert_eq;
use serde_json::json;
const SERVICE: &str = "Test Service";
const BASE_KEY: &str = "base";
#[test]
fn json_storage_round_trips_using_split_backend() {
let store = MockKeyringStore::default();
let expected = json!({
"token": "secret",
"nested": {"id": 7}
});
save_json_to_keyring(&store, SERVICE, BASE_KEY, &expected).expect("JSON should save");
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON should load")
.expect("JSON should exist");
assert_eq!(loaded, expected);
assert!(
store.saved_secret(BASE_KEY).is_none(),
"split storage should not write the full record under the base key"
);
assert!(
store.contains(&layout_key(BASE_KEY, MANIFEST_ENTRY)),
"split storage should write manifest metadata"
);
}
#[test]
fn json_storage_does_not_load_legacy_single_entry() {
let store = MockKeyringStore::default();
let expected = json!({
"token": "secret",
"nested": {"id": 9}
});
store
.save(
SERVICE,
BASE_KEY,
&serde_json::to_string(&expected).expect("JSON should serialize"),
)
.expect("legacy JSON should save");
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY).expect("JSON should load");
assert_eq!(loaded, None);
}
#[test]
fn json_storage_save_preserves_legacy_single_entry() {
let store = MockKeyringStore::default();
let current = json!({"current": true});
let legacy = json!({"legacy": true});
store
.save(
SERVICE,
BASE_KEY,
&serde_json::to_string(&legacy).expect("JSON should serialize"),
)
.expect("legacy JSON should save");
save_json_to_keyring(&store, SERVICE, BASE_KEY, &current).expect("JSON should save");
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON should load")
.expect("JSON should exist");
assert_eq!(loaded, current);
assert_eq!(
store.saved_value(BASE_KEY),
Some(serde_json::to_string(&legacy).expect("JSON should serialize"))
);
}
#[test]
fn json_storage_delete_removes_only_split_entries() {
let store = MockKeyringStore::default();
let current = json!({"current": true});
let legacy = json!({"legacy": true});
store
.save(
SERVICE,
BASE_KEY,
&serde_json::to_string(&legacy).expect("JSON should serialize"),
)
.expect("legacy JSON should save");
save_json_to_keyring(&store, SERVICE, BASE_KEY, &current).expect("JSON should save");
let removed = delete_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON delete should succeed");
assert!(removed);
assert!(
load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("JSON load should succeed")
.is_none()
);
assert!(store.contains(BASE_KEY));
assert!(!store.contains(&layout_key(BASE_KEY, MANIFEST_ENTRY)));
}
#[test]
fn split_json_round_trips_nested_values() {
let store = MockKeyringStore::default();
let expected = json!({
"name": "codex",
"enabled": true,
"count": 3,
"nested": {
"items": [null, {"hello": "world"}],
"slash/key": "~value~",
},
});
save_json_to_keyring(&store, SERVICE, BASE_KEY, &expected).expect("split JSON should save");
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("split JSON should load")
.expect("split JSON should exist");
assert_eq!(loaded, expected);
}
#[test]
fn split_json_supports_scalar_root_values() {
let store = MockKeyringStore::default();
let expected = json!("value");
save_json_to_keyring(&store, SERVICE, BASE_KEY, &expected).expect("split JSON should save");
let root_value_key = value_key(BASE_KEY, "");
assert_eq!(
store.saved_secret_utf8(&root_value_key),
Some("\"value\"".to_string())
);
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("split JSON should load")
.expect("split JSON should exist");
assert_eq!(loaded, expected);
}
#[test]
fn split_json_delete_removes_saved_entries() {
let store = MockKeyringStore::default();
let expected = json!({
"token": "secret",
"nested": {
"id": 123,
},
});
save_json_to_keyring(&store, SERVICE, BASE_KEY, &expected).expect("split JSON should save");
let manifest_key = layout_key(BASE_KEY, MANIFEST_ENTRY);
let token_key = value_key(BASE_KEY, "/token");
let nested_id_key = value_key(BASE_KEY, "/nested/id");
let removed = delete_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("split JSON delete should succeed");
assert!(removed);
assert!(!store.contains(&manifest_key));
assert!(!store.contains(&token_key));
assert!(!store.contains(&nested_id_key));
}
#[test]
fn split_json_save_replaces_previous_values() {
let store = MockKeyringStore::default();
let first = json!({"value": "first", "stale": true});
let second = json!({"value": "second", "extra": 1});
save_json_to_keyring(&store, SERVICE, BASE_KEY, &first)
.expect("first split JSON save should succeed");
let manifest_key = layout_key(BASE_KEY, MANIFEST_ENTRY);
let stale_value_key = value_key(BASE_KEY, "/stale");
assert!(store.contains(&manifest_key));
assert!(store.contains(&stale_value_key));
save_json_to_keyring(&store, SERVICE, BASE_KEY, &second)
.expect("second split JSON save should succeed");
assert!(!store.contains(&stale_value_key));
assert!(store.contains(&manifest_key));
assert_eq!(
store.saved_secret_utf8(&value_key(BASE_KEY, "/value")),
Some("\"second\"".to_string())
);
assert_eq!(
store.saved_secret_utf8(&value_key(BASE_KEY, "/extra")),
Some("1".to_string())
);
let loaded = load_json_from_keyring(&store, SERVICE, BASE_KEY)
.expect("split JSON should load")
.expect("split JSON should exist");
assert_eq!(loaded, second);
}
#[test]
fn split_json_uses_distinct_layout_version() {
assert_eq!(LAYOUT_VERSION, "v1");
}
}

View File

@@ -5,6 +5,19 @@ use std::fmt;
use std::fmt::Debug;
use tracing::trace;
#[cfg(not(windows))]
#[path = "json_store_full.rs"]
mod json_store;
#[cfg(windows)]
#[path = "json_store_split.rs"]
mod json_store;
pub use json_store::JsonKeyringError;
pub use json_store::delete_json_from_keyring;
pub use json_store::load_json_from_keyring;
pub use json_store::save_json_to_keyring;
#[derive(Debug)]
pub enum CredentialStoreError {
Other(KeyringError),
@@ -41,7 +54,18 @@ impl Error for CredentialStoreError {}
/// Shared credential store abstraction for keyring-backed implementations.
pub trait KeyringStore: Debug + Send + Sync {
fn load(&self, service: &str, account: &str) -> Result<Option<String>, CredentialStoreError>;
fn load_secret(
&self,
service: &str,
account: &str,
) -> Result<Option<Vec<u8>>, CredentialStoreError>;
fn save(&self, service: &str, account: &str, value: &str) -> Result<(), CredentialStoreError>;
fn save_secret(
&self,
service: &str,
account: &str,
value: &[u8],
) -> Result<(), CredentialStoreError>;
fn delete(&self, service: &str, account: &str) -> Result<bool, CredentialStoreError>;
}
@@ -68,6 +92,31 @@ impl KeyringStore for DefaultKeyringStore {
}
}
fn load_secret(
&self,
service: &str,
account: &str,
) -> Result<Option<Vec<u8>>, CredentialStoreError> {
trace!("keyring.load_secret start, service={service}, account={account}");
let entry = Entry::new(service, account).map_err(CredentialStoreError::new)?;
match entry.get_secret() {
Ok(secret) => {
trace!("keyring.load_secret success, service={service}, account={account}");
Ok(Some(secret))
}
Err(keyring::Error::NoEntry) => {
trace!("keyring.load_secret no entry, service={service}, account={account}");
Ok(None)
}
Err(error) => {
trace!(
"keyring.load_secret error, service={service}, account={account}, error={error}"
);
Err(CredentialStoreError::new(error))
}
}
}
fn save(&self, service: &str, account: &str, value: &str) -> Result<(), CredentialStoreError> {
trace!(
"keyring.save start, service={service}, account={account}, value_len={}",
@@ -86,6 +135,31 @@ impl KeyringStore for DefaultKeyringStore {
}
}
fn save_secret(
&self,
service: &str,
account: &str,
value: &[u8],
) -> Result<(), CredentialStoreError> {
trace!(
"keyring.save_secret start, service={service}, account={account}, value_len={}",
value.len()
);
let entry = Entry::new(service, account).map_err(CredentialStoreError::new)?;
match entry.set_secret(value) {
Ok(()) => {
trace!("keyring.save_secret success, service={service}, account={account}");
Ok(())
}
Err(error) => {
trace!(
"keyring.save_secret error, service={service}, account={account}, error={error}"
);
Err(CredentialStoreError::new(error))
}
}
}
fn delete(&self, service: &str, account: &str) -> Result<bool, CredentialStoreError> {
trace!("keyring.delete start, service={service}, account={account}");
let entry = Entry::new(service, account).map_err(CredentialStoreError::new)?;
@@ -145,6 +219,22 @@ pub mod tests {
credential.get_password().ok()
}
pub fn saved_secret(&self, account: &str) -> Option<Vec<u8>> {
let credential = {
let guard = self
.credentials
.lock()
.unwrap_or_else(PoisonError::into_inner);
guard.get(account).cloned()
}?;
credential.get_secret().ok()
}
pub fn saved_secret_utf8(&self, account: &str) -> Option<String> {
let secret = self.saved_secret(account)?;
String::from_utf8(secret).ok()
}
pub fn set_error(&self, account: &str, error: KeyringError) {
let credential = self.credential(account);
credential.set_error(error);
@@ -184,6 +274,30 @@ pub mod tests {
}
}
fn load_secret(
&self,
_service: &str,
account: &str,
) -> Result<Option<Vec<u8>>, CredentialStoreError> {
let credential = {
let guard = self
.credentials
.lock()
.unwrap_or_else(PoisonError::into_inner);
guard.get(account).cloned()
};
let Some(credential) = credential else {
return Ok(None);
};
match credential.get_secret() {
Ok(secret) => Ok(Some(secret)),
Err(KeyringError::NoEntry) => Ok(None),
Err(error) => Err(CredentialStoreError::new(error)),
}
}
fn save(
&self,
_service: &str,
@@ -196,6 +310,18 @@ pub mod tests {
.map_err(CredentialStoreError::new)
}
fn save_secret(
&self,
_service: &str,
account: &str,
value: &[u8],
) -> Result<(), CredentialStoreError> {
let credential = self.credential(account);
credential
.set_secret(value)
.map_err(CredentialStoreError::new)
}
fn delete(&self, _service: &str, account: &str) -> Result<bool, CredentialStoreError> {
let credential = {
let guard = self

View File

@@ -23,6 +23,9 @@ use crate::token_data::TokenData;
use codex_app_server_protocol::AuthMode;
use codex_keyring_store::DefaultKeyringStore;
use codex_keyring_store::KeyringStore;
use codex_keyring_store::delete_json_from_keyring;
use codex_keyring_store::load_json_from_keyring;
use codex_keyring_store::save_json_to_keyring;
use once_cell::sync::Lazy;
/// Determine where Codex should store CLI auth credentials.
@@ -162,47 +165,39 @@ impl KeyringAuthStorage {
}
}
fn load_from_keyring(&self, key: &str) -> std::io::Result<Option<AuthDotJson>> {
match self.keyring_store.load(KEYRING_SERVICE, key) {
Ok(Some(serialized)) => serde_json::from_str(&serialized).map(Some).map_err(|err| {
std::io::Error::other(format!(
"failed to deserialize CLI auth from keyring: {err}"
))
}),
Ok(None) => Ok(None),
Err(error) => Err(std::io::Error::other(format!(
"failed to load CLI auth from keyring: {}",
error.message()
))),
}
}
fn save_to_keyring(&self, key: &str, value: &str) -> std::io::Result<()> {
match self.keyring_store.save(KEYRING_SERVICE, key, value) {
Ok(()) => Ok(()),
Err(error) => {
let message = format!(
"failed to write OAuth tokens to keyring: {}",
error.message()
);
warn!("{message}");
Err(std::io::Error::other(message))
}
}
fn load_auth_from_keyring(&self, base_key: &str) -> std::io::Result<Option<AuthDotJson>> {
let Some(value) =
load_json_from_keyring(self.keyring_store.as_ref(), KEYRING_SERVICE, base_key)
.map_err(|err| {
std::io::Error::other(format!("failed to load CLI auth from keyring: {err}"))
})?
else {
return Ok(None);
};
serde_json::from_value(value).map(Some).map_err(|err| {
std::io::Error::other(format!(
"failed to deserialize CLI auth from keyring: {err}"
))
})
}
}
impl AuthStorageBackend for KeyringAuthStorage {
fn load(&self) -> std::io::Result<Option<AuthDotJson>> {
let key = compute_store_key(&self.codex_home)?;
self.load_from_keyring(&key)
self.load_auth_from_keyring(&key)
}
fn save(&self, auth: &AuthDotJson) -> std::io::Result<()> {
let key = compute_store_key(&self.codex_home)?;
// Simpler error mapping per style: prefer method reference over closure
let serialized = serde_json::to_string(auth).map_err(std::io::Error::other)?;
self.save_to_keyring(&key, &serialized)?;
let base_key = compute_store_key(&self.codex_home)?;
let value = serde_json::to_value(auth).map_err(std::io::Error::other)?;
save_json_to_keyring(
self.keyring_store.as_ref(),
KEYRING_SERVICE,
&base_key,
&value,
)
.map_err(|err| std::io::Error::other(format!("failed to write auth to keyring: {err}")))?;
if let Err(err) = delete_file_if_exists(&self.codex_home) {
warn!("failed to remove CLI auth fallback file: {err}");
}
@@ -210,13 +205,12 @@ impl AuthStorageBackend for KeyringAuthStorage {
}
fn delete(&self) -> std::io::Result<bool> {
let key = compute_store_key(&self.codex_home)?;
let keyring_removed = self
.keyring_store
.delete(KEYRING_SERVICE, &key)
.map_err(|err| {
std::io::Error::other(format!("failed to delete auth from keyring: {err}"))
})?;
let base_key = compute_store_key(&self.codex_home)?;
let keyring_removed =
delete_json_from_keyring(self.keyring_store.as_ref(), KEYRING_SERVICE, &base_key)
.map_err(|err| {
std::io::Error::other(format!("failed to delete auth from keyring: {err}"))
})?;
let file_removed = delete_file_if_exists(&self.codex_home)?;
Ok(keyring_removed || file_removed)
}

View File

@@ -6,9 +6,88 @@ use pretty_assertions::assert_eq;
use serde_json::json;
use tempfile::tempdir;
use codex_keyring_store::CredentialStoreError;
use codex_keyring_store::tests::MockKeyringStore;
use keyring::Error as KeyringError;
#[derive(Clone, Debug)]
struct SaveSecretErrorKeyringStore {
inner: MockKeyringStore,
}
impl KeyringStore for SaveSecretErrorKeyringStore {
fn load(&self, service: &str, account: &str) -> Result<Option<String>, CredentialStoreError> {
self.inner.load(service, account)
}
fn load_secret(
&self,
service: &str,
account: &str,
) -> Result<Option<Vec<u8>>, CredentialStoreError> {
self.inner.load_secret(service, account)
}
fn save(&self, service: &str, account: &str, value: &str) -> Result<(), CredentialStoreError> {
self.inner.save(service, account, value)
}
fn save_secret(
&self,
_service: &str,
_account: &str,
_value: &[u8],
) -> Result<(), CredentialStoreError> {
Err(CredentialStoreError::new(KeyringError::Invalid(
"error".into(),
"save".into(),
)))
}
fn delete(&self, service: &str, account: &str) -> Result<bool, CredentialStoreError> {
self.inner.delete(service, account)
}
}
#[derive(Clone, Debug)]
struct LoadSecretErrorKeyringStore {
inner: MockKeyringStore,
}
impl KeyringStore for LoadSecretErrorKeyringStore {
fn load(&self, service: &str, account: &str) -> Result<Option<String>, CredentialStoreError> {
self.inner.load(service, account)
}
fn load_secret(
&self,
_service: &str,
_account: &str,
) -> Result<Option<Vec<u8>>, CredentialStoreError> {
Err(CredentialStoreError::new(KeyringError::Invalid(
"error".into(),
"load".into(),
)))
}
fn save(&self, service: &str, account: &str, value: &str) -> Result<(), CredentialStoreError> {
self.inner.save(service, account, value)
}
fn save_secret(
&self,
service: &str,
account: &str,
value: &[u8],
) -> Result<(), CredentialStoreError> {
self.inner.save_secret(service, account, value)
}
fn delete(&self, service: &str, account: &str) -> Result<bool, CredentialStoreError> {
self.inner.delete(service, account)
}
}
#[tokio::test]
async fn file_storage_load_returns_auth_dot_json() -> anyhow::Result<()> {
let codex_home = tempdir()?;
@@ -97,19 +176,16 @@ fn ephemeral_storage_save_load_delete_is_in_memory_only() -> anyhow::Result<()>
Ok(())
}
fn seed_keyring_and_fallback_auth_file_for_delete<F>(
mock_keyring: &MockKeyringStore,
fn seed_keyring_and_fallback_auth_file_for_delete(
storage: &KeyringAuthStorage,
codex_home: &Path,
compute_key: F,
) -> anyhow::Result<(String, PathBuf)>
where
F: FnOnce() -> std::io::Result<String>,
{
let key = compute_key()?;
mock_keyring.save(KEYRING_SERVICE, &key, "{}")?;
auth: &AuthDotJson,
) -> anyhow::Result<(String, PathBuf)> {
storage.save(auth)?;
let base_key = compute_store_key(codex_home)?;
let auth_file = get_auth_file(codex_home);
std::fs::write(&auth_file, "stale")?;
Ok((key, auth_file))
Ok((base_key, auth_file))
}
fn seed_keyring_with_auth<F>(
@@ -128,15 +204,26 @@ where
fn assert_keyring_saved_auth_and_removed_fallback(
mock_keyring: &MockKeyringStore,
key: &str,
base_key: &str,
codex_home: &Path,
expected: &AuthDotJson,
) {
let saved_value = mock_keyring
.saved_value(key)
.expect("keyring entry should exist");
let expected_serialized = serde_json::to_string(expected).expect("serialize expected auth");
assert_eq!(saved_value, expected_serialized);
let expected_json = serde_json::to_value(expected).expect("auth should serialize");
let loaded = load_json_from_keyring(mock_keyring, KEYRING_SERVICE, base_key)
.expect("auth should load from keyring")
.expect("auth should exist");
assert_eq!(loaded, expected_json);
#[cfg(windows)]
assert!(
mock_keyring.saved_secret(base_key).is_none(),
"windows should store auth using split keyring entries"
);
#[cfg(not(windows))]
assert_eq!(
mock_keyring.saved_secret(base_key),
Some(serde_json::to_vec(&expected_json).expect("auth should serialize")),
"non-windows should store auth as one JSON secret"
);
let auth_file = get_auth_file(codex_home);
assert!(
!auth_file.exists(),
@@ -185,7 +272,7 @@ fn auth_with_prefix(prefix: &str) -> AuthDotJson {
}
#[test]
fn keyring_auth_storage_load_returns_deserialized_auth() -> anyhow::Result<()> {
fn keyring_auth_storage_load_supports_legacy_single_entry() -> anyhow::Result<()> {
let codex_home = tempdir()?;
let mock_keyring = MockKeyringStore::default();
let storage = KeyringAuthStorage::new(
@@ -204,6 +291,29 @@ fn keyring_auth_storage_load_returns_deserialized_auth() -> anyhow::Result<()> {
&expected,
)?;
let loaded = storage.load()?;
#[cfg(not(windows))]
{
assert_eq!(Some(expected), loaded);
}
#[cfg(windows)]
{
assert_eq!(None, loaded);
}
Ok(())
}
#[test]
fn keyring_auth_storage_load_returns_deserialized_keyring_auth() -> anyhow::Result<()> {
let codex_home = tempdir()?;
let mock_keyring = MockKeyringStore::default();
let storage = KeyringAuthStorage::new(codex_home.path().to_path_buf(), Arc::new(mock_keyring));
let expected = auth_with_prefix("keyring");
storage.save(&expected)?;
let loaded = storage.load()?;
assert_eq!(Some(expected), loaded);
Ok(())
@@ -256,17 +366,16 @@ fn keyring_auth_storage_delete_removes_keyring_and_file() -> anyhow::Result<()>
codex_home.path().to_path_buf(),
Arc::new(mock_keyring.clone()),
);
let (key, auth_file) =
seed_keyring_and_fallback_auth_file_for_delete(&mock_keyring, codex_home.path(), || {
compute_store_key(codex_home.path())
})?;
let auth = auth_with_prefix("delete");
let (base_key, auth_file) =
seed_keyring_and_fallback_auth_file_for_delete(&storage, codex_home.path(), &auth)?;
let removed = storage.delete()?;
assert!(removed, "delete should report removal");
assert!(
!mock_keyring.contains(&key),
"keyring entry should be removed"
load_json_from_keyring(&mock_keyring, KEYRING_SERVICE, &base_key)?.is_none(),
"keyring auth should be removed"
);
assert!(
!auth_file.exists(),
@@ -279,16 +388,9 @@ fn keyring_auth_storage_delete_removes_keyring_and_file() -> anyhow::Result<()>
fn auto_auth_storage_load_prefers_keyring_value() -> anyhow::Result<()> {
let codex_home = tempdir()?;
let mock_keyring = MockKeyringStore::default();
let storage = AutoAuthStorage::new(
codex_home.path().to_path_buf(),
Arc::new(mock_keyring.clone()),
);
let storage = AutoAuthStorage::new(codex_home.path().to_path_buf(), Arc::new(mock_keyring));
let keyring_auth = auth_with_prefix("keyring");
seed_keyring_with_auth(
&mock_keyring,
|| compute_store_key(codex_home.path()),
&keyring_auth,
)?;
storage.keyring_storage.save(&keyring_auth)?;
let file_auth = auth_with_prefix("file");
storage.file_storage.save(&file_auth)?;
@@ -316,12 +418,10 @@ fn auto_auth_storage_load_uses_file_when_keyring_empty() -> anyhow::Result<()> {
fn auto_auth_storage_load_falls_back_when_keyring_errors() -> anyhow::Result<()> {
let codex_home = tempdir()?;
let mock_keyring = MockKeyringStore::default();
let storage = AutoAuthStorage::new(
codex_home.path().to_path_buf(),
Arc::new(mock_keyring.clone()),
);
let key = compute_store_key(codex_home.path())?;
mock_keyring.set_error(&key, KeyringError::Invalid("error".into(), "load".into()));
let failing_keyring = LoadSecretErrorKeyringStore {
inner: mock_keyring,
};
let storage = AutoAuthStorage::new(codex_home.path().to_path_buf(), Arc::new(failing_keyring));
let expected = auth_with_prefix("fallback");
storage.file_storage.save(&expected)?;
@@ -360,12 +460,11 @@ fn auto_auth_storage_save_prefers_keyring() -> anyhow::Result<()> {
fn auto_auth_storage_save_falls_back_when_keyring_errors() -> anyhow::Result<()> {
let codex_home = tempdir()?;
let mock_keyring = MockKeyringStore::default();
let storage = AutoAuthStorage::new(
codex_home.path().to_path_buf(),
Arc::new(mock_keyring.clone()),
);
let failing_keyring = SaveSecretErrorKeyringStore {
inner: mock_keyring.clone(),
};
let storage = AutoAuthStorage::new(codex_home.path().to_path_buf(), Arc::new(failing_keyring));
let key = compute_store_key(codex_home.path())?;
mock_keyring.set_error(&key, KeyringError::Invalid("error".into(), "save".into()));
let auth = auth_with_prefix("fallback");
storage.save(&auth)?;
@@ -381,8 +480,8 @@ fn auto_auth_storage_save_falls_back_when_keyring_errors() -> anyhow::Result<()>
.context("fallback auth should exist")?;
assert_eq!(saved, auth);
assert!(
mock_keyring.saved_value(&key).is_none(),
"keyring should not contain value when save fails"
load_json_from_keyring(&mock_keyring, KEYRING_SERVICE, &key)?.is_none(),
"keyring should not point to saved auth when save fails"
);
Ok(())
}
@@ -395,17 +494,19 @@ fn auto_auth_storage_delete_removes_keyring_and_file() -> anyhow::Result<()> {
codex_home.path().to_path_buf(),
Arc::new(mock_keyring.clone()),
);
let (key, auth_file) =
seed_keyring_and_fallback_auth_file_for_delete(&mock_keyring, codex_home.path(), || {
compute_store_key(codex_home.path())
})?;
let auth = auth_with_prefix("auto-delete");
let (base_key, auth_file) = seed_keyring_and_fallback_auth_file_for_delete(
storage.keyring_storage.as_ref(),
codex_home.path(),
&auth,
)?;
let removed = storage.delete()?;
assert!(removed, "delete should report removal");
assert!(
!mock_keyring.contains(&key),
"keyring entry should be removed"
load_json_from_keyring(&mock_keyring, KEYRING_SERVICE, &base_key)?.is_none(),
"keyring auth should be removed"
);
assert!(
!auth_file.exists(),

View File

@@ -45,6 +45,9 @@ use tracing::warn;
use codex_keyring_store::DefaultKeyringStore;
use codex_keyring_store::KeyringStore;
use codex_keyring_store::delete_json_from_keyring;
use codex_keyring_store::load_json_from_keyring;
use codex_keyring_store::save_json_to_keyring;
use rmcp::transport::auth::AuthorizationManager;
use tokio::sync::Mutex;
@@ -155,16 +158,15 @@ fn load_oauth_tokens_from_keyring<K: KeyringStore>(
url: &str,
) -> Result<Option<StoredOAuthTokens>> {
let key = compute_store_key(server_name, url)?;
match keyring_store.load(KEYRING_SERVICE, &key) {
Ok(Some(serialized)) => {
let mut tokens: StoredOAuthTokens = serde_json::from_str(&serialized)
.context("failed to deserialize OAuth tokens from keyring")?;
refresh_expires_in_from_timestamp(&mut tokens);
Ok(Some(tokens))
}
Ok(None) => Ok(None),
Err(error) => Err(Error::new(error.into_error())),
}
let Some(value) = load_json_from_keyring(keyring_store, KEYRING_SERVICE, &key)
.map_err(|err| Error::msg(err.to_string()))?
else {
return Ok(None);
};
let mut tokens: StoredOAuthTokens =
serde_json::from_value(value).context("failed to deserialize OAuth tokens from keyring")?;
refresh_expires_in_from_timestamp(&mut tokens);
Ok(Some(tokens))
}
pub fn save_oauth_tokens(
@@ -191,10 +193,9 @@ fn save_oauth_tokens_with_keyring<K: KeyringStore>(
server_name: &str,
tokens: &StoredOAuthTokens,
) -> Result<()> {
let serialized = serde_json::to_string(tokens).context("failed to serialize OAuth tokens")?;
let value = serde_json::to_value(tokens).context("failed to serialize OAuth tokens")?;
let key = compute_store_key(server_name, &tokens.url)?;
match keyring_store.save(KEYRING_SERVICE, &key, &serialized) {
match save_json_to_keyring(keyring_store, KEYRING_SERVICE, &key, &value) {
Ok(()) => {
if let Err(error) = delete_oauth_tokens_from_file(&key) {
warn!("failed to remove OAuth tokens from fallback storage: {error:?}");
@@ -202,12 +203,9 @@ fn save_oauth_tokens_with_keyring<K: KeyringStore>(
Ok(())
}
Err(error) => {
let message = format!(
"failed to write OAuth tokens to keyring: {}",
error.message()
);
let message = format!("failed to write OAuth tokens to keyring: {error}");
warn!("{message}");
Err(Error::new(error.into_error()).context(message))
Err(Error::msg(message))
}
}
}
@@ -244,22 +242,20 @@ fn delete_oauth_tokens_from_keyring_and_file<K: KeyringStore>(
url: &str,
) -> Result<bool> {
let key = compute_store_key(server_name, url)?;
let keyring_result = keyring_store.delete(KEYRING_SERVICE, &key);
let keyring_removed = match keyring_result {
let keyring_removed = match delete_json_from_keyring(keyring_store, KEYRING_SERVICE, &key) {
Ok(removed) => removed,
Err(error) => {
let message = error.message();
let message = error.to_string();
warn!("failed to delete OAuth tokens from keyring: {message}");
match store_mode {
OAuthCredentialsStoreMode::Auto | OAuthCredentialsStoreMode::Keyring => {
return Err(error.into_error())
return Err(Error::msg(message))
.context("failed to delete OAuth tokens from keyring");
}
OAuthCredentialsStoreMode::File => false,
}
}
};
let file_removed = delete_oauth_tokens_from_file(&key)?;
Ok(keyring_removed || file_removed)
}
@@ -604,6 +600,9 @@ fn sha_256_prefix(value: &Value) -> Result<String> {
mod tests {
use super::*;
use anyhow::Result;
use codex_keyring_store::CredentialStoreError;
use codex_keyring_store::load_json_from_keyring;
use codex_keyring_store::save_json_to_keyring;
use keyring::Error as KeyringError;
use pretty_assertions::assert_eq;
use std::sync::Mutex;
@@ -614,6 +613,101 @@ mod tests {
use codex_keyring_store::tests::MockKeyringStore;
#[derive(Clone, Debug)]
struct KeyringStoreWithError {
inner: MockKeyringStore,
fail_delete: bool,
fail_load_secret: bool,
fail_save_secret: bool,
}
impl KeyringStoreWithError {
fn fail_delete(inner: MockKeyringStore) -> Self {
Self {
inner,
fail_delete: true,
fail_load_secret: false,
fail_save_secret: false,
}
}
fn fail_load_secret(inner: MockKeyringStore) -> Self {
Self {
inner,
fail_delete: false,
fail_load_secret: true,
fail_save_secret: false,
}
}
fn fail_save_secret(inner: MockKeyringStore) -> Self {
Self {
inner,
fail_delete: false,
fail_load_secret: false,
fail_save_secret: true,
}
}
}
impl KeyringStore for KeyringStoreWithError {
fn load(
&self,
service: &str,
account: &str,
) -> Result<Option<String>, CredentialStoreError> {
self.inner.load(service, account)
}
fn load_secret(
&self,
service: &str,
account: &str,
) -> Result<Option<Vec<u8>>, CredentialStoreError> {
if self.fail_load_secret {
return Err(CredentialStoreError::new(KeyringError::Invalid(
"error".into(),
"load".into(),
)));
}
self.inner.load_secret(service, account)
}
fn save(
&self,
service: &str,
account: &str,
value: &str,
) -> Result<(), CredentialStoreError> {
self.inner.save(service, account, value)
}
fn save_secret(
&self,
service: &str,
account: &str,
value: &[u8],
) -> Result<(), CredentialStoreError> {
if self.fail_save_secret {
return Err(CredentialStoreError::new(KeyringError::Invalid(
"error".into(),
"save".into(),
)));
}
self.inner.save_secret(service, account, value)
}
fn delete(&self, service: &str, account: &str) -> Result<bool, CredentialStoreError> {
if self.fail_delete {
return Err(CredentialStoreError::new(KeyringError::Invalid(
"error".into(),
"delete".into(),
)));
}
self.inner.delete(service, account)
}
}
struct TempCodexHome {
_guard: MutexGuard<'static, ()>,
_dir: tempfile::TempDir,
@@ -651,9 +745,9 @@ mod tests {
let store = MockKeyringStore::default();
let tokens = sample_tokens();
let expected = tokens.clone();
let serialized = serde_json::to_string(&tokens)?;
let key = super::compute_store_key(&tokens.server_name, &tokens.url)?;
store.save(KEYRING_SERVICE, &key, &serialized)?;
let value = serde_json::to_value(&tokens)?;
save_json_to_keyring(&store, KEYRING_SERVICE, &key, &value)?;
let loaded =
super::load_oauth_tokens_from_keyring(&store, &tokens.server_name, &tokens.url)?
@@ -662,6 +756,31 @@ mod tests {
Ok(())
}
#[test]
fn load_oauth_tokens_supports_legacy_single_entry() -> Result<()> {
let _env = TempCodexHome::new();
let store = MockKeyringStore::default();
let tokens = sample_tokens();
let serialized = serde_json::to_string(&tokens)?;
let key = super::compute_store_key(&tokens.server_name, &tokens.url)?;
store.save(KEYRING_SERVICE, &key, &serialized)?;
let loaded =
super::load_oauth_tokens_from_keyring(&store, &tokens.server_name, &tokens.url)?;
#[cfg(not(windows))]
{
let loaded = loaded.expect("tokens should load from keyring");
assert_tokens_match_without_expiry(&loaded, &tokens);
}
#[cfg(windows)]
{
assert!(loaded.is_none());
}
Ok(())
}
#[test]
fn load_oauth_tokens_falls_back_when_missing_in_keyring() -> Result<()> {
let _env = TempCodexHome::new();
@@ -684,11 +803,9 @@ mod tests {
#[test]
fn load_oauth_tokens_falls_back_when_keyring_errors() -> Result<()> {
let _env = TempCodexHome::new();
let store = MockKeyringStore::default();
let store = KeyringStoreWithError::fail_load_secret(MockKeyringStore::default());
let tokens = sample_tokens();
let expected = tokens.clone();
let key = super::compute_store_key(&tokens.server_name, &tokens.url)?;
store.set_error(&key, KeyringError::Invalid("error".into(), "load".into()));
super::save_oauth_tokens_to_file(&tokens)?;
@@ -719,18 +836,29 @@ mod tests {
let fallback_path = super::fallback_file_path()?;
assert!(!fallback_path.exists(), "fallback file should be removed");
let stored = store.saved_value(&key).expect("value saved to keyring");
assert_eq!(serde_json::from_str::<StoredOAuthTokens>(&stored)?, tokens);
#[cfg(windows)]
assert!(
store.saved_secret(&key).is_none(),
"windows should not store the full JSON record under the base key"
);
#[cfg(not(windows))]
assert!(
store.saved_secret(&key).is_some(),
"non-windows should store the full JSON record as one secret"
);
let stored =
super::load_oauth_tokens_from_keyring(&store, &tokens.server_name, &tokens.url)?
.expect("value saved to keyring");
assert_tokens_match_without_expiry(&stored, &tokens);
Ok(())
}
#[test]
fn save_oauth_tokens_writes_fallback_when_keyring_fails() -> Result<()> {
let _env = TempCodexHome::new();
let store = MockKeyringStore::default();
let mock_keyring = MockKeyringStore::default();
let store = KeyringStoreWithError::fail_save_secret(mock_keyring.clone());
let tokens = sample_tokens();
let key = super::compute_store_key(&tokens.server_name, &tokens.url)?;
store.set_error(&key, KeyringError::Invalid("error".into(), "save".into()));
super::save_oauth_tokens_with_keyring_with_fallback_to_file(
&store,
@@ -750,18 +878,22 @@ mod tests {
entry.access_token,
tokens.token_response.0.access_token().secret().as_str()
);
assert!(store.saved_value(&key).is_none());
assert!(mock_keyring.saved_value(&key).is_none());
assert!(
load_json_from_keyring(&mock_keyring, KEYRING_SERVICE, &key)?.is_none(),
"keyring should not point at saved OAuth tokens when save fails"
);
Ok(())
}
#[test]
fn delete_oauth_tokens_removes_all_storage() -> Result<()> {
fn delete_oauth_tokens_removes_active_storage() -> Result<()> {
let _env = TempCodexHome::new();
let store = MockKeyringStore::default();
let tokens = sample_tokens();
let serialized = serde_json::to_string(&tokens)?;
let key = super::compute_store_key(&tokens.server_name, &tokens.url)?;
store.save(KEYRING_SERVICE, &key, &serialized)?;
let value = serde_json::to_value(&tokens)?;
save_json_to_keyring(&store, KEYRING_SERVICE, &key, &value)?;
super::save_oauth_tokens_to_file(&tokens)?;
let removed = super::delete_oauth_tokens_from_keyring_and_file(
@@ -771,7 +903,10 @@ mod tests {
&tokens.url,
)?;
assert!(removed);
assert!(!store.contains(&key));
assert!(
load_json_from_keyring(&store, KEYRING_SERVICE, &key)?.is_none(),
"keyring entry should be removed"
);
assert!(!super::fallback_file_path()?.exists());
Ok(())
}
@@ -781,10 +916,13 @@ mod tests {
let _env = TempCodexHome::new();
let store = MockKeyringStore::default();
let tokens = sample_tokens();
let serialized = serde_json::to_string(&tokens)?;
let key = super::compute_store_key(&tokens.server_name, &tokens.url)?;
store.save(KEYRING_SERVICE, &key, &serialized)?;
assert!(store.contains(&key));
let value = serde_json::to_value(&tokens)?;
save_json_to_keyring(&store, KEYRING_SERVICE, &key, &value)?;
assert!(
super::load_oauth_tokens_from_keyring(&store, &tokens.server_name, &tokens.url)?
.is_some()
);
let removed = super::delete_oauth_tokens_from_keyring_and_file(
&store,
@@ -793,7 +931,14 @@ mod tests {
&tokens.url,
)?;
assert!(removed);
assert!(!store.contains(&key));
assert!(
load_json_from_keyring(&store, KEYRING_SERVICE, &key)?.is_none(),
"keyring entry should be removed"
);
assert!(
super::load_oauth_tokens_from_keyring(&store, &tokens.server_name, &tokens.url)?
.is_none()
);
assert!(!super::fallback_file_path()?.exists());
Ok(())
}
@@ -801,10 +946,11 @@ mod tests {
#[test]
fn delete_oauth_tokens_propagates_keyring_errors() -> Result<()> {
let _env = TempCodexHome::new();
let store = MockKeyringStore::default();
let store = KeyringStoreWithError::fail_delete(MockKeyringStore::default());
let tokens = sample_tokens();
let key = super::compute_store_key(&tokens.server_name, &tokens.url)?;
store.set_error(&key, KeyringError::Invalid("error".into(), "delete".into()));
let value = serde_json::to_value(&tokens)?;
save_json_to_keyring(&store, KEYRING_SERVICE, &key, &value)?;
super::save_oauth_tokens_to_file(&tokens).unwrap();
let result = super::delete_oauth_tokens_from_keyring_and_file(

2
codex-rs/v8-poc/.gitignore vendored Normal file
View File

@@ -0,0 +1,2 @@
/target/
/target-rs/

View File

@@ -0,0 +1,12 @@
load("//:defs.bzl", "codex_rust_crate")
codex_rust_crate(
name = "v8-poc",
crate_name = "codex_v8_poc",
deps_extra = ["@crates//:v8"],
)
alias(
name = "v8-poc-rusty-v8",
actual = ":v8-poc",
)

View File

@@ -0,0 +1,18 @@
[package]
name = "codex-v8-poc"
version.workspace = true
edition.workspace = true
license.workspace = true
[lib]
name = "codex_v8_poc"
path = "src/lib.rs"
[lints]
workspace = true
[dependencies]
v8 = { workspace = true }
[dev-dependencies]
pretty_assertions = { workspace = true }

View File

@@ -0,0 +1,65 @@
//! Bazel-wired proof-of-concept crate reserved for future V8 experiments.
/// Returns the Bazel label for this proof-of-concept crate.
#[must_use]
pub fn bazel_target() -> &'static str {
"//codex-rs/v8-poc:v8-poc"
}
/// Returns the embedded V8 version.
#[must_use]
pub fn embedded_v8_version() -> &'static str {
v8::V8::get_version()
}
#[cfg(test)]
mod tests {
use pretty_assertions::assert_eq;
use std::sync::Once;
use super::bazel_target;
fn initialize_v8() {
static INIT: Once = Once::new();
INIT.call_once(|| {
v8::V8::initialize_platform(v8::new_default_platform(0, false).make_shared());
v8::V8::initialize();
});
}
fn evaluate_expression(expression: &str) -> String {
initialize_v8();
let isolate = &mut v8::Isolate::new(Default::default());
v8::scope!(let scope, isolate);
let context = v8::Context::new(scope, Default::default());
let scope = &mut v8::ContextScope::new(scope, context);
let source = v8::String::new(scope, expression).expect("expression should be valid UTF-8");
let script = v8::Script::compile(scope, source, None).expect("expression should compile");
let result = script.run(scope).expect("expression should evaluate");
result.to_rust_string_lossy(scope)
}
#[test]
fn exposes_expected_bazel_target() {
assert_eq!(bazel_target(), "//codex-rs/v8-poc:v8-poc");
}
#[test]
fn exposes_embedded_v8_version() {
assert!(!super::embedded_v8_version().is_empty());
}
#[test]
fn evaluates_integer_addition() {
assert_eq!(evaluate_expression("1 + 2"), "3");
}
#[test]
fn evaluates_string_concatenation() {
assert_eq!(evaluate_expression("'hello ' + 'world'"), "hello world");
}
}

View File

@@ -1,5 +1,6 @@
exports_files([
"aws-lc-sys_memcmp_check.patch",
"rusty_v8_prebuilt_out_dir.patch",
"v8_bazel_rules.patch",
"v8_module_deps.patch",
"v8_source_portability.patch",

View File

@@ -44,7 +44,7 @@ diff --git a/builder/cc_builder.rs b/builder/cc_builder.rs
self.manifest_dir
.join("aws-lc")
@@ -742,6 +761,40 @@
}
);
let _ = fs::remove_file(exec_path);
}
+
@@ -84,3 +84,31 @@ diff --git a/builder/cc_builder.rs b/builder/cc_builder.rs
fn run_compiler_checks(&self, cc_build: &mut cc::Build) {
if self.compiler_check("stdalign_check", Vec::<&'static str>::new()) {
cc_build.define("AWS_LC_STDALIGN_AVAILABLE", Some("1"));
diff --git a/builder/main.rs b/builder/main.rs
--- a/builder/main.rs
+++ b/builder/main.rs
@@ -944,10 +944,12 @@
// iterate over all the include paths and copy them into the final output
for path in include_paths {
for child in std::fs::read_dir(path).into_iter().flatten().flatten() {
- if child.file_type().map_or(false, |t| t.is_file()) {
+ let child_path = child.path();
+
+ if child_path.is_file() {
std::fs::copy(
- child.path(),
- include_dir.join(child.path().file_name().unwrap()),
+ &child_path,
+ include_dir.join(child_path.file_name().unwrap()),
)
.expect("Failed to copy include file during build setup");
continue;
@@ -957,7 +959,7 @@
let options = fs_extra::dir::CopyOptions::new()
.skip_exist(true)
.copy_inside(true);
- fs_extra::dir::copy(child.path(), &include_dir, &options)
+ fs_extra::dir::copy(child_path, &include_dir, &options)
.expect("Failed to copy include directory during build setup");
}
}

View File

@@ -0,0 +1,52 @@
--- a/build.rs
+++ b/build.rs
@@ -577,7 +577,23 @@
path
}
+fn out_dir_abs() -> PathBuf {
+ let cwd = env::current_dir().unwrap();
+
+ // target/debug/build/rusty_v8-d9e5a424d4f96994/out/
+ let out_dir = env::var_os("OUT_DIR").expect(
+ "The 'OUT_DIR' environment is not set (it should be something like \
+ 'target/debug/rusty_v8-{hash}').",
+ );
+
+ cwd.join(out_dir)
+}
+
fn static_lib_dir() -> PathBuf {
+ if env::var_os("RUSTY_V8_ARCHIVE").is_some() {
+ return out_dir_abs().join("gn_out").join("obj");
+ }
+
build_dir().join("gn_out").join("obj")
}
@@ -794,22 +810,23 @@
}
fn print_link_flags() {
+ let target = env::var("TARGET").unwrap();
println!("cargo:rustc-link-lib=static=rusty_v8");
let should_dyn_link_libcxx = env::var("CARGO_FEATURE_USE_CUSTOM_LIBCXX")
.is_err()
+ || (target.contains("apple") && env::var("RUSTY_V8_ARCHIVE").is_ok())
|| env::var("GN_ARGS").is_ok_and(|gn_args| {
gn_args
.split_whitespace()
.any(|ba| ba == "use_custom_libcxx=false")
});
if should_dyn_link_libcxx {
// Based on https://github.com/alexcrichton/cc-rs/blob/fba7feded71ee4f63cfe885673ead6d7b4f2f454/src/lib.rs#L2462
if let Ok(stdlib) = env::var("CXXSTDLIB") {
if !stdlib.is_empty() {
println!("cargo:rustc-link-lib=dylib={stdlib}");
}
} else {
- let target = env::var("TARGET").unwrap();
if target.contains("msvc") {
// nothing to link to
} else if target.contains("apple")

View File

@@ -121,7 +121,7 @@ index 85f31b7..7314584 100644
],
outs = [
"include/inspector/Debugger.h",
@@ -4426,15 +4426,19 @@ genrule(
@@ -4426,15 +4426,18 @@ genrule(
"src/inspector/protocol/Schema.cpp",
"src/inspector/protocol/Schema.h",
],
@@ -134,7 +134,7 @@ index 85f31b7..7314584 100644
--config $(location :src/inspector/inspector_protocol_config.json) \
--config_value protocol.path=$(location :include/js_protocol.pdl) \
--output_base $(@D)/src/inspector",
local = 1,
- local = 1,
message = "Generating inspector files",
tools = [
- ":code_generator",

View File

@@ -4,6 +4,158 @@ load("@rules_cc//cc:defs.bzl", "cc_library")
package(default_visibility = ["//visibility:public"])
config_setting(
name = "platform_aarch64_unknown_linux_musl",
constraint_values = [
"@platforms//cpu:aarch64",
"@platforms//os:linux",
"@llvm//constraints/libc:musl",
],
)
config_setting(
name = "platform_x86_64_unknown_linux_musl",
constraint_values = [
"@platforms//cpu:x86_64",
"@platforms//os:linux",
"@llvm//constraints/libc:musl",
],
)
alias(
name = "v8_146_4_0_x86_64_apple_darwin",
actual = "@rusty_v8_146_4_0_x86_64_apple_darwin_archive//file",
)
alias(
name = "v8_146_4_0_aarch64_apple_darwin",
actual = "@rusty_v8_146_4_0_aarch64_apple_darwin_archive//file",
)
alias(
name = "v8_146_4_0_x86_64_unknown_linux_gnu",
actual = "@rusty_v8_146_4_0_x86_64_unknown_linux_gnu_archive//file",
)
alias(
name = "v8_146_4_0_aarch64_unknown_linux_gnu",
actual = "@rusty_v8_146_4_0_aarch64_unknown_linux_gnu_archive//file",
)
alias(
name = "v8_146_4_0_x86_64_unknown_linux_musl",
actual = "@rusty_v8_146_4_0_x86_64_unknown_linux_musl_archive//file",
)
alias(
name = "v8_146_4_0_aarch64_unknown_linux_musl",
actual = "@rusty_v8_146_4_0_aarch64_unknown_linux_musl_archive//file",
)
alias(
name = "v8_146_4_0_x86_64_pc_windows_msvc",
actual = "@rusty_v8_146_4_0_x86_64_pc_windows_msvc_archive//file",
)
alias(
name = "v8_146_4_0_aarch64_pc_windows_msvc",
actual = "@rusty_v8_146_4_0_aarch64_pc_windows_msvc_archive//file",
)
alias(
name = "v8_146_4_0_aarch64_pc_windows_gnullvm",
actual = ":v8_146_4_0_aarch64_pc_windows_msvc",
)
alias(
name = "v8_146_4_0_x86_64_pc_windows_gnullvm",
actual = ":v8_146_4_0_x86_64_pc_windows_msvc",
)
filegroup(
name = "src_binding_release_x86_64_apple_darwin",
srcs = ["@v8_crate_146_4_0//:src_binding_release_x86_64_apple_darwin"],
)
filegroup(
name = "src_binding_release_aarch64_apple_darwin",
srcs = ["@v8_crate_146_4_0//:src_binding_release_aarch64_apple_darwin"],
)
filegroup(
name = "src_binding_release_aarch64_unknown_linux_gnu",
srcs = ["@v8_crate_146_4_0//:src_binding_release_aarch64_unknown_linux_gnu"],
)
filegroup(
name = "src_binding_release_x86_64_unknown_linux_gnu",
srcs = ["@v8_crate_146_4_0//:src_binding_release_x86_64_unknown_linux_gnu"],
)
alias(
name = "src_binding_release_x86_64_unknown_linux_musl",
actual = "@rusty_v8_146_4_0_x86_64_unknown_linux_musl_binding//file",
)
alias(
name = "src_binding_release_aarch64_unknown_linux_musl",
actual = "@rusty_v8_146_4_0_aarch64_unknown_linux_musl_binding//file",
)
filegroup(
name = "src_binding_release_x86_64_pc_windows_msvc",
srcs = ["@v8_crate_146_4_0//:src_binding_release_x86_64_pc_windows_msvc"],
)
filegroup(
name = "src_binding_release_aarch64_pc_windows_msvc",
srcs = ["@v8_crate_146_4_0//:src_binding_release_aarch64_pc_windows_msvc"],
)
alias(
name = "src_binding_release_x86_64_pc_windows_gnullvm",
actual = ":src_binding_release_x86_64_pc_windows_msvc",
)
alias(
name = "src_binding_release_aarch64_pc_windows_gnullvm",
actual = ":src_binding_release_aarch64_pc_windows_msvc",
)
alias(
name = "rusty_v8_archive_for_target",
actual = select({
"@rules_rs//rs/experimental/platforms/config:aarch64-apple-darwin": ":v8_146_4_0_aarch64_apple_darwin_bazel",
"@rules_rs//rs/experimental/platforms/config:aarch64-pc-windows-gnullvm": ":v8_146_4_0_aarch64_pc_windows_gnullvm",
"@rules_rs//rs/experimental/platforms/config:aarch64-pc-windows-msvc": ":v8_146_4_0_aarch64_pc_windows_msvc",
"@rules_rs//rs/experimental/platforms/config:aarch64-unknown-linux-gnu": ":v8_146_4_0_aarch64_unknown_linux_gnu_bazel",
":platform_aarch64_unknown_linux_musl": ":v8_146_4_0_aarch64_unknown_linux_musl_release_base",
"@rules_rs//rs/experimental/platforms/config:x86_64-apple-darwin": ":v8_146_4_0_x86_64_apple_darwin_bazel",
"@rules_rs//rs/experimental/platforms/config:x86_64-pc-windows-gnullvm": ":v8_146_4_0_x86_64_pc_windows_gnullvm",
"@rules_rs//rs/experimental/platforms/config:x86_64-pc-windows-msvc": ":v8_146_4_0_x86_64_pc_windows_msvc",
"@rules_rs//rs/experimental/platforms/config:x86_64-unknown-linux-gnu": ":v8_146_4_0_x86_64_unknown_linux_gnu_bazel",
":platform_x86_64_unknown_linux_musl": ":v8_146_4_0_x86_64_unknown_linux_musl_release",
"//conditions:default": ":v8_146_4_0_x86_64_unknown_linux_gnu_bazel",
}),
)
alias(
name = "rusty_v8_binding_for_target",
actual = select({
"@rules_rs//rs/experimental/platforms/config:aarch64-apple-darwin": ":src_binding_release_aarch64_apple_darwin",
"@rules_rs//rs/experimental/platforms/config:aarch64-pc-windows-gnullvm": ":src_binding_release_aarch64_pc_windows_gnullvm",
"@rules_rs//rs/experimental/platforms/config:aarch64-pc-windows-msvc": ":src_binding_release_aarch64_pc_windows_msvc",
"@rules_rs//rs/experimental/platforms/config:aarch64-unknown-linux-gnu": ":src_binding_release_aarch64_unknown_linux_gnu",
":platform_aarch64_unknown_linux_musl": ":src_binding_release_aarch64_unknown_linux_musl",
"@rules_rs//rs/experimental/platforms/config:x86_64-apple-darwin": ":src_binding_release_x86_64_apple_darwin",
"@rules_rs//rs/experimental/platforms/config:x86_64-pc-windows-gnullvm": ":src_binding_release_x86_64_pc_windows_gnullvm",
"@rules_rs//rs/experimental/platforms/config:x86_64-pc-windows-msvc": ":src_binding_release_x86_64_pc_windows_msvc",
"@rules_rs//rs/experimental/platforms/config:x86_64-unknown-linux-gnu": ":src_binding_release_x86_64_unknown_linux_gnu",
":platform_x86_64_unknown_linux_musl": ":src_binding_release_x86_64_unknown_linux_musl",
"//conditions:default": ":src_binding_release_x86_64_unknown_linux_gnu",
}),
)
V8_COPTS = ["-std=c++20"]
V8_STATIC_LIBRARY_FEATURES = [
@@ -45,39 +197,39 @@ cc_library(
)
cc_static_library(
name = "v8_146_4_0_x86_64_apple_darwin",
name = "v8_146_4_0_aarch64_apple_darwin_bazel",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
cc_static_library(
name = "v8_146_4_0_aarch64_apple_darwin",
name = "v8_146_4_0_aarch64_unknown_linux_gnu_bazel",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
cc_static_library(
name = "v8_146_4_0_aarch64_unknown_linux_gnu",
name = "v8_146_4_0_x86_64_apple_darwin_bazel",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
cc_static_library(
name = "v8_146_4_0_x86_64_unknown_linux_gnu",
name = "v8_146_4_0_x86_64_unknown_linux_gnu_bazel",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
cc_static_library(
name = "v8_146_4_0_aarch64_unknown_linux_musl_base",
name = "v8_146_4_0_aarch64_unknown_linux_musl_release_base",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
genrule(
name = "v8_146_4_0_aarch64_unknown_linux_musl",
name = "v8_146_4_0_aarch64_unknown_linux_musl_release",
srcs = [
":v8_146_4_0_aarch64_unknown_linux_musl_base",
":v8_146_4_0_aarch64_unknown_linux_musl_release_base",
"@llvm//runtimes/compiler-rt:clang_rt.builtins.static",
],
tools = [
@@ -88,7 +240,7 @@ genrule(
cmd = """
cat > "$(@D)/merge.mri" <<'EOF'
create $@
addlib $(location :v8_146_4_0_aarch64_unknown_linux_musl_base)
addlib $(location :v8_146_4_0_aarch64_unknown_linux_musl_release_base)
addlib $(location @llvm//runtimes/compiler-rt:clang_rt.builtins.static)
save
end
@@ -99,83 +251,21 @@ EOF
)
cc_static_library(
name = "v8_146_4_0_x86_64_unknown_linux_musl",
name = "v8_146_4_0_x86_64_unknown_linux_musl_release",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
cc_static_library(
name = "v8_146_4_0_aarch64_pc_windows_msvc",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
cc_static_library(
name = "v8_146_4_0_x86_64_pc_windows_msvc",
deps = [":v8_146_4_0_binding"],
features = V8_STATIC_LIBRARY_FEATURES,
)
alias(
name = "v8_146_4_0_aarch64_pc_windows_gnullvm",
actual = ":v8_146_4_0_aarch64_pc_windows_msvc",
)
alias(
name = "v8_146_4_0_x86_64_pc_windows_gnullvm",
actual = ":v8_146_4_0_x86_64_pc_windows_msvc",
)
filegroup(
name = "src_binding_release_x86_64_apple_darwin",
srcs = ["@v8_crate_146_4_0//:src_binding_release_x86_64_apple_darwin"],
)
filegroup(
name = "src_binding_release_aarch64_apple_darwin",
srcs = ["@v8_crate_146_4_0//:src_binding_release_aarch64_apple_darwin"],
)
filegroup(
name = "src_binding_release_aarch64_unknown_linux_gnu",
name = "src_binding_release_aarch64_unknown_linux_musl_release",
srcs = ["@v8_crate_146_4_0//:src_binding_release_aarch64_unknown_linux_gnu"],
)
filegroup(
name = "src_binding_release_x86_64_unknown_linux_gnu",
name = "src_binding_release_x86_64_unknown_linux_musl_release",
srcs = ["@v8_crate_146_4_0//:src_binding_release_x86_64_unknown_linux_gnu"],
)
filegroup(
name = "src_binding_release_aarch64_unknown_linux_musl",
srcs = ["@v8_crate_146_4_0//:src_binding_release_aarch64_unknown_linux_gnu"],
)
filegroup(
name = "src_binding_release_x86_64_unknown_linux_musl",
srcs = ["@v8_crate_146_4_0//:src_binding_release_x86_64_unknown_linux_gnu"],
)
filegroup(
name = "src_binding_release_x86_64_pc_windows_msvc",
srcs = ["@v8_crate_146_4_0//:src_binding_release_x86_64_pc_windows_msvc"],
)
filegroup(
name = "src_binding_release_aarch64_pc_windows_msvc",
srcs = ["@v8_crate_146_4_0//:src_binding_release_aarch64_pc_windows_msvc"],
)
alias(
name = "src_binding_release_x86_64_pc_windows_gnullvm",
actual = ":src_binding_release_x86_64_pc_windows_msvc",
)
alias(
name = "src_binding_release_aarch64_pc_windows_gnullvm",
actual = ":src_binding_release_aarch64_pc_windows_msvc",
)
filegroup(
name = "rusty_v8_release_pair_x86_64_apple_darwin",
srcs = [
@@ -211,16 +301,16 @@ filegroup(
filegroup(
name = "rusty_v8_release_pair_x86_64_unknown_linux_musl",
srcs = [
":v8_146_4_0_x86_64_unknown_linux_musl",
":src_binding_release_x86_64_unknown_linux_musl",
":v8_146_4_0_x86_64_unknown_linux_musl_release",
":src_binding_release_x86_64_unknown_linux_musl_release",
],
)
filegroup(
name = "rusty_v8_release_pair_aarch64_unknown_linux_musl",
srcs = [
":v8_146_4_0_aarch64_unknown_linux_musl",
":src_binding_release_aarch64_unknown_linux_musl",
":v8_146_4_0_aarch64_unknown_linux_musl_release",
":src_binding_release_aarch64_unknown_linux_musl_release",
],
)

View File

@@ -1,45 +1,47 @@
# `rusty_v8` Release Artifacts
# `rusty_v8` Consumer Artifacts
This directory contains the Bazel packaging used to build and stage
target-specific `rusty_v8` release artifacts for Bazel-managed consumers.
This directory wires the `v8` crate to exact-version Bazel inputs.
Bazel consumer builds use:
- upstream `denoland/rusty_v8` release archives on Windows
- source-built V8 archives on Darwin, GNU Linux, and musl Linux
- `openai/codex` release assets for published musl release pairs
Cargo builds still use prebuilt `rusty_v8` archives by default. Only Bazel
overrides `RUSTY_V8_ARCHIVE`/`RUSTY_V8_SRC_BINDING_PATH` in `MODULE.bazel` to
select source-built local archives for its consumer builds.
Current pinned versions:
- Rust crate: `v8 = =146.4.0`
- Embedded upstream V8 source: `14.6.202.9`
- Embedded upstream V8 source for musl release builds: `14.6.202.9`
The generated release pairs include:
The consumer-facing selectors are:
- `//third_party/v8:rusty_v8_archive_for_target`
- `//third_party/v8:rusty_v8_binding_for_target`
Musl release assets are expected at the tag:
- `rusty-v8-v<crate_version>`
with these raw asset names:
- `librusty_v8_release_<target>.a.gz`
- `src_binding_release_<target>.rs`
The dedicated publishing workflow is `.github/workflows/rusty-v8-release.yml`.
It builds musl release pairs from source and keeps the release artifacts as the
statically linked form:
- `//third_party/v8:rusty_v8_release_pair_x86_64_apple_darwin`
- `//third_party/v8:rusty_v8_release_pair_aarch64_apple_darwin`
- `//third_party/v8:rusty_v8_release_pair_x86_64_unknown_linux_gnu`
- `//third_party/v8:rusty_v8_release_pair_aarch64_unknown_linux_gnu`
- `//third_party/v8:rusty_v8_release_pair_x86_64_unknown_linux_musl`
- `//third_party/v8:rusty_v8_release_pair_aarch64_unknown_linux_musl`
- `//third_party/v8:rusty_v8_release_pair_x86_64_pc_windows_msvc`
- `//third_party/v8:rusty_v8_release_pair_aarch64_pc_windows_msvc`
Each release pair contains:
- a static library built from source
- a Rust binding file copied from the exact same `v8` crate version for that
target
Cargo musl builds use `RUSTY_V8_ARCHIVE` plus a downloaded
`RUSTY_V8_SRC_BINDING_PATH` to point at those `openai/codex` release assets
directly. We do not use `RUSTY_V8_MIRROR` for musl because the upstream `v8`
crate hardcodes a `v<crate_version>` tag layout, while our musl artifacts are
published under `rusty-v8-v<crate_version>`.
Do not mix artifacts across crate versions. The archive and binding must match
the exact pinned `v8` crate version used by this repo.
The dedicated publishing workflow is:
- `.github/workflows/rusty-v8-release.yml`
That workflow currently stages musl artifacts:
- `librusty_v8_release_x86_64-unknown-linux-musl.a.gz`
- `librusty_v8_release_aarch64-unknown-linux-musl.a.gz`
- `src_binding_release_x86_64-unknown-linux-musl.rs`
- `src_binding_release_aarch64-unknown-linux-musl.rs`
During musl staging, the produced static archive is merged with the target's
LLVM `libc++` and `libc++abi` static runtime archives. Rust's musl toolchain
already provides the matching `libunwind`, so staging does not bundle a second
copy.
the exact resolved `v8` crate version in `codex-rs/Cargo.lock`.