Compare commits

...

4 Commits

Author SHA1 Message Date
Michael Bolin
a8719ac1d6 ci: tune just bazel helpers for remote iteration 2026-03-28 16:27:41 -07:00
Michael Bolin
4e27a87ec6 codex-tools: extract configured tool specs (#16129)
## Why

This continues the `codex-tools` migration by moving another passive
tool-spec layer out of `codex-core`.

After `ToolSpec` moved into `codex-tools`, `codex-core` still owned
`ConfiguredToolSpec` and `create_tools_json_for_responses_api()`. Both
are data-model and serialization helpers rather than runtime
orchestration, so keeping them in `core/src/tools/registry.rs` and
`core/src/tools/spec.rs` left passive tool-definition code coupled to
`codex-core` longer than necessary.

## What changed

- moved `ConfiguredToolSpec` into `codex-rs/tools/src/tool_spec.rs`
- moved `create_tools_json_for_responses_api()` into
`codex-rs/tools/src/tool_spec.rs`
- re-exported the new surface from `codex-rs/tools/src/lib.rs`, which
remains exports-only
- updated `core/src/client.rs`, `core/src/tools/registry.rs`, and
`core/src/tools/router.rs` to consume the extracted types and serializer
from `codex-tools`
- moved the tool-list serialization test into
`codex-rs/tools/src/tool_spec_tests.rs`
- added focused unit coverage for `ConfiguredToolSpec::name()`
- simplified `core/src/tools/spec_tests.rs` to use the extracted
`ConfiguredToolSpec::name()` directly and removed the now-redundant
local `tool_name()` helper
- updated `codex-rs/tools/README.md` so the crate boundary reflects the
newly extracted tool-spec wrapper and serialization helper

## Test plan

- `cargo test -p codex-tools`
- `CARGO_TARGET_DIR=/tmp/codex-core-configured-spec cargo test -p
codex-core --lib tools::spec::`
- `CARGO_TARGET_DIR=/tmp/codex-core-configured-spec cargo test -p
codex-core --lib client::`
- `just fix -p codex-tools -p codex-core`
- `just argument-comment-lint`

## References

- #15923
- #15928
- #15944
- #15953
- #16031
- #16047
2026-03-28 14:24:14 -07:00
Michael Bolin
ae8a3be958 bazel: refresh the expired macOS SDK pin (#16128)
## Why

macOS BuildBuddy started failing before target analysis because the
Apple CDN object pinned in
[`MODULE.bazel`](fce0f76d57/MODULE.bazel (L28-L36))
now returns `403 Forbidden`. The failure report that triggered this
change was this [BuildBuddy
invocation](https://app.buildbuddy.io/invocation/c57590e0-1bdb-4e19-a86f-74d4a7ded228).

This repo uses `@llvm//extensions:osx.bzl` via `osx.from_archive(...)`,
and that API does not discover a current SDK URL for us. It fetches
exactly the `urls`, `sha256`, and `strip_prefix` we pin. Once Apple
retires that `swcdn.apple.com` object, `@macos_sdk` stops resolving and
every downstream macOS build fails during external repository fetch.

This is the same basic failure mode we hit in
[b9fa08ec61](b9fa08ec61):
the pin itself aged out.

## How I tracked it down

1. I started from the BuildBuddy error and copied the exact
`swcdn.apple.com/.../CLTools_macOSNMOS_SDK.pkg` URL from the failure.
2. I reproduced the issue outside CI by opening that URL directly in a
browser and by running `curl -I` against it locally. Both returned `403
Forbidden`, which ruled out BuildBuddy as the root cause.
3. I searched the repo for that URL and found it hardcoded in
`MODULE.bazel`.
4. I inspected the `llvm` Bzlmod `osx` extension implementation to
confirm that `osx.from_archive(...)` is just a literal fetch of the
pinned archive metadata. There is no automatic fallback or catalog
lookup behind it.
5. I queried Apple's software update catalogs to find the current
Command Line Tools package for macOS 26.x. The useful catalog was:
-
`https://swscan.apple.com/content/catalogs/others/index-26-15-14-13-12-10.16-10.15-10.14-10.13-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz`

This is scriptable; it does not require opening a website in a browser.
The catalog is a gzip-compressed plist served over HTTP, so the workflow
is just:

   1. fetch the catalog,
   2. decompress it,
   3. search or parse the plist for `CLTools_macOSNMOS_SDK.pkg` entries,
   4. inspect the matching product metadata.

   The quick shell version I used was:

   ```shell
   curl -L <catalog-url> \
     | gzip -dc \
     | rg -n -C 6 'CLTools_macOSNMOS_SDK\.pkg|PostDate|English\.dist'
   ```

That is enough to surface the current product id, package URL, post
date, and the matching `.dist` file. If we want something less
grep-driven next time, the same catalog can be parsed structurally. For
example:

   ```python
   import gzip
   import plistlib
   import urllib.request

url =
"https://swscan.apple.com/content/catalogs/others/index-26-15-14-13-12-10.16-10.15-10.14-10.13-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz"
   with urllib.request.urlopen(url) as resp:
       catalog = plistlib.loads(gzip.decompress(resp.read()))

   for product_id, product in catalog["Products"].items():
       for package in product.get("Packages", []):
           package_url = package.get("URL", "")
           if package_url.endswith("CLTools_macOSNMOS_SDK.pkg"):
               print(product_id)
               print(product.get("PostDate"))
               print(package_url)
               print(product.get("Distributions", {}).get("English"))
   ```

In practice, `curl` was only the transport. The important part is that
the catalog itself is a machine-readable plist, so this can be
automated.
6. That catalog contains the newer `047-96692` Command Line Tools
release, and its distribution file identifies it as [Command Line Tools
for Xcode
26.4](https://swdist.apple.com/content/downloads/32/53/047-96692-A_OAHIHT53YB/ybtshxmrcju8m2qvw3w5elr4rajtg1x3y3/047-96692.English.dist).
7. I downloaded that package locally, computed its SHA-256, expanded it
with `pkgutil --expand-full`, and verified that it contains
`Payload/Library/Developer/CommandLineTools/SDKs/MacOSX26.4.sdk`, which
is the correct new `strip_prefix` for this pin.

The core debugging loop looked like this:

```shell
curl -I <stale swcdn URL>
rg 'swcdn\.apple\.com|osx\.from_archive' MODULE.bazel
curl -L <apple 26.x sucatalog> | gzip -dc | rg 'CLTools_macOSNMOS_SDK.pkg'
pkgutil --expand-full CLTools_macOSNMOS_SDK.pkg expanded
find expanded/Payload/Library/Developer/CommandLineTools/SDKs -maxdepth 1 -mindepth 1
```

## What changed

- Updated `MODULE.bazel` to point `osx.from_archive(...)` at the
currently live `047-96692` `CLTools_macOSNMOS_SDK.pkg` object.
- Updated the pinned `sha256` to match that package.
- Updated the `strip_prefix` from `MacOSX26.2.sdk` to `MacOSX26.4.sdk`.

## Verification

- `bazel --output_user_root="$(mktemp -d
/tmp/codex-bazel-sdk-fetch.XXXXXX)" build @macos_sdk//sysroot`

## Notes for next time

As long as we pin raw `swcdn.apple.com` objects, this will likely happen
again. When it does, the expected recovery path is:

1. Reproduce the `403` against the exact URL from CI.
2. Find the stale pin in `MODULE.bazel`.
3. Look up the current CLTools package in the relevant Apple software
update catalog for that macOS major version.
4. Download the replacement package and refresh both `sha256` and
`strip_prefix`.
5. Validate the new pin with a fresh `@macos_sdk` fetch, not just an
incremental Bazel build.

The important detail is that the non-`26` catalog did not surface the
macOS 26.x SDK package here; the `index-26-15-14-...` catalog was the
one that exposed the currently live replacement.
2026-03-28 21:08:19 +00:00
Michael Bolin
bc53d42fd9 codex-tools: extract tool spec models (#16047)
## Why

This continues the `codex-tools` migration by moving another passive
tool-definition layer out of `codex-core`.

After `ResponsesApiTool` and the lower-level schema adapters moved into
`codex-tools`, `core/src/client_common.rs` was still owning `ToolSpec`
and the web-search request wire types even though they are serialized
data models rather than runtime orchestration. Keeping those types in
`codex-core` makes the crate boundary look smaller than it really is and
leaves non-runtime tool-shape code coupled to core.

## What changed

- moved `ToolSpec`, `ResponsesApiWebSearchFilters`, and
`ResponsesApiWebSearchUserLocation` into
`codex-rs/tools/src/tool_spec.rs`
- added focused unit tests in `codex-rs/tools/src/tool_spec_tests.rs`
for:
  - `ToolSpec::name()`
  - web-search config conversions
  - `ToolSpec` serialization for `web_search` and `tool_search`
- kept `codex-rs/tools/src/lib.rs` exports-only by re-exporting the new
module from `lib.rs`
- reduced `core/src/client_common.rs` to a compatibility shim that
re-exports the extracted tool-spec types for current core call sites
- updated `core/src/tools/spec_tests.rs` to consume the extracted
web-search types directly from `codex-tools`
- updated `codex-rs/tools/README.md` so the crate contract reflects that
`codex-tools` now owns the passive tool-spec request models in addition
to the lower-level Responses API structs

## Test plan

- `cargo test -p codex-tools`
- `cargo test -p codex-core --lib tools::spec::`
- `cargo test -p codex-core --lib client_common::`
- `just fix -p codex-tools -p codex-core`
- `just argument-comment-lint`

## References

- #15923
- #15928
- #15944
- #15953
- #16031
2026-03-28 13:37:00 -07:00
13 changed files with 509 additions and 233 deletions

View File

@@ -5,6 +5,8 @@ set -euo pipefail
print_failed_bazel_test_logs=0
use_node_test_env=0
remote_download_toplevel=0
use_ci_config=0
ci_config_override=""
while [[ $# -gt 0 ]]; do
case "$1" in
@@ -20,6 +22,24 @@ while [[ $# -gt 0 ]]; do
remote_download_toplevel=1
shift
;;
--use-ci-config)
use_ci_config=1
shift
;;
--ci-config=*)
use_ci_config=1
ci_config_override="${1#*=}"
shift
;;
--ci-config)
if [[ $# -lt 2 ]]; then
echo "Expected a value after --ci-config" >&2
exit 1
fi
use_ci_config=1
ci_config_override="$2"
shift 2
;;
--)
shift
break
@@ -32,7 +52,7 @@ while [[ $# -gt 0 ]]; do
done
if [[ $# -eq 0 ]]; then
echo "Usage: $0 [--print-failed-test-logs] [--use-node-test-env] [--remote-download-toplevel] -- <bazel args> -- <targets>" >&2
echo "Usage: $0 [--print-failed-test-logs] [--use-node-test-env] [--remote-download-toplevel] [--use-ci-config] [--ci-config=<config>] -- <bazel args> -- <targets>" >&2
exit 1
fi
@@ -41,8 +61,23 @@ if [[ -n "${BAZEL_OUTPUT_USER_ROOT:-}" ]]; then
bazel_startup_args+=("--output_user_root=${BAZEL_OUTPUT_USER_ROOT}")
fi
runner_os="${RUNNER_OS:-}"
if [[ -z "${runner_os}" ]]; then
case "$(uname -s)" in
Darwin)
runner_os=macOS
;;
Linux)
runner_os=Linux
;;
MINGW*|MSYS*|CYGWIN*)
runner_os=Windows
;;
esac
fi
ci_config=ci-linux
case "${RUNNER_OS:-}" in
case "${runner_os}" in
macOS)
ci_config=ci-macos
;;
@@ -51,6 +86,10 @@ case "${RUNNER_OS:-}" in
;;
esac
if [[ -n "${ci_config_override}" ]]; then
ci_config="${ci_config_override}"
fi
print_bazel_test_log_tails() {
local console_log="$1"
local testlogs_dir
@@ -112,7 +151,7 @@ if [[ ${#bazel_args[@]} -eq 0 || ${#bazel_targets[@]} -eq 0 ]]; then
exit 1
fi
if [[ $use_node_test_env -eq 1 && "${RUNNER_OS:-}" != "Windows" ]]; then
if [[ $use_node_test_env -eq 1 && "${runner_os}" != "Windows" ]]; then
# Bazel test sandboxes on macOS may resolve an older Homebrew `node`
# before the `actions/setup-node` runtime on PATH.
node_bin="$(which node)"
@@ -134,6 +173,11 @@ if (( ${#bazel_startup_args[@]} > 0 )); then
bazel_cmd+=("${bazel_startup_args[@]}")
fi
ci_config_args=()
if [[ -n "${BUILDBUDDY_API_KEY:-}" || $use_ci_config -eq 1 ]]; then
ci_config_args+=("--config=${ci_config}")
fi
if [[ -n "${BUILDBUDDY_API_KEY:-}" ]]; then
echo "BuildBuddy API key is available; using remote Bazel configuration."
# Work around Bazel 9 remote repo contents cache / overlay materialization failures
@@ -142,7 +186,7 @@ if [[ -n "${BUILDBUDDY_API_KEY:-}" ]]; then
# remote execution/cache; this only disables the startup-level repo contents cache.
bazel_run_args=(
"${bazel_args[@]}"
"--config=${ci_config}"
"${ci_config_args[@]}"
"--remote_header=x-buildbuddy-api-key=${BUILDBUDDY_API_KEY}"
)
if (( ${#post_config_bazel_args[@]} > 0 )); then
@@ -158,28 +202,36 @@ if [[ -n "${BUILDBUDDY_API_KEY:-}" ]]; then
bazel_status=${PIPESTATUS[0]}
set -e
else
echo "BuildBuddy API key is not available; using local Bazel configuration."
# Keep fork/community PRs on Bazel but disable remote services that are
# configured in .bazelrc and require auth.
#
# Flag docs:
# - Command-line reference: https://bazel.build/reference/command-line-reference
# - Remote caching overview: https://bazel.build/remote/caching
# - Remote execution overview: https://bazel.build/remote/rbe
# - Build Event Protocol overview: https://bazel.build/remote/bep
#
# --noexperimental_remote_repo_contents_cache:
# disable remote repo contents cache enabled in .bazelrc startup options.
# https://bazel.build/reference/command-line-reference#startup_options-flag--experimental_remote_repo_contents_cache
# --remote_cache= and --remote_executor=:
# clear remote cache/execution endpoints configured in .bazelrc.
# https://bazel.build/reference/command-line-reference#common_options-flag--remote_cache
# https://bazel.build/reference/command-line-reference#common_options-flag--remote_executor
bazel_run_args=(
"${bazel_args[@]}"
--remote_cache=
--remote_executor=
)
bazel_run_args=("${bazel_args[@]}" "${ci_config_args[@]}")
if [[ -n "${GITHUB_ACTIONS:-}" ]]; then
echo "BuildBuddy API key is not available in GitHub Actions; disabling remote Bazel services."
# Keep fork/community PRs on Bazel but disable remote services that are
# configured in .bazelrc and require auth.
#
# Flag docs:
# - Command-line reference: https://bazel.build/reference/command-line-reference
# - Remote caching overview: https://bazel.build/remote/caching
# - Remote execution overview: https://bazel.build/remote/rbe
# - Build Event Protocol overview: https://bazel.build/remote/bep
#
# --noexperimental_remote_repo_contents_cache:
# disable remote repo contents cache enabled in .bazelrc startup options.
# https://bazel.build/reference/command-line-reference#startup_options-flag--experimental_remote_repo_contents_cache
# --remote_cache= and --remote_executor=:
# clear remote cache/execution endpoints configured in .bazelrc.
# https://bazel.build/reference/command-line-reference#common_options-flag--remote_cache
# https://bazel.build/reference/command-line-reference#common_options-flag--remote_executor
bazel_run_args+=(
--remote_cache=
--remote_executor=
)
elif [[ $use_ci_config -eq 1 ]]; then
echo "BuildBuddy API key env var is not available; using ${ci_config} with remote settings from local Bazel config."
else
echo "BuildBuddy API key env var is not available; preserving Bazel remote settings from local Bazel config."
fi
if (( ${#post_config_bazel_args[@]} > 0 )); then
bazel_run_args+=("${post_config_bazel_args[@]}")
fi

View File

@@ -27,11 +27,11 @@ register_toolchains("@llvm//toolchain:all")
osx = use_extension("@llvm//extensions:osx.bzl", "osx")
osx.from_archive(
sha256 = "6a4922f89487a96d7054ec6ca5065bfddd9f1d017c74d82f1d79cecf7feb8228",
strip_prefix = "Payload/Library/Developer/CommandLineTools/SDKs/MacOSX26.2.sdk",
sha256 = "1bde70c0b1c2ab89ff454acbebf6741390d7b7eb149ca2a3ca24cc9203a408b7",
strip_prefix = "Payload/Library/Developer/CommandLineTools/SDKs/MacOSX26.4.sdk",
type = "pkg",
urls = [
"https://swcdn.apple.com/content/downloads/26/44/047-81934-A_28TPKM5SD1/ps6pk6dk4x02vgfa5qsctq6tgf23t5f0w2/CLTools_macOSNMOS_SDK.pkg",
"https://swcdn.apple.com/content/downloads/32/53/047-96692-A_OAHIHT53YB/ybtshxmrcju8m2qvw3w5elr4rajtg1x3y3/CLTools_macOSNMOS_SDK.pkg",
],
)
osx.frameworks(names = [

View File

@@ -72,6 +72,7 @@ use codex_protocol::openai_models::ModelInfo;
use codex_protocol::openai_models::ReasoningEffort as ReasoningEffortConfig;
use codex_protocol::protocol::SessionSource;
use codex_protocol::protocol::W3cTraceContext;
use codex_tools::create_tools_json_for_responses_api;
use eventsource_stream::Event;
use eventsource_stream::EventStreamError;
use futures::StreamExt;
@@ -107,7 +108,6 @@ use crate::response_debug_context::extract_response_debug_context;
use crate::response_debug_context::extract_response_debug_context_from_api_error;
use crate::response_debug_context::telemetry_api_error_message;
use crate::response_debug_context::telemetry_transport_error_message;
use crate::tools::spec::create_tools_json_for_responses_api;
use crate::util::FeedbackRequestTags;
use crate::util::emit_feedback_auth_recovery_tags;
use crate::util::emit_feedback_request_tags_with_auth_env;

View File

@@ -1,10 +1,10 @@
use crate::client_common::tools::ToolSpec;
use crate::config::types::Personality;
use crate::error::Result;
pub use codex_api::common::ResponseEvent;
use codex_protocol::models::BaseInstructions;
use codex_protocol::models::FunctionCallOutputBody;
use codex_protocol::models::ResponseItem;
use codex_tools::ToolSpec;
use futures::Stream;
use serde::Deserialize;
use serde_json::Value;
@@ -157,107 +157,11 @@ fn strip_total_output_header(output: &str) -> Option<(&str, u32)> {
}
pub(crate) mod tools {
use codex_protocol::config_types::WebSearchContextSize;
use codex_protocol::config_types::WebSearchFilters as ConfigWebSearchFilters;
use codex_protocol::config_types::WebSearchUserLocation as ConfigWebSearchUserLocation;
use codex_protocol::config_types::WebSearchUserLocationType;
pub(crate) use codex_tools::FreeformTool;
pub(crate) use codex_tools::FreeformToolFormat;
use codex_tools::JsonSchema;
pub(crate) use codex_tools::ResponsesApiTool;
pub(crate) use codex_tools::ToolSearchOutputTool;
use serde::Serialize;
/// When serialized as JSON, this produces a valid "Tool" in the OpenAI
/// Responses API.
#[derive(Debug, Clone, Serialize, PartialEq)]
#[serde(tag = "type")]
pub(crate) enum ToolSpec {
#[serde(rename = "function")]
Function(ResponsesApiTool),
#[serde(rename = "tool_search")]
ToolSearch {
execution: String,
description: String,
parameters: JsonSchema,
},
#[serde(rename = "local_shell")]
LocalShell {},
#[serde(rename = "image_generation")]
ImageGeneration { output_format: String },
// TODO: Understand why we get an error on web_search although the API docs say it's supported.
// https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses#:~:text=%7B%20type%3A%20%22web_search%22%20%7D%2C
// The `external_web_access` field determines whether the web search is over cached or live content.
// https://platform.openai.com/docs/guides/tools-web-search#live-internet-access
#[serde(rename = "web_search")]
WebSearch {
#[serde(skip_serializing_if = "Option::is_none")]
external_web_access: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
filters: Option<ResponsesApiWebSearchFilters>,
#[serde(skip_serializing_if = "Option::is_none")]
user_location: Option<ResponsesApiWebSearchUserLocation>,
#[serde(skip_serializing_if = "Option::is_none")]
search_context_size: Option<WebSearchContextSize>,
#[serde(skip_serializing_if = "Option::is_none")]
search_content_types: Option<Vec<String>>,
},
#[serde(rename = "custom")]
Freeform(FreeformTool),
}
impl ToolSpec {
pub(crate) fn name(&self) -> &str {
match self {
ToolSpec::Function(tool) => tool.name.as_str(),
ToolSpec::ToolSearch { .. } => "tool_search",
ToolSpec::LocalShell {} => "local_shell",
ToolSpec::ImageGeneration { .. } => "image_generation",
ToolSpec::WebSearch { .. } => "web_search",
ToolSpec::Freeform(tool) => tool.name.as_str(),
}
}
}
#[derive(Debug, Clone, Serialize, PartialEq)]
pub(crate) struct ResponsesApiWebSearchFilters {
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) allowed_domains: Option<Vec<String>>,
}
impl From<ConfigWebSearchFilters> for ResponsesApiWebSearchFilters {
fn from(filters: ConfigWebSearchFilters) -> Self {
Self {
allowed_domains: filters.allowed_domains,
}
}
}
#[derive(Debug, Clone, Serialize, PartialEq)]
pub(crate) struct ResponsesApiWebSearchUserLocation {
#[serde(rename = "type")]
pub(crate) r#type: WebSearchUserLocationType,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) country: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) region: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) city: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub(crate) timezone: Option<String>,
}
impl From<ConfigWebSearchUserLocation> for ResponsesApiWebSearchUserLocation {
fn from(user_location: ConfigWebSearchUserLocation) -> Self {
Self {
r#type: user_location.r#type,
country: user_location.country,
region: user_location.region,
city: user_location.city,
timezone: user_location.timezone,
}
}
}
pub(crate) use codex_tools::ToolSpec;
}
pub struct ResponseStream {

View File

@@ -24,6 +24,7 @@ use codex_hooks::HookToolInput;
use codex_hooks::HookToolInputLocalShell;
use codex_hooks::HookToolKind;
use codex_protocol::models::ResponseInputItem;
use codex_tools::ConfiguredToolSpec;
use codex_utils_readiness::Readiness;
use serde_json::Value;
use tracing::warn;
@@ -436,21 +437,6 @@ impl ToolRegistry {
}
}
#[derive(Debug, Clone)]
pub struct ConfiguredToolSpec {
pub spec: ToolSpec,
pub supports_parallel_tool_calls: bool,
}
impl ConfiguredToolSpec {
pub fn new(spec: ToolSpec, supports_parallel_tool_calls: bool) -> Self {
Self {
spec,
supports_parallel_tool_calls,
}
}
}
pub struct ToolRegistryBuilder {
handlers: HashMap<String, Arc<dyn AnyToolHandler>>,
specs: Vec<ConfiguredToolSpec>,

View File

@@ -9,7 +9,6 @@ use crate::tools::context::ToolInvocation;
use crate::tools::context::ToolPayload;
use crate::tools::discoverable::DiscoverableTool;
use crate::tools::registry::AnyToolResult;
use crate::tools::registry::ConfiguredToolSpec;
use crate::tools::registry::ToolRegistry;
use crate::tools::spec::ToolsConfig;
use crate::tools::spec::build_specs_with_discoverable_tools;
@@ -18,6 +17,7 @@ use codex_protocol::models::LocalShellAction;
use codex_protocol::models::ResponseItem;
use codex_protocol::models::SearchToolCallParams;
use codex_protocol::models::ShellToolCallParams;
use codex_tools::ConfiguredToolSpec;
use rmcp::model::Tool;
use std::collections::HashMap;
use std::sync::Arc;
@@ -66,7 +66,7 @@ impl ToolRouter {
specs
.iter()
.filter_map(|configured_tool| {
if !codex_code_mode::is_code_mode_nested_tool(configured_tool.spec.name()) {
if !codex_code_mode::is_code_mode_nested_tool(configured_tool.name()) {
Some(configured_tool.spec.clone())
} else {
None
@@ -101,7 +101,7 @@ impl ToolRouter {
pub fn find_spec(&self, tool_name: &str) -> Option<ToolSpec> {
self.specs
.iter()
.find(|config| config.spec.name() == tool_name)
.find(|config| config.name() == tool_name)
.map(|config| config.spec.clone())
}
@@ -109,7 +109,7 @@ impl ToolRouter {
self.specs
.iter()
.filter(|config| config.supports_parallel_tool_calls)
.any(|config| config.spec.name() == tool_name)
.any(|config| config.name() == tool_name)
}
#[instrument(level = "trace", skip_all, err)]

View File

@@ -2351,22 +2351,6 @@ pub(crate) struct ApplyPatchToolArgs {
pub(crate) input: String,
}
/// Returns JSON values that are compatible with Function Calling in the
/// Responses API:
/// https://platform.openai.com/docs/guides/function-calling?api-mode=responses
pub fn create_tools_json_for_responses_api(
tools: &[ToolSpec],
) -> crate::error::Result<Vec<serde_json::Value>> {
let mut tools_json = Vec::new();
for tool in tools {
let json = serde_json::to_value(tool)?;
tools_json.push(json);
}
Ok(tools_json)
}
fn push_tool_spec(
builder: &mut ToolRegistryBuilder,
spec: ToolSpec,

View File

@@ -1,17 +1,19 @@
use crate::client_common::tools::FreeformTool;
use crate::config::test_config;
use crate::models_manager::manager::ModelsManager;
use crate::models_manager::model_info::with_config_overrides;
use crate::shell::Shell;
use crate::shell::ShellType;
use crate::tools::ToolRouter;
use crate::tools::registry::ConfiguredToolSpec;
use crate::tools::router::ToolRouterParams;
use codex_app_server_protocol::AppInfo;
use codex_protocol::openai_models::InputModality;
use codex_protocol::openai_models::ModelInfo;
use codex_protocol::openai_models::ModelsResponse;
use codex_tools::AdditionalProperties;
use codex_tools::ConfiguredToolSpec;
use codex_tools::FreeformTool;
use codex_tools::ResponsesApiWebSearchFilters;
use codex_tools::ResponsesApiWebSearchUserLocation;
use codex_tools::mcp_tool_to_deferred_responses_api_tool;
use codex_utils_absolute_path::AbsolutePathBuf;
use pretty_assertions::assert_eq;
@@ -105,23 +107,12 @@ fn deferred_responses_api_tool_serializes_with_defer_loading() {
);
}
fn tool_name(tool: &ToolSpec) -> &str {
match tool {
ToolSpec::Function(ResponsesApiTool { name, .. }) => name,
ToolSpec::ToolSearch { .. } => "tool_search",
ToolSpec::LocalShell {} => "local_shell",
ToolSpec::ImageGeneration { .. } => "image_generation",
ToolSpec::WebSearch { .. } => "web_search",
ToolSpec::Freeform(FreeformTool { name, .. }) => name,
}
}
// Avoid order-based assertions; compare via set containment instead.
fn assert_contains_tool_names(tools: &[ConfiguredToolSpec], expected_subset: &[&str]) {
use std::collections::HashSet;
let mut names = HashSet::new();
let mut duplicates = Vec::new();
for name in tools.iter().map(|t| tool_name(&t.spec)) {
for name in tools.iter().map(ConfiguredToolSpec::name) {
if !names.insert(name) {
duplicates.push(name);
}
@@ -141,7 +132,7 @@ fn assert_contains_tool_names(tools: &[ConfiguredToolSpec], expected_subset: &[&
fn assert_lacks_tool_name(tools: &[ConfiguredToolSpec], expected_absent: &str) {
let names = tools
.iter()
.map(|tool| tool_name(&tool.spec))
.map(ConfiguredToolSpec::name)
.collect::<Vec<_>>();
assert!(
!names.contains(&expected_absent),
@@ -162,7 +153,7 @@ fn shell_tool_name(config: &ToolsConfig) -> Option<&'static str> {
fn find_tool<'a>(tools: &'a [ConfiguredToolSpec], expected_name: &str) -> &'a ConfiguredToolSpec {
tools
.iter()
.find(|tool| tool_name(&tool.spec) == expected_name)
.find(|tool| tool.name() == expected_name)
.unwrap_or_else(|| panic!("expected tool {expected_name}"))
}
@@ -294,7 +285,7 @@ fn test_full_toolset_specs_for_gpt5_codex_unified_exec_web_search() {
let mut actual: BTreeMap<String, ToolSpec> = BTreeMap::from([]);
let mut duplicate_names = Vec::new();
for t in &tools {
let name = tool_name(&t.spec).to_string();
let name = t.name().to_string();
if actual.insert(name.clone(), t.spec.clone()).is_some() {
duplicate_names.push(name);
}
@@ -323,7 +314,7 @@ fn test_full_toolset_specs_for_gpt5_codex_unified_exec_web_search() {
},
create_view_image_tool(config.can_request_original_image_detail),
] {
expected.insert(tool_name(&spec).to_string(), spec);
expected.insert(spec.name().to_string(), spec);
}
let collab_specs = if config.multi_agent_v2 {
vec![
@@ -341,16 +332,16 @@ fn test_full_toolset_specs_for_gpt5_codex_unified_exec_web_search() {
]
};
for spec in collab_specs {
expected.insert(tool_name(&spec).to_string(), spec);
expected.insert(spec.name().to_string(), spec);
}
if !config.multi_agent_v2 {
let spec = create_resume_agent_tool();
expected.insert(tool_name(&spec).to_string(), spec);
expected.insert(spec.name().to_string(), spec);
}
if config.exec_permission_approvals_enabled {
let spec = create_request_permissions_tool();
expected.insert(tool_name(&spec).to_string(), spec);
expected.insert(spec.name().to_string(), spec);
}
// Exact name set match — this is the only test allowed to fail when tools change.
@@ -1205,10 +1196,10 @@ fn web_search_config_is_forwarded_to_tool_spec() {
external_web_access: Some(true),
filters: web_search_config
.filters
.map(crate::client_common::tools::ResponsesApiWebSearchFilters::from),
.map(ResponsesApiWebSearchFilters::from),
user_location: web_search_config
.user_location
.map(crate::client_common::tools::ResponsesApiWebSearchUserLocation::from),
.map(ResponsesApiWebSearchUserLocation::from),
search_context_size: web_search_config.search_context_size,
search_content_types: None,
}
@@ -1684,11 +1675,7 @@ fn test_test_model_info_includes_sync_tool() {
)
.build();
assert!(
tools
.iter()
.any(|tool| tool_name(&tool.spec) == "test_sync_tool")
);
assert!(tools.iter().any(|tool| tool.name() == "test_sync_tool"));
}
#[test]
@@ -1822,7 +1809,7 @@ fn test_build_specs_mcp_tools_sorted_by_name() {
// Only assert that the MCP tools themselves are sorted by fully-qualified name.
let mcp_names: Vec<_> = tools
.iter()
.map(|t| tool_name(&t.spec).to_string())
.map(|t| t.name().to_string())
.filter(|n| n.starts_with("test_server/"))
.collect();
let expected = vec![
@@ -2069,7 +2056,7 @@ fn tool_suggest_is_not_registered_without_feature_flag() {
assert!(
!tools
.iter()
.any(|tool| tool_name(&tool.spec) == TOOL_SUGGEST_TOOL_NAME)
.any(|tool| tool.name() == TOOL_SUGGEST_TOOL_NAME)
);
}
@@ -2160,7 +2147,7 @@ fn tool_suggest_requires_apps_and_plugins_features() {
assert!(
!tools
.iter()
.any(|tool| tool_name(&tool.spec) == TOOL_SUGGEST_TOOL_NAME),
.any(|tool| tool.name() == TOOL_SUGGEST_TOOL_NAME),
"tool_suggest should be absent when {disabled_feature:?} is disabled"
);
}
@@ -3111,38 +3098,3 @@ fn code_mode_exec_description_omits_nested_tool_details_when_not_code_mode_only(
assert!(!description.contains("### `update_plan` (`update_plan`)"));
assert!(!description.contains("### `view_image` (`view_image`)"));
}
#[test]
fn chat_tools_include_top_level_name() {
let properties =
BTreeMap::from([("foo".to_string(), JsonSchema::String { description: None })]);
let tools = vec![ToolSpec::Function(ResponsesApiTool {
name: "demo".to_string(),
description: "A demo tool".to_string(),
strict: false,
defer_loading: None,
parameters: JsonSchema::Object {
properties,
required: None,
additional_properties: None,
},
output_schema: None,
})];
let responses_json = create_tools_json_for_responses_api(&tools).unwrap();
assert_eq!(
responses_json,
vec![json!({
"type": "function",
"name": "demo",
"description": "A demo tool",
"strict": false,
"parameters": {
"type": "object",
"properties": {
"foo": { "type": "string" }
},
},
})]
);
}

View File

@@ -11,15 +11,20 @@ schema and Responses API tool primitives that no longer need to live in
- `JsonSchema`
- `AdditionalProperties`
- `ToolDefinition`
- `ToolSpec`
- `ConfiguredToolSpec`
- `ResponsesApiTool`
- `FreeformTool`
- `FreeformToolFormat`
- `ToolSearchOutputTool`
- `ResponsesApiWebSearchFilters`
- `ResponsesApiWebSearchUserLocation`
- `ResponsesApiNamespace`
- `ResponsesApiNamespaceTool`
- `parse_tool_input_schema()`
- `parse_dynamic_tool()`
- `parse_mcp_tool()`
- `create_tools_json_for_responses_api()`
- `mcp_call_tool_result_output_schema()`
- `tool_definition_to_responses_api_tool()`
- `dynamic_tool_to_responses_api_tool()`

View File

@@ -6,6 +6,7 @@ mod json_schema;
mod mcp_tool;
mod responses_api;
mod tool_definition;
mod tool_spec;
pub use dynamic_tool::parse_dynamic_tool;
pub use json_schema::AdditionalProperties;
@@ -24,3 +25,8 @@ pub use responses_api::mcp_tool_to_deferred_responses_api_tool;
pub use responses_api::mcp_tool_to_responses_api_tool;
pub use responses_api::tool_definition_to_responses_api_tool;
pub use tool_definition::ToolDefinition;
pub use tool_spec::ConfiguredToolSpec;
pub use tool_spec::ResponsesApiWebSearchFilters;
pub use tool_spec::ResponsesApiWebSearchUserLocation;
pub use tool_spec::ToolSpec;
pub use tool_spec::create_tools_json_for_responses_api;

View File

@@ -0,0 +1,141 @@
use crate::FreeformTool;
use crate::JsonSchema;
use crate::ResponsesApiTool;
use codex_protocol::config_types::WebSearchContextSize;
use codex_protocol::config_types::WebSearchFilters as ConfigWebSearchFilters;
use codex_protocol::config_types::WebSearchUserLocation as ConfigWebSearchUserLocation;
use codex_protocol::config_types::WebSearchUserLocationType;
use serde::Serialize;
use serde_json::Value;
/// When serialized as JSON, this produces a valid "Tool" in the OpenAI
/// Responses API.
#[derive(Debug, Clone, Serialize, PartialEq)]
#[serde(tag = "type")]
pub enum ToolSpec {
#[serde(rename = "function")]
Function(ResponsesApiTool),
#[serde(rename = "tool_search")]
ToolSearch {
execution: String,
description: String,
parameters: JsonSchema,
},
#[serde(rename = "local_shell")]
LocalShell {},
#[serde(rename = "image_generation")]
ImageGeneration { output_format: String },
// TODO: Understand why we get an error on web_search although the API docs
// say it's supported.
// https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses#:~:text=%7B%20type%3A%20%22web_search%22%20%7D%2C
// The `external_web_access` field determines whether the web search is over
// cached or live content.
// https://platform.openai.com/docs/guides/tools-web-search#live-internet-access
#[serde(rename = "web_search")]
WebSearch {
#[serde(skip_serializing_if = "Option::is_none")]
external_web_access: Option<bool>,
#[serde(skip_serializing_if = "Option::is_none")]
filters: Option<ResponsesApiWebSearchFilters>,
#[serde(skip_serializing_if = "Option::is_none")]
user_location: Option<ResponsesApiWebSearchUserLocation>,
#[serde(skip_serializing_if = "Option::is_none")]
search_context_size: Option<WebSearchContextSize>,
#[serde(skip_serializing_if = "Option::is_none")]
search_content_types: Option<Vec<String>>,
},
#[serde(rename = "custom")]
Freeform(FreeformTool),
}
impl ToolSpec {
pub fn name(&self) -> &str {
match self {
ToolSpec::Function(tool) => tool.name.as_str(),
ToolSpec::ToolSearch { .. } => "tool_search",
ToolSpec::LocalShell {} => "local_shell",
ToolSpec::ImageGeneration { .. } => "image_generation",
ToolSpec::WebSearch { .. } => "web_search",
ToolSpec::Freeform(tool) => tool.name.as_str(),
}
}
}
#[derive(Debug, Clone, PartialEq)]
pub struct ConfiguredToolSpec {
pub spec: ToolSpec,
pub supports_parallel_tool_calls: bool,
}
impl ConfiguredToolSpec {
pub fn new(spec: ToolSpec, supports_parallel_tool_calls: bool) -> Self {
Self {
spec,
supports_parallel_tool_calls,
}
}
pub fn name(&self) -> &str {
self.spec.name()
}
}
/// Returns JSON values that are compatible with Function Calling in the
/// Responses API:
/// https://platform.openai.com/docs/guides/function-calling?api-mode=responses
pub fn create_tools_json_for_responses_api(
tools: &[ToolSpec],
) -> Result<Vec<Value>, serde_json::Error> {
let mut tools_json = Vec::new();
for tool in tools {
let json = serde_json::to_value(tool)?;
tools_json.push(json);
}
Ok(tools_json)
}
#[derive(Debug, Clone, Serialize, PartialEq)]
pub struct ResponsesApiWebSearchFilters {
#[serde(skip_serializing_if = "Option::is_none")]
pub allowed_domains: Option<Vec<String>>,
}
impl From<ConfigWebSearchFilters> for ResponsesApiWebSearchFilters {
fn from(filters: ConfigWebSearchFilters) -> Self {
Self {
allowed_domains: filters.allowed_domains,
}
}
}
#[derive(Debug, Clone, Serialize, PartialEq)]
pub struct ResponsesApiWebSearchUserLocation {
#[serde(rename = "type")]
pub r#type: WebSearchUserLocationType,
#[serde(skip_serializing_if = "Option::is_none")]
pub country: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub region: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub city: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub timezone: Option<String>,
}
impl From<ConfigWebSearchUserLocation> for ResponsesApiWebSearchUserLocation {
fn from(user_location: ConfigWebSearchUserLocation) -> Self {
Self {
r#type: user_location.r#type,
country: user_location.country,
region: user_location.region,
city: user_location.city,
timezone: user_location.timezone,
}
}
}
#[cfg(test)]
#[path = "tool_spec_tests.rs"]
mod tests;

View File

@@ -0,0 +1,242 @@
use super::ConfiguredToolSpec;
use super::ResponsesApiWebSearchFilters;
use super::ResponsesApiWebSearchUserLocation;
use super::ToolSpec;
use crate::AdditionalProperties;
use crate::FreeformTool;
use crate::FreeformToolFormat;
use crate::JsonSchema;
use crate::ResponsesApiTool;
use crate::create_tools_json_for_responses_api;
use codex_protocol::config_types::WebSearchContextSize;
use codex_protocol::config_types::WebSearchFilters as ConfigWebSearchFilters;
use codex_protocol::config_types::WebSearchUserLocation as ConfigWebSearchUserLocation;
use codex_protocol::config_types::WebSearchUserLocationType;
use pretty_assertions::assert_eq;
use serde_json::json;
use std::collections::BTreeMap;
#[test]
fn tool_spec_name_covers_all_variants() {
assert_eq!(
ToolSpec::Function(ResponsesApiTool {
name: "lookup_order".to_string(),
description: "Look up an order".to_string(),
strict: false,
defer_loading: None,
parameters: JsonSchema::Object {
properties: BTreeMap::new(),
required: None,
additional_properties: None,
},
output_schema: None,
})
.name(),
"lookup_order"
);
assert_eq!(
ToolSpec::ToolSearch {
execution: "sync".to_string(),
description: "Search for tools".to_string(),
parameters: JsonSchema::Object {
properties: BTreeMap::new(),
required: None,
additional_properties: None,
},
}
.name(),
"tool_search"
);
assert_eq!(ToolSpec::LocalShell {}.name(), "local_shell");
assert_eq!(
ToolSpec::ImageGeneration {
output_format: "png".to_string(),
}
.name(),
"image_generation"
);
assert_eq!(
ToolSpec::WebSearch {
external_web_access: Some(true),
filters: None,
user_location: None,
search_context_size: None,
search_content_types: None,
}
.name(),
"web_search"
);
assert_eq!(
ToolSpec::Freeform(FreeformTool {
name: "exec".to_string(),
description: "Run a command".to_string(),
format: FreeformToolFormat {
r#type: "grammar".to_string(),
syntax: "lark".to_string(),
definition: "start: \"exec\"".to_string(),
},
})
.name(),
"exec"
);
}
#[test]
fn configured_tool_spec_name_delegates_to_tool_spec() {
assert_eq!(
ConfiguredToolSpec::new(
ToolSpec::Function(ResponsesApiTool {
name: "lookup_order".to_string(),
description: "Look up an order".to_string(),
strict: false,
defer_loading: None,
parameters: JsonSchema::Object {
properties: BTreeMap::new(),
required: None,
additional_properties: None,
},
output_schema: None,
}),
/*supports_parallel_tool_calls*/ true,
)
.name(),
"lookup_order"
);
}
#[test]
fn web_search_config_converts_to_responses_api_types() {
assert_eq!(
ResponsesApiWebSearchFilters::from(ConfigWebSearchFilters {
allowed_domains: Some(vec!["example.com".to_string()]),
}),
ResponsesApiWebSearchFilters {
allowed_domains: Some(vec!["example.com".to_string()]),
}
);
assert_eq!(
ResponsesApiWebSearchUserLocation::from(ConfigWebSearchUserLocation {
r#type: WebSearchUserLocationType::Approximate,
country: Some("US".to_string()),
region: Some("California".to_string()),
city: Some("San Francisco".to_string()),
timezone: Some("America/Los_Angeles".to_string()),
}),
ResponsesApiWebSearchUserLocation {
r#type: WebSearchUserLocationType::Approximate,
country: Some("US".to_string()),
region: Some("California".to_string()),
city: Some("San Francisco".to_string()),
timezone: Some("America/Los_Angeles".to_string()),
}
);
}
#[test]
fn create_tools_json_for_responses_api_includes_top_level_name() {
assert_eq!(
create_tools_json_for_responses_api(&[ToolSpec::Function(ResponsesApiTool {
name: "demo".to_string(),
description: "A demo tool".to_string(),
strict: false,
defer_loading: None,
parameters: JsonSchema::Object {
properties: BTreeMap::from([(
"foo".to_string(),
JsonSchema::String { description: None },
)]),
required: None,
additional_properties: None,
},
output_schema: None,
})])
.expect("serialize tools"),
vec![json!({
"type": "function",
"name": "demo",
"description": "A demo tool",
"strict": false,
"parameters": {
"type": "object",
"properties": {
"foo": { "type": "string" }
},
},
})]
);
}
#[test]
fn web_search_tool_spec_serializes_expected_wire_shape() {
assert_eq!(
serde_json::to_value(ToolSpec::WebSearch {
external_web_access: Some(true),
filters: Some(ResponsesApiWebSearchFilters {
allowed_domains: Some(vec!["example.com".to_string()]),
}),
user_location: Some(ResponsesApiWebSearchUserLocation {
r#type: WebSearchUserLocationType::Approximate,
country: Some("US".to_string()),
region: Some("California".to_string()),
city: Some("San Francisco".to_string()),
timezone: Some("America/Los_Angeles".to_string()),
}),
search_context_size: Some(WebSearchContextSize::High),
search_content_types: Some(vec!["text".to_string(), "image".to_string()]),
})
.expect("serialize web_search"),
json!({
"type": "web_search",
"external_web_access": true,
"filters": {
"allowed_domains": ["example.com"],
},
"user_location": {
"type": "approximate",
"country": "US",
"region": "California",
"city": "San Francisco",
"timezone": "America/Los_Angeles",
},
"search_context_size": "high",
"search_content_types": ["text", "image"],
})
);
}
#[test]
fn tool_search_tool_spec_serializes_expected_wire_shape() {
assert_eq!(
serde_json::to_value(ToolSpec::ToolSearch {
execution: "sync".to_string(),
description: "Search app tools".to_string(),
parameters: JsonSchema::Object {
properties: BTreeMap::from([(
"query".to_string(),
JsonSchema::String {
description: Some("Tool search query".to_string()),
},
)]),
required: Some(vec!["query".to_string()]),
additional_properties: Some(AdditionalProperties::Boolean(false)),
},
})
.expect("serialize tool_search"),
json!({
"type": "tool_search",
"execution": "sync",
"description": "Search app tools",
"parameters": {
"type": "object",
"properties": {
"query": {
"type": "string",
"description": "Tool search query",
}
},
"required": ["query"],
"additionalProperties": false,
},
})
);
}

View File

@@ -67,17 +67,21 @@ bazel-lock-check:
./scripts/check-module-bazel-lock.sh
bazel-test:
bazel test --test_tag_filters=-argument-comment-lint //... --keep_going
bazel test --test_tag_filters=-argument-comment-lint --keep_going -- //... -//third_party/v8:all
bazel-clippy:
bazel build --config=clippy -- //codex-rs/... -//codex-rs/v8-poc:all
[no-cd]
bazel-argument-comment-lint:
bazel build --config=argument-comment-lint -- //codex-rs/...
./.github/scripts/run-bazel-ci.sh -- build --config=argument-comment-lint --keep_going -- //codex-rs/...
# Fast local iteration helper: prefer the fully remote Linux path, even on
# macOS, to keep local CPU/RAM use down. For best same-clone reruns, keep
# BuildBuddy auth plus `build/test --watchfs` in `~/.bazelrc`.
[no-cd]
bazel-remote-test:
bazel test --test_tag_filters=-argument-comment-lint //... --config=remote --platforms=//:rbe --keep_going
./.github/scripts/run-bazel-ci.sh --print-failed-test-logs --use-node-test-env --ci-config=ci-linux -- test --test_tag_filters=-argument-comment-lint --test_verbose_timeout_warnings -- //... -//third_party/v8:all
build-for-release:
bazel build //codex-rs/cli:release_binaries --config=remote
@@ -102,7 +106,7 @@ write-hooks-schema:
[no-cd]
argument-comment-lint *args:
if [ "$#" -eq 0 ]; then \
bazel build --config=argument-comment-lint -- //codex-rs/...; \
./.github/scripts/run-bazel-ci.sh -- build --config=argument-comment-lint --keep_going -- //codex-rs/...; \
else \
./tools/argument-comment-lint/run-prebuilt-linter.py "$@"; \
fi