[codex] Move config loading into codex-config (#19487)

## Why

Config loading had become split across crates: `codex-config` owned the
config types and merge logic, while `codex-core` still owned the loader
that assembled the layer stack. This change consolidates that
responsibility in `codex-config`, so the crate that defines config
behavior also owns how configs are discovered and loaded.

To make that move possible without reintroducing the old dependency
cycle, the shell-environment policy types and helpers that
`codex-exec-server` needs now live in `codex-protocol` instead of
flowing through `codex-config`.

This also makes the migrated loader tests more deterministic on machines
that already have managed or system Codex config installed by letting
tests override the system config and requirements paths instead of
reading the host's `/etc/codex`.

## What Changed

- moved the config loader implementation from `codex-core` into
`codex-config::loader` and deleted the old `core::config_loader` module
instead of leaving a compatibility shim
- moved shell-environment policy types and helpers into
`codex-protocol`, then updated `codex-exec-server` and other downstream
crates to import them from their new home
- updated downstream callers to use loader/config APIs from
`codex-config`
- added test-only loader overrides for system config and requirements
paths so loader-focused tests do not depend on host-managed config state
- cleaned up now-unused dependency entries and platform-specific cfgs
that were surfaced by post-push CI

## Testing

- `cargo test -p codex-config`
- `cargo test -p codex-core config_loader_tests::`
- `cargo test -p codex-protocol -p codex-exec-server -p
codex-cloud-requirements -p codex-rmcp-client --lib`
- `cargo test --lib -p codex-app-server-client -p codex-exec`
- `cargo test --no-run --lib -p codex-app-server`
- `cargo test -p codex-linux-sandbox --lib`
- `cargo shear`
- `just bazel-lock-check`

## Notes

- I did not chase unrelated full-suite failures outside the migrated
loader surface.
- `cargo test -p codex-core --lib` still hits unrelated proxy-sensitive
failures on this machine, and Windows CI still shows unrelated
long-running/timeouting test noise outside the loader migration itself.

codex-rs/config/src/loader/README.md

@@ -0,0 +1,79 @@
+# `codex-config` loader
+
+This module is the canonical place to **load and describe Codex configuration layers** (user config, CLI/session overrides, managed config, and MDM-managed preferences) and to produce:
+
+- An **effective merged** TOML config.
+- **Per-key origins** metadata (which layer “wins” for a given key).
+- **Per-layer versions** (stable fingerprints) used for optimistic concurrency / conflict detection.
+
+## Public surface
+
+Exported from `codex_config::loader`:
+
+- `load_config_layers_state(fs, codex_home, cwd_opt, cli_overrides, overrides, cloud_requirements, thread_config_loader, host_name) -> ConfigLayerStack`
+- `ConfigLayerStack`
+  - `effective_config() -> toml::Value`
+  - `origins() -> HashMap<String, ConfigLayerMetadata>`
+  - `layers_high_to_low() -> Vec<ConfigLayer>`
+  - `with_user_config(user_config) -> ConfigLayerStack`
+- `ConfigLayerEntry` (one layer’s `{name, config, version, disabled_reason}`; `name` carries source metadata)
+- `LoaderOverrides` (test/override hooks for managed config sources)
+- `merge_toml_values(base, overlay)` (public helper used elsewhere)
+
+## Layering model
+
+Precedence is **top overrides bottom**:
+
+1. **MDM** managed preferences (macOS only)
+2. **System** managed config (e.g. `managed_config.toml`)
+3. **Session flags** (CLI overrides, applied as dotted-path TOML writes)
+4. **User** config (`config.toml`)
+
+Thread config entries supplied by `thread_config_loader` are inserted according
+to their translated `ConfigLayerSource` precedence.
+
+Layers with a `disabled_reason` are still surfaced for UI, but are ignored when
+computing the effective config and origins metadata. This is what
+`ConfigLayerStack::effective_config()` implements.
+
+## Typical usage
+
+Most callers want the effective config plus metadata:
+
+```rust
+use codex_config::NoopThreadConfigLoader;
+use codex_config::CloudRequirementsLoader;
+use codex_config::LoaderOverrides;
+use codex_config::loader::load_config_layers_state;
+use codex_exec_server::LOCAL_FS;
+use codex_utils_absolute_path::AbsolutePathBuf;
+use toml::Value as TomlValue;
+
+let cli_overrides: Vec<(String, TomlValue)> = Vec::new();
+let cwd = AbsolutePathBuf::current_dir()?;
+let layers = load_config_layers_state(
+    LOCAL_FS.as_ref(),
+    &codex_home,
+    Some(cwd),
+    &cli_overrides,
+    LoaderOverrides::default(),
+    CloudRequirementsLoader::default(),
+    &NoopThreadConfigLoader,
+    /*host_name*/ None,
+).await?;
+
+let effective = layers.effective_config();
+let origins = layers.origins();
+let layers_for_ui = layers.layers_high_to_low();
+```
+
+## Internal layout
+
+Implementation is split by concern:
+
+- `state.rs`: public types (`ConfigLayerEntry`, `ConfigLayerStack`) + merge/origins convenience methods.
+- `layer_io.rs`: reading `config.toml`, managed config, and managed preferences inputs.
+- `overrides.rs`: CLI dotted-path overrides → TOML “session flags” layer.
+- `merge.rs`: recursive TOML merge.
+- `fingerprint.rs`: stable per-layer hashing and per-key origins traversal.
+- `macos.rs`: managed preferences integration (macOS only).

codex-rs/config/src/loader/layer_io.rs

@@ -0,0 +1,137 @@
+#[cfg(target_os = "macos")]
+use super::macos::ManagedAdminConfigLayer;
+#[cfg(target_os = "macos")]
+use super::macos::load_managed_admin_config_layer;
+use crate::diagnostics::config_error_from_toml;
+use crate::diagnostics::io_error_from_config_error;
+use crate::state::LoaderOverrides;
+use codex_exec_server::ExecutorFileSystem;
+use codex_utils_absolute_path::AbsolutePathBuf;
+use std::io;
+use std::path::Path;
+use std::path::PathBuf;
+use toml::Value as TomlValue;
+
+#[cfg(unix)]
+const CODEX_MANAGED_CONFIG_SYSTEM_PATH: &str = "/etc/codex/managed_config.toml";
+
+#[derive(Debug, Clone)]
+pub(super) struct MangedConfigFromFile {
+    pub managed_config: TomlValue,
+    pub file: AbsolutePathBuf,
+}
+
+#[derive(Debug, Clone)]
+pub(super) struct ManagedConfigFromMdm {
+    pub managed_config: TomlValue,
+    pub raw_toml: String,
+}
+
+#[derive(Debug, Clone)]
+pub(super) struct LoadedConfigLayers {
+    /// If present, data read from a file such as `/etc/codex/managed_config.toml`.
+    pub managed_config: Option<MangedConfigFromFile>,
+    /// If present, data read from managed preferences (macOS only).
+    pub managed_config_from_mdm: Option<ManagedConfigFromMdm>,
+}
+
+pub(super) async fn load_config_layers_internal(
+    fs: &dyn ExecutorFileSystem,
+    codex_home: &Path,
+    overrides: LoaderOverrides,
+) -> io::Result<LoadedConfigLayers> {
+    #[cfg(target_os = "macos")]
+    let LoaderOverrides {
+        managed_config_path,
+        managed_preferences_base64,
+        ..
+    } = overrides;
+
+    #[cfg(not(target_os = "macos"))]
+    let LoaderOverrides {
+        managed_config_path,
+        ..
+    } = overrides;
+
+    let managed_config_path = AbsolutePathBuf::from_absolute_path(
+        managed_config_path.unwrap_or_else(|| managed_config_default_path(codex_home)),
+    )?;
+
+    let managed_config =
+        read_config_from_path(fs, &managed_config_path, /*log_missing_as_info*/ false)
+            .await?
+            .map(|managed_config| MangedConfigFromFile {
+                managed_config,
+                file: managed_config_path.clone(),
+            });
+
+    #[cfg(target_os = "macos")]
+    let managed_preferences =
+        load_managed_admin_config_layer(managed_preferences_base64.as_deref())
+            .await?
+            .map(map_managed_admin_layer);
+
+    #[cfg(not(target_os = "macos"))]
+    let managed_preferences = None;
+
+    Ok(LoadedConfigLayers {
+        managed_config,
+        managed_config_from_mdm: managed_preferences,
+    })
+}
+
+#[cfg(target_os = "macos")]
+fn map_managed_admin_layer(layer: ManagedAdminConfigLayer) -> ManagedConfigFromMdm {
+    let ManagedAdminConfigLayer { config, raw_toml } = layer;
+    ManagedConfigFromMdm {
+        managed_config: config,
+        raw_toml,
+    }
+}
+
+pub(super) async fn read_config_from_path(
+    fs: &dyn ExecutorFileSystem,
+    path: &AbsolutePathBuf,
+    log_missing_as_info: bool,
+) -> io::Result<Option<TomlValue>> {
+    match fs.read_file_text(path, /*sandbox*/ None).await {
+        Ok(contents) => match toml::from_str::<TomlValue>(&contents) {
+            Ok(value) => Ok(Some(value)),
+            Err(err) => {
+                tracing::error!("Failed to parse {}: {err}", path.as_path().display());
+                let config_error = config_error_from_toml(path.as_path(), &contents, err.clone());
+                Err(io_error_from_config_error(
+                    io::ErrorKind::InvalidData,
+                    config_error,
+                    Some(err),
+                ))
+            }
+        },
+        Err(err) if err.kind() == io::ErrorKind::NotFound => {
+            if log_missing_as_info {
+                tracing::info!("{} not found, using defaults", path.as_path().display());
+            } else {
+                tracing::debug!("{} not found", path.as_path().display());
+            }
+            Ok(None)
+        }
+        Err(err) => {
+            tracing::error!("Failed to read {}: {err}", path.as_path().display());
+            Err(err)
+        }
+    }
+}
+
+/// Return the default managed config path.
+pub(super) fn managed_config_default_path(codex_home: &Path) -> PathBuf {
+    #[cfg(unix)]
+    {
+        let _ = codex_home;
+        PathBuf::from(CODEX_MANAGED_CONFIG_SYSTEM_PATH)
+    }
+
+    #[cfg(not(unix))]
+    {
+        codex_home.join("managed_config.toml")
+    }
+}

codex-rs/config/src/loader/macos.rs

@@ -0,0 +1,179 @@
+use super::merge_requirements_with_remote_sandbox_config;
+use crate::config_requirements::ConfigRequirementsToml;
+use crate::config_requirements::ConfigRequirementsWithSources;
+use crate::config_requirements::RequirementSource;
+use base64::Engine;
+use base64::prelude::BASE64_STANDARD;
+use core_foundation::base::TCFType;
+use core_foundation::string::CFString;
+use core_foundation::string::CFStringRef;
+use std::ffi::c_void;
+use std::io;
+use tokio::task;
+use toml::Value as TomlValue;
+
+const MANAGED_PREFERENCES_APPLICATION_ID: &str = "com.openai.codex";
+const MANAGED_PREFERENCES_CONFIG_KEY: &str = "config_toml_base64";
+const MANAGED_PREFERENCES_REQUIREMENTS_KEY: &str = "requirements_toml_base64";
+
+#[derive(Debug, Clone)]
+pub(super) struct ManagedAdminConfigLayer {
+    pub config: TomlValue,
+    pub raw_toml: String,
+}
+
+pub(super) fn managed_preferences_requirements_source() -> RequirementSource {
+    RequirementSource::MdmManagedPreferences {
+        domain: MANAGED_PREFERENCES_APPLICATION_ID.to_string(),
+        key: MANAGED_PREFERENCES_REQUIREMENTS_KEY.to_string(),
+    }
+}
+
+pub(crate) async fn load_managed_admin_config_layer(
+    override_base64: Option<&str>,
+) -> io::Result<Option<ManagedAdminConfigLayer>> {
+    if let Some(encoded) = override_base64 {
+        let trimmed = encoded.trim();
+        return if trimmed.is_empty() {
+            Ok(None)
+        } else {
+            parse_managed_config_base64(trimmed).map(Some)
+        };
+    }
+
+    match task::spawn_blocking(load_managed_admin_config).await {
+        Ok(result) => result,
+        Err(join_err) => {
+            if join_err.is_cancelled() {
+                tracing::error!("Managed config load task was cancelled");
+            } else {
+                tracing::error!("Managed config load task failed: {join_err}");
+            }
+            Err(io::Error::other("Failed to load managed config"))
+        }
+    }
+}
+
+fn load_managed_admin_config() -> io::Result<Option<ManagedAdminConfigLayer>> {
+    load_managed_preference(MANAGED_PREFERENCES_CONFIG_KEY)?
+        .as_deref()
+        .map(str::trim)
+        .map(parse_managed_config_base64)
+        .transpose()
+}
+
+pub(crate) async fn load_managed_admin_requirements_toml(
+    target: &mut ConfigRequirementsWithSources,
+    override_base64: Option<&str>,
+    host_name: Option<&str>,
+) -> io::Result<()> {
+    if let Some(encoded) = override_base64 {
+        let trimmed = encoded.trim();
+        if trimmed.is_empty() {
+            return Ok(());
+        }
+
+        merge_requirements_with_remote_sandbox_config(
+            target,
+            managed_preferences_requirements_source(),
+            parse_managed_requirements_base64(trimmed)?,
+            host_name,
+        );
+        return Ok(());
+    }
+
+    match task::spawn_blocking(load_managed_admin_requirements).await {
+        Ok(result) => {
+            if let Some(requirements) = result? {
+                merge_requirements_with_remote_sandbox_config(
+                    target,
+                    managed_preferences_requirements_source(),
+                    requirements,
+                    host_name,
+                );
+            }
+            Ok(())
+        }
+        Err(join_err) => {
+            if join_err.is_cancelled() {
+                tracing::error!("Managed requirements load task was cancelled");
+            } else {
+                tracing::error!("Managed requirements load task failed: {join_err}");
+            }
+            Err(io::Error::other("Failed to load managed requirements"))
+        }
+    }
+}
+
+fn load_managed_admin_requirements() -> io::Result<Option<ConfigRequirementsToml>> {
+    load_managed_preference(MANAGED_PREFERENCES_REQUIREMENTS_KEY)?
+        .as_deref()
+        .map(str::trim)
+        .map(parse_managed_requirements_base64)
+        .transpose()
+}
+
+fn load_managed_preference(key_name: &str) -> io::Result<Option<String>> {
+    #[link(name = "CoreFoundation", kind = "framework")]
+    unsafe extern "C" {
+        fn CFPreferencesCopyAppValue(key: CFStringRef, application_id: CFStringRef) -> *mut c_void;
+    }
+
+    let value_ref = unsafe {
+        CFPreferencesCopyAppValue(
+            CFString::new(key_name).as_concrete_TypeRef(),
+            CFString::new(MANAGED_PREFERENCES_APPLICATION_ID).as_concrete_TypeRef(),
+        )
+    };
+
+    if value_ref.is_null() {
+        tracing::debug!(
+            "Managed preferences for {MANAGED_PREFERENCES_APPLICATION_ID} key {key_name} not found",
+        );
+        return Ok(None);
+    }
+
+    let value = unsafe { CFString::wrap_under_create_rule(value_ref as _) }.to_string();
+    Ok(Some(value))
+}
+
+fn parse_managed_config_base64(encoded: &str) -> io::Result<ManagedAdminConfigLayer> {
+    let raw_toml = decode_managed_preferences_base64(encoded)?;
+    match toml::from_str::<TomlValue>(&raw_toml) {
+        Ok(TomlValue::Table(parsed)) => Ok(ManagedAdminConfigLayer {
+            config: TomlValue::Table(parsed),
+            raw_toml,
+        }),
+        Ok(other) => {
+            tracing::error!("Managed config TOML must have a table at the root, found {other:?}",);
+            Err(io::Error::new(
+                io::ErrorKind::InvalidData,
+                "managed config root must be a table",
+            ))
+        }
+        Err(err) => {
+            tracing::error!("Failed to parse managed config TOML: {err}");
+            Err(io::Error::new(io::ErrorKind::InvalidData, err))
+        }
+    }
+}
+
+fn parse_managed_requirements_base64(encoded: &str) -> io::Result<ConfigRequirementsToml> {
+    toml::from_str::<ConfigRequirementsToml>(&decode_managed_preferences_base64(encoded)?).map_err(
+        |err| {
+            tracing::error!("Failed to parse managed requirements TOML: {err}");
+            io::Error::new(io::ErrorKind::InvalidData, err)
+        },
+    )
+}
+
+fn decode_managed_preferences_base64(encoded: &str) -> io::Result<String> {
+    String::from_utf8(BASE64_STANDARD.decode(encoded.as_bytes()).map_err(|err| {
+        tracing::error!("Failed to decode managed value as base64: {err}",);
+        io::Error::new(io::ErrorKind::InvalidData, err)
+    })?)
+    .map_err(|err| {
+        tracing::error!("Managed value base64 contents were not valid UTF-8: {err}",);
+        io::Error::new(io::ErrorKind::InvalidData, err)
+    })
+}