extract models manager and related ownership from core (#16508)

## Summary
- split `models-manager` out of `core` and add `ModelsManagerConfig`
plus `Config::to_models_manager_config()` so model metadata paths stop
depending on `core::Config`
- move login-owned/auth-owned code out of `core` into `codex-login`,
move model provider config into `codex-model-provider-info`, move API
bridge mapping into `codex-api`, move protocol-owned types/impls into
`codex-protocol`, and move response debug helpers into a dedicated
`response-debug-context` crate
- move feedback tag emission into `codex-feedback`, relocate tests to
the crates that now own the code, and keep broad temporary re-exports so
this PR avoids a giant import-only rewrite

## Major moves and decisions
- created `codex-models-manager` as the owner for model
cache/catalog/config/model info logic, including the new
`ModelsManagerConfig` struct
- created `codex-model-provider-info` as the owner for provider config
parsing/defaults and kept temporary `codex-login`/`codex-core`
re-exports for old import paths
- moved `api_bridge` error mapping + `CoreAuthProvider` into
`codex-api`, while `codex-login::api_bridge` temporarily re-exports
those symbols and keeps the `auth_provider_from_auth` wrapper
- moved `auth_env_telemetry` and `provider_auth` ownership to
`codex-login`
- moved `CodexErr` ownership to `codex-protocol::error`, plus
`StreamOutput`, `bytes_to_string_smart`, and network policy helpers to
protocol-owned modules
- created `codex-response-debug-context` for
`extract_response_debug_context`, `telemetry_transport_error_message`,
and related response-debug plumbing instead of leaving that behavior in
`core`
- moved `FeedbackRequestTags`, `emit_feedback_request_tags`, and
`emit_feedback_request_tags_with_auth_env` to `codex-feedback`
- deferred removal of temporary re-exports and the mechanical import
rewrites to a stacked follow-up PR so this PR stays reviewable

## Test moves
- moved auth refresh coverage from `core/tests/suite/auth_refresh.rs` to
`login/tests/suite/auth_refresh.rs`
- moved text encoding coverage from
`core/tests/suite/text_encoding_fix.rs` to
`protocol/src/exec_output_tests.rs`
- moved model info override coverage from
`core/tests/suite/model_info_overrides.rs` to
`models-manager/src/model_info_overrides_tests.rs`

---------

Co-authored-by: Codex <noreply@openai.com>
This commit is contained in:
Ahmed Ibrahim
2026-04-02 23:00:02 -07:00
committed by GitHub
parent 8cd7f20b48
commit 6fff9955f1
99 changed files with 1095 additions and 1137 deletions

View File

@@ -0,0 +1,77 @@
//! Integration test for the text encoding fix for issue #6178.
//!
//! These tests simulate VSCode's shell preview on Windows/WSL where the output
//! may be encoded with a legacy code page before it reaches Codex.
use super::StreamOutput;
use pretty_assertions::assert_eq;
#[test]
fn test_utf8_shell_output() {
// Baseline: UTF-8 output should bypass the detector and remain unchanged.
assert_eq!(decode_shell_output("пример".as_bytes()), "пример");
}
#[test]
fn test_cp1251_shell_output() {
// VS Code shells on Windows frequently surface CP1251 bytes for Cyrillic text.
assert_eq!(decode_shell_output(b"\xEF\xF0\xE8\xEC\xE5\xF0"), "пример");
}
#[test]
fn test_cp866_shell_output() {
// Native cmd.exe still defaults to CP866; make sure we recognize that too.
assert_eq!(decode_shell_output(b"\xAF\xE0\xA8\xAC\xA5\xE0"), "пример");
}
#[test]
fn test_windows_1252_smart_decoding() {
// Smart detection should turn fancy quotes/dashes into the proper Unicode glyphs.
assert_eq!(
decode_shell_output(b"\x93\x94 test \x96 dash"),
"\u{201C}\u{201D} test \u{2013} dash"
);
}
#[test]
fn test_smart_decoding_improves_over_lossy_utf8() {
// Regression guard: String::from_utf8_lossy() alone used to emit replacement chars here.
let bytes = b"\x93\x94 test \x96 dash";
assert!(
String::from_utf8_lossy(bytes).contains('\u{FFFD}'),
"lossy UTF-8 should inject replacement chars"
);
assert_eq!(
decode_shell_output(bytes),
"\u{201C}\u{201D} test \u{2013} dash",
"smart decoding should keep curly quotes intact"
);
}
#[test]
fn test_mixed_ascii_and_legacy_encoding() {
// Commands tend to mix ASCII status text with Latin-1 bytes (e.g. café).
assert_eq!(decode_shell_output(b"Output: caf\xE9"), "Output: café"); // codespell:ignore caf
}
#[test]
fn test_pure_latin1_shell_output() {
// Latin-1 by itself should still decode correctly (regression coverage for the older tests).
assert_eq!(decode_shell_output(b"caf\xE9"), "café"); // codespell:ignore caf
}
#[test]
fn test_invalid_bytes_still_fall_back_to_lossy() {
// If detection fails, we still want the user to see replacement characters.
let bytes = b"\xFF\xFE\xFD";
assert_eq!(decode_shell_output(bytes), String::from_utf8_lossy(bytes));
}
fn decode_shell_output(bytes: &[u8]) -> String {
StreamOutput {
text: bytes.to_vec(),
truncated_after_lines: None,
}
.from_utf8_lossy()
.text
}