[Codex][CLI] Gate image inputs by model modalities (#10271)

###### Summary

- Add input_modalities to model metadata so clients can determine
supported input types.
- Gate image paste/attach in TUI when the selected model does not
support images.
- Block submits that include images for unsupported models and show a
clear warning.
- Propagate modality metadata through app-server protocol/model-list
responses.
  - Update related tests/fixtures.

  ###### Rationale

  - Models support different input modalities.
- Clients need an explicit capability signal to prevent unsupported
requests.
- Backward-compatible defaults preserve existing behavior when modality
metadata is absent.

  ###### Scope

  - codex-rs/protocol, codex-rs/core, codex-rs/tui
  - codex-rs/app-server-protocol, codex-rs/app-server
  - Generated app-server types / schema fixtures

  ###### Trade-offs

- Default behavior assumes text + image when field is absent for
compatibility.
  - Server-side validation remains the source of truth.

  ###### Follow-up

- Non-TUI clients should consume input_modalities to disable unsupported
attachments.
- Model catalogs should explicitly set input_modalities for text-only
models.

  ###### Testing

  - cargo fmt --all
  - cargo test -p codex-tui
  - env -u GITHUB_APP_KEY cargo test -p codex-core --lib
  - just write-app-server-schema
- cargo run -p codex-cli --bin codex -- app-server generate-ts --out
app-server-types
  - test against local backend
  
<img width="695" height="199" alt="image"
src="https://github.com/user-attachments/assets/d22dd04f-5eba-4db9-a7c5-a2506f60ec44"
/>

---------

Co-authored-by: Josh McKinney <joshka@openai.com>
This commit is contained in:
Colin Young
2026-02-02 18:56:39 -08:00
committed by GitHub
parent b8addcddb9
commit 7e07ec8f73
23 changed files with 373 additions and 3 deletions

View File

@@ -209,6 +209,14 @@ impl BottomPane {
self.request_redraw();
}
/// Update image-paste behavior for the active composer and repaint immediately.
///
/// Callers use this to keep composer affordances aligned with model capabilities.
pub fn set_image_paste_enabled(&mut self, enabled: bool) {
self.composer.set_image_paste_enabled(enabled);
self.request_redraw();
}
pub fn set_connectors_snapshot(&mut self, snapshot: Option<ConnectorsSnapshot>) {
self.composer.set_connector_mentions(snapshot);
self.request_redraw();
@@ -402,6 +410,10 @@ impl BottomPane {
}
/// Replace the composer text with `text`.
///
/// This is intended for fresh input where mention linkage does not need to
/// survive; it routes to `ChatComposer::set_text_content`, which resets
/// `mention_paths`.
pub(crate) fn set_composer_text(
&mut self,
text: String,
@@ -414,6 +426,27 @@ impl BottomPane {
self.request_redraw();
}
/// Replace the composer text while preserving mention link targets.
///
/// Use this when rehydrating a draft after a local validation/gating
/// failure (for example unsupported image submit) so previously selected
/// mention targets remain stable across retry.
pub(crate) fn set_composer_text_with_mention_paths(
&mut self,
text: String,
text_elements: Vec<TextElement>,
local_image_paths: Vec<PathBuf>,
mention_paths: HashMap<String, String>,
) {
self.composer.set_text_content_with_mention_paths(
text,
text_elements,
local_image_paths,
mention_paths,
);
self.request_redraw();
}
#[allow(dead_code)]
pub(crate) fn set_composer_input_enabled(
&mut self,