Feature Request: Lightweight OpenAI-Compatible Proxy (FastAPI + YAML Config)

[mbakkali]

Feature Request: Lightweight OpenAI-Compatible Proxy (FastAPI + YAML Config)

Ce document contient une proposition de fonctionnalité pour transformer Celeste en un proxy robuste pour IDE, inspiré par la simplicité de LiteLLM mais avec la rigueur de typage de Celeste.

---

Issue GitHub (à copier-coller)

Title: [Feature Request] Lightweight OpenAI-Compatible Proxy (FastAPI + YAML Config)

Is your feature request related to a problem? Please describe.

Currently, developers using Celeste with tools like Cursor, Continue.dev, or generic OpenAI clients must write their own boilerplate API wrappers.

While generic proxies like LiteLLM exist, they often require fragile "custom middlewares" (Python scripts) to fix provider-specific API friction. For example, using Gemini with Cursor often requires a manual script to recursively clean tool_choice="validated" or fix malformed tool objects, as generic proxies pass these payloads through blindly.

I want the simplicity of LiteLLM (a single CLI command + a Config File) combined with Celeste's strict validation, where parameter sanitization happens natively "by design" rather than via patch scripts.

Describe the solution you'd like

I propose a new celeste.proxy sub-package that launches a lightweight, OpenAI-compatible server using FastAPI.

It should be usable via a simple CLI command:

celeste serve --config celeste_config.yaml --port 8000

1. Configuration-First Approach (YAML)

Inspired by LiteLLM's config, but typed for Celeste:

models:
  - alias: "cursor-fast"
    provider: "google"
    model: "gemini-2.0-flash-exp"
    parameters:
      temperature: 0.7
  
  - alias: "cursor-smart"
    provider: "anthropic"
    model: "claude-3-5-sonnet-20240620"

2. Native Parameter Sanitization

Instead of requiring user-maintained middleware scripts (like middleware_gemini.py), the proxy should leverage Celeste's internal ParameterMapper and Constraint systems to:

• Automatically map/remove tool_choice fields that providers don't support.
• Validate and fix tool definitions (e.g., ensuring name exists for Anthropic) before the request is sent.
• Reject invalid requests with clear errors (via Pydantic) rather than obscure upstream API 400s.

3. Standardized Endpoints

• POST /v1/chat/completions: Routes requests to the appropriate Celeste Provider.
• GET /v1/models: Returns the list of models defined in the YAML config.
• Streaming Support: Native SSE (Server-Sent Events) support using Celeste's streaming.py.

4. Optional Lightweight UI

A simple Dashboard endpoint (/dashboard) to:

• Visualize loaded models and their configurations.
• View real-time logs of parameter transformation (e.g., "Sanitized tool_choice for Gemini").
• Test connectivity.

Describe alternatives you've considered

• LiteLLM Proxy: Excellent for breadth (100+ models), but for deep integration with complex models (Gemini 2.0, Claude 3.5), it often requires writing and maintaining complex Python middleware hooks (async_pre_call_hook) to fix API inconsistencies.
• Custom FastAPI Wrapper: Forces every team to re-implement the same server logic.

Comparison: Celeste Proxy vs LiteLLM Proxy

Feature	LiteLLM Proxy	Proposed Celeste Proxy
Philosophy	Broad support (100+ providers), "Pass-through" logic.	Deep support, "Correct-by-design" validation.
Configuration	YAML (config.yaml).	YAML (celeste_config.yaml).
Fixing API Bugs	Requires custom Python Middleware scripts.	Native: Handled by internal Type Mappers.
Validation	Loose (relies on upstream errors).	Strict: Validated locally before network call.
Use Case	General Gateway.	Robust Development Proxy (IDE, CI/CD).

Additional context

This feature would make Celeste the "cleanest" way to use advanced AI models in Cursor, removing the need for "hacky" middleware scripts to get tool_choice or multi-modal inputs working correctly.

[Kamilbenkirane]

Thanks @mbakkali — the need is real, but I don’t think it belongs in celeste-ai core.

1. Celeste is an SDK / primitives library; an OpenAI-compatible proxy is a long-running server + CLI + config + ops surface. “OpenAI-compat” also means tracking client quirks (streaming/tool calling/error formats) across IDEs, not just providers.
2. Also, a proxy usually needs a “compat mode” that is more permissive (drop/transform fields to keep IDEs working), whereas Celeste core aims to stay strict/typed-by-default.

That said, I’m open to it as a separate package/repo (e.g. celeste-proxy) or an optional extra (celeste-ai[proxy]) with a tightly-scoped MVP: /v1/chat/completions, /v1/models, SSE streaming, config-first routing.

If you’re up for it, a short design doc or PR proposal would be very welcome.