fix(router): sync model name changes across replicas#22290
fix(router): sync model name changes across replicas#22290muraliavarma wants to merge 2 commits intoBerriAI:mainfrom
Conversation
upsert_deployment() only compared litellm_params when deciding whether
to update an existing deployment. If only the model_name was changed
(e.g. via PATCH /model/{id}/update), the polling job on other replicas
would see matching litellm_params and skip the update entirely. This
caused model name changes to never propagate to other pods — only a
full restart would fix it.
Add model_name to the equality check so name-only changes are detected
and applied during the 30s polling cycle.
Fixes BerriAI#22190
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryFixes a bug where model name changes via
Confidence Score: 5/5
|
| Filename | Overview |
|---|---|
| litellm/router.py | Adds model_name comparison to the equality check in upsert_deployment() so that name-only changes are detected during the 30s polling cycle. Correct and minimal fix. One stale comment noted. |
| tests/litellm/test_router_upsert_deployment.py | New test file with three well-structured regression tests covering: model_name change detection, no-op skip when nothing changed, and existing litellm_params change behavior. |
Sequence Diagram
sequenceDiagram
participant User as Admin User
participant API as PATCH /model/{id}/update
participant DB as Database
participant R1 as Replica 1 (Router)
participant R2 as Replica 2 (Router)
User->>API: Update model_name only
API->>DB: Save updated model_name
API->>R1: Direct update on handling replica
Note over R2: 30s polling cycle
R2->>DB: Fetch all models
DB-->>R2: Returns model with new name
alt Before fix
R2->>R2: Compare litellm_params only
R2->>R2: Params match → skip update ❌
Note over R2: Name change lost!
end
alt After fix
R2->>R2: Compare litellm_params AND model_name
R2->>R2: Name differs → upsert deployment ✅
Note over R2: Name change propagated!
end
Last reviewed commit: 577a165
Additional Comments (1)
|
Relevant issues
Fixes #22190
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewCI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🐛 Bug Fix
Changes
upsert_deployment()inrouter.pyonly comparedlitellm_paramswhen deciding whether to update an existing deployment during the 30s polling cycle. If onlymodel_namewas changed (e.g. viaPATCH /model/{id}/update), the comparison returned equal and the update was skipped. This caused model name changes to never propagate to other replicas — only a full restart would fix it.Fix: Added
model_nameto the equality check inupsert_deployment()so that name-only changes are detected and the deployment is replaced during the next poll.Tests added (
tests/litellm/test_router_upsert_deployment.py):test_upsert_deployment_detects_model_name_change— verifies upsert updates when only model_name changestest_upsert_deployment_skips_when_nothing_changed— verifies no unnecessary updates when nothing changedtest_upsert_deployment_detects_litellm_params_change— verifies existing behavior (param changes) is preserved