Skip to content

fix(router): sync model name changes across replicas#22290

Open
muraliavarma wants to merge 2 commits intoBerriAI:mainfrom
muraliavarma:fix/upsert-deployment-model-name-sync
Open

fix(router): sync model name changes across replicas#22290
muraliavarma wants to merge 2 commits intoBerriAI:mainfrom
muraliavarma:fix/upsert-deployment-model-name-sync

Conversation

@muraliavarma
Copy link
Contributor

Relevant issues

Fixes #22190

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🐛 Bug Fix

Changes

upsert_deployment() in router.py only compared litellm_params when deciding whether to update an existing deployment during the 30s polling cycle. If only model_name was changed (e.g. via PATCH /model/{id}/update), the comparison returned equal and the update was skipped. This caused model name changes to never propagate to other replicas — only a full restart would fix it.

Fix: Added model_name to the equality check in upsert_deployment() so that name-only changes are detected and the deployment is replaced during the next poll.

Tests added (tests/litellm/test_router_upsert_deployment.py):

  • test_upsert_deployment_detects_model_name_change — verifies upsert updates when only model_name changes
  • test_upsert_deployment_skips_when_nothing_changed — verifies no unnecessary updates when nothing changed
  • test_upsert_deployment_detects_litellm_params_change — verifies existing behavior (param changes) is preserved

upsert_deployment() only compared litellm_params when deciding whether
to update an existing deployment. If only the model_name was changed
(e.g. via PATCH /model/{id}/update), the polling job on other replicas
would see matching litellm_params and skip the update entirely. This
caused model name changes to never propagate to other pods — only a
full restart would fix it.

Add model_name to the equality check so name-only changes are detected
and applied during the 30s polling cycle.

Fixes BerriAI#22190
@vercel
Copy link

vercel bot commented Feb 27, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 27, 2026 3:54pm

Request Review

@muraliavarma
Copy link
Contributor Author

@greptileai

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 27, 2026

Greptile Summary

Fixes a bug where model name changes via PATCH /model/{id}/update were not propagated across proxy replicas. The upsert_deployment() method only compared litellm_params when deciding whether to update during the 30-second polling cycle — this meant name-only changes were silently skipped. The fix adds model_name to the equality check, which is both necessary and sufficient since pricing fields are already synced through litellm_params and comparing model_info timestamps would cause unnecessary churn.

  • Bug fix: Added model_name comparison to the no-op guard in Router.upsert_deployment() (litellm/router.py:6771-6776)
  • Tests: 3 new regression tests covering name change detection, no-op skip, and existing params-change behavior
  • No custom rule violations detected

Confidence Score: 5/5

  • This PR is safe to merge — it's a minimal, well-tested bug fix with no risk of regression.
  • The change is a single additional condition in an equality check. It correctly fixes the reported issue ([Bug]: Config changes not syncing across 3 replicas with Redis #22190), has comprehensive test coverage for all three comparison scenarios, does not affect the critical request path, and introduces no new dependencies or architectural changes.
  • No files require special attention.

Important Files Changed

Filename Overview
litellm/router.py Adds model_name comparison to the equality check in upsert_deployment() so that name-only changes are detected during the 30s polling cycle. Correct and minimal fix. One stale comment noted.
tests/litellm/test_router_upsert_deployment.py New test file with three well-structured regression tests covering: model_name change detection, no-op skip when nothing changed, and existing litellm_params change behavior.

Sequence Diagram

sequenceDiagram
    participant User as Admin User
    participant API as PATCH /model/{id}/update
    participant DB as Database
    participant R1 as Replica 1 (Router)
    participant R2 as Replica 2 (Router)

    User->>API: Update model_name only
    API->>DB: Save updated model_name
    API->>R1: Direct update on handling replica

    Note over R2: 30s polling cycle
    R2->>DB: Fetch all models
    DB-->>R2: Returns model with new name

    alt Before fix
        R2->>R2: Compare litellm_params only
        R2->>R2: Params match → skip update ❌
        Note over R2: Name change lost!
    end

    alt After fix
        R2->>R2: Compare litellm_params AND model_name
        R2->>R2: Name differs → upsert deployment ✅
        Note over R2: Name change propagated!
    end
Loading

Last reviewed commit: 577a165

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 27, 2026

Additional Comments (1)

litellm/router.py
Stale comment after logic change
This comment says "if there is a new litellm param" but the condition above now also triggers on model_name changes. Consider updating it to reflect the broader scope.

                # if there is a new litellm param or model name -> then update the deployment

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Config changes not syncing across 3 replicas with Redis

1 participant