Skip to content

fix(types): filter null fields from reasoning output items#22370

Merged
krrishdholakia merged 43 commits intoBerriAI:litellm_oss_staging_02_28_2026from
shivaaang:fix/responses-reasoning-status-null
Feb 28, 2026
Merged

fix(types): filter null fields from reasoning output items#22370
krrishdholakia merged 43 commits intoBerriAI:litellm_oss_staging_02_28_2026from
shivaaang:fix/responses-reasoning-status-null

Conversation

@shivaaang
Copy link
Contributor

@shivaaang shivaaang commented Feb 28, 2026

Relevant issues

Fixes #16824

Pre-Submission checklist

  • I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

Bug Fix

Changes

Problem

When providers return reasoning output items without status, content, or encrypted_content, Pydantic's Optional defaults serialize them as null. This breaks downstream SDKs - for example, the OpenAI C# SDK crashes on status=null in reasoning items.

The request-side already handles this correctly via OpenAIResponsesAPIConfig._handle_reasoning_item(), which filters these fields before sending requests to providers. But the response-side had no equivalent filtering, so null fields leak into serialized responses.

Solution

Added a @field_serializer("output", mode="wrap") on ResponsesAPIResponse that removes null status, content, and encrypted_content from reasoning items during serialization. This mirrors the request-side filtering pattern.

  • Only affects reasoning items (type == "reasoning"); all other output types (messages, function calls, etc.) are untouched
  • Preserves non-null values (e.g., encrypted_content used by GitHub Copilot)
  • Works for both model_dump() and model_dump_json() serialization paths

Files changed

  • litellm/types/llms/openai.py - Added field_serializer import and _serialize_output_filter_reasoning_nulls method on ResponsesAPIResponse
  • tests/test_litellm/types/llms/test_types_llms_openai.py - 7 new tests in TestResponsesAPIReasoningNullFields class

Testing

  • 7 new tests covering null field removal (model_dump and model_dump_json), non-null value preservation, message items unaffected, mixed reasoning + message output, core field preservation, and top-level null fields unaffected
  • All 16 tests in the file pass (9 existing + 7 new)
  • Verified edge cases: empty output list, model_construct(), multiple reasoning items, exclude_none=True, mode="json"

@greptileai @shin-bot-litellm

ZeroClover and others added 30 commits February 25, 2026 01:29
…tion

Add headers parameter to image_generation() and aimage_generation() methods
in OpenAI provider, and pass headers from images/main.py to ensure custom
headers like cf-aig-authorization are properly forwarded to the OpenAI API.
Aligns behavior with completion() method and Azure provider implementation.
Verify that extra_headers are correctly forwarded to OpenAI's
images.generate() in both sync and async paths, and that they
are absent when not provided.
When a gunicorn worker exits (e.g. from max_requests recycling), its
per-process prometheus .db files remain on disk. For gauges using
livesum/liveall mode, this means the dead worker's last-known values
persist as if the process were still alive. Wire gunicorn's child_exit
hook to call mark_process_dead() so live-tracking gauges accurately
reflect only running workers.
…d_exit_cleanup

Add Prometheus child_exit cleanup for gunicorn workers
…ng, and LLM Gateway (BerriAI#21130)

* docs: update AssemblyAI docs with Universal-3 Pro, Speech Understanding, and LLM Gateway provider config

* feat: add AssemblyAI LLM Gateway as OpenAI-compatible provider
…th_info

Tests were mocking the old method name `filter_server_ids_by_ip` but production
code at server.py:774 calls `filter_server_ids_by_ip_with_info` which returns
a (server_ids, blocked_count) tuple. The unmocked method on AsyncMock returned
a coroutine, causing "cannot unpack non-iterable coroutine object" errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ion behavior

Tests were asserting no response.create/conversation.item.create sent to
backend when guardrail blocks, but the implementation intentionally sends
these to have the LLM voice the guardrail violation message to the user.

Updated assertions to verify the correct guardrail flow:
- response.cancel is sent to stop any in-progress response
- conversation.item.create with violation message is injected
- response.create is sent to voice the violation
- original blocked content is NOT forwarded

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The revert in 8565c70 removed the parallel_tool_calls handling from
map_openai_params, and the subsequent fix d0445e1 only re-added the
transform_request consumption but forgot to re-add the map_openai_params
producer that sets _parallel_tool_use_config. This meant parallel_tool_calls
was silently ignored for all Bedrock models.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Commit 99c62ca removed "azure" from _RESPONSES_API_PROVIDERS,
routing Azure models through litellm.completion instead of
litellm.responses. The test was not updated to match, causing it
to assert against the wrong mock.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…erriAI#22319)

* feat: add in_flight_requests metric to /health/backlog + prometheus

* refactor: clean class with static methods, add tests, fix sentinel pattern

* docs: add in_flight_requests to prometheus metrics and latency troubleshooting
PR BerriAI#22271 added the LiteLLM_ClaudeCodePluginTable model to
schema.prisma but did not include a corresponding migration file,
causing test_aaaasschema_migration_check to fail.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Addresses Greptile review feedback.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…test-assertions

fix(test): update realtime guardrail test assertions for voice violation behavior
…gh-azure-test

fix(test): update Azure pass-through test after Responses API routing change
…migration

fix(db): add missing migration for LiteLLM_ClaudeCodePluginTable
…ol-calls-map-params

fix(bedrock): restore parallel_tool_calls mapping in map_openai_params
…ove_key to prevent unawaited coroutine warnings

Fixes BerriAI#22128
…e agents (BerriAI#22329)

* fix: enforce RBAC on agent endpoints — block non-admin create/update/delete

- Add /v1/agents/{agent_id} to agent_routes so internal users can
  access GET-by-ID (previously returned 403 due to missing route pattern)
- Add _check_agent_management_permission() guard to POST, PUT, PATCH,
  DELETE agent endpoints — only PROXY_ADMIN may mutate agents
- Add user_api_key_dict param to delete_agent so the role check works
- Add comprehensive unit tests for RBAC enforcement across all roles

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix: mock prisma_client in internal user get-agent-by-id test

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* feat(ui): hide agent create/delete controls for non-admin users

Match MCP servers pattern: wrap '+ Add New Agent' button in
isAdmin conditional so internal users see a read-only agents view.
Delete buttons in card and table were already gated.
Update empty-state copy for non-admin users.
Add 7 Vitest tests covering role-based visibility.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
…r-method-name

fix(mcp): update test mocks for renamed filter_server_ids_by_ip_with_info
)

* fix: Add PROXY_ADMIN role to system user for key rotation

The key rotation worker was failing with 'You are not authorized to regenerate this key'
when rotating team keys. This was because the system user created by
get_litellm_internal_jobs_user_api_key_auth() was missing the user_role field.

Without user_role=PROXY_ADMIN, the system user couldn't bypass team permission checks
in can_team_member_execute_key_management_endpoint(), causing authorization failures
for team key rotation.

This fix adds user_role=LitellmUserRoles.PROXY_ADMIN to the system user, allowing
it to bypass team permission checks and successfully rotate keys for all teams.

* test: Add unit test for system user PROXY_ADMIN role

- Verify internal jobs system user has PROXY_ADMIN role
- Critical for key rotation to bypass team permission checks
- Regression test for PR BerriAI#21896
…rriAI#22239)

* fix: populate user_id and user_info for admin users in /user/info endpoint

Fixes BerriAI#22179

When admin users call /user/info without a user_id parameter, the endpoint
was returning null for both user_id and user_info fields. This broke
budgeting tooling that relies on /user/info to look up current budget and spend.

Changes:
- Modified _get_user_info_for_proxy_admin() to accept user_api_key_dict parameter
- Added logic to fetch admin's own user info from database
- Updated function to return admin's user_id and user_info instead of null
- Updated unit test to verify admin user_id is populated

The fix ensures admin users get their own user information just like regular users.

* test: make mock get_data signature match real method

- Updated MockPrismaClientDB.get_data() to accept all parameters that the real method accepts
- Makes mock more robust against future refactors
- Added datetime and Union imports
- Mock now returns None when user_id is not provided
…unawaited-coroutine

fix(caching): store task references in LLMClientCache._remove_key
fix(image_generation): propagate extra_headers to Upstream
…onses and chat completions (BerriAI#22291)

* fixed dynamic auth for /responses with mcp

* fixed greptile concern
Fixes BerriAI#18381: When using both tools and response_format with Bedrock
Converse API, LiteLLM internally adds json_tool_call to handle structured
output. Bedrock may return both this internal tool AND real user-defined
tools, breaking consumers like OpenAI Agents SDK.

Changes:
- Non-streaming: Added _filter_json_mode_tools() to handle 3 scenarios:
  only json_tool_call (convert to content), mixed (filter it out), or
  no json_tool_call (pass through)
- Streaming: Added json_mode tracking to AWSEventStreamDecoder to suppress
  json_tool_call chunks and convert to text content
- Fixed optional_params.pop() mutation issue

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Addresses review comment from greptile-apps:
BerriAI#21107 (review)

Changes:
- Added `_unwrap_bedrock_properties()` helper method to eliminate code duplication
- Replaced two identical JSON unwrapping blocks (lines 1592-1601 and 1612-1620)
  with calls to the new helper method
- Improves maintainability - single source of truth for Bedrock properties unwrapping logic

The helper method:
- Parses JSON string
- Checks for single "properties" key structure
- Unwraps and returns the properties value
- Returns original string if unwrapping not needed or parsing fails

No functional changes - pure refactoring.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Fixed MyPy errors where BedrockConverseConfig was used instead of
AmazonConverseConfig in the _unwrap_bedrock_properties() calls.

Errors:
- Line 1619: BedrockConverseConfig -> AmazonConverseConfig
- Line 1631: BedrockConverseConfig -> AmazonConverseConfig

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
…th support

Fixes BerriAI#21941

The generated result filenames from _save_confusion_results contained
parentheses, dots, and full yaml filenames, producing paths that exceed
the Windows 260-char MAX_PATH limit. Rework the safe_label logic to
produce short {topic}_{method_abbrev} filenames (e.g. insults_cf.json)
while preserving the full label inside the JSON content.

Rename existing tracked result files to match the new naming convention.
…r/guardrail_benchmarks/test_eval.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…filenames

fix: shorten guardrail benchmark result filenames for Windows long path support
rasmi and others added 13 commits February 27, 2026 19:33
* fix(mcp): default available_on_public_internet to true

MCPs were defaulting to private (available_on_public_internet=false) which
was a breaking change. This reverts the default to public (true) across:
- Pydantic models (AddMCPServerRequest, UpdateMCPServerRequest, LiteLLM_MCPServerTable)
- Prisma schema @default
- mcp_server_manager.py YAML config + DB loading fallbacks
- UI form initialValue and setFieldValue defaults

* fix(ui): add forceRender to Collapse.Panel so toggle defaults render correctly

Ant Design's Collapse.Panel lazy-renders children by default. Without
forceRender, the Form.Item for 'Available on Public Internet' isn't
mounted when the useEffect fires form.setFieldValue, causing the Switch
to visually show OFF even though the intended default is true.

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* fix(mcp): update remaining schema copies and MCPServer type default to true

Missed in previous commit per Greptile review:
- schema.prisma (root)
- litellm-proxy-extras/litellm_proxy_extras/schema.prisma
- litellm/types/mcp_server/mcp_server_manager.py MCPServer class

* ui(mcp): reframe network access as 'Internal network only' restriction

Replace scary 'Available on Public Internet' toggle with 'Internal network only'
opt-in restriction. Toggle OFF (default) = all networks allowed. Toggle ON =
restricted to internal network only. Auth is always required either way.

- MCPPermissionManagement: new label/tooltip/description, invert display via
  getValueProps/getValueFromEvent so underlying available_on_public_internet
  value is unchanged
- mcp_server_view: 'Public' → 'All networks', 'Internal' → 'Internal only' (orange)
- mcp_server_columns: same badge updates

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
…-tool-call-scoped

fix(bedrock): filter internal json_tool_call when mixed with real tools
…or hints (BerriAI#22336)

* fix(jwt): support OIDC discovery URLs, handle roles array, improve error hints

Three fixes for Azure AD JWT auth:

1. OIDC discovery URL support - JWT_PUBLIC_KEY_URL can now be set to
   .well-known/openid-configuration endpoints. The proxy fetches the
   discovery doc, extracts jwks_uri, and caches it.

2. Handle roles claim as array - when team_id_jwt_field points to a list
   (e.g. AAD's "roles": ["team1"]), auto-unwrap the first element instead
   of crashing with 'unhashable type: list'.

3. Better error hint for dot-notation indexing - when team_id_jwt_field is
   set to "roles.0" or "roles[0]", the 401 error now explains to use
   "roles" instead and that LiteLLM auto-unwraps lists.

* Add integration demo script for JWT auth fixes (OIDC discovery, array roles, dot-notation hints)

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Add demo_servers.py for manual JWT auth testing with mock JWKS/OIDC endpoints

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Add demo screenshots for PR comment

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* Add integration test results with screenshots for PR review

Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>

* address greptile review feedback (greploop iteration 1)

- fix: add HTTP status code check in _resolve_jwks_url before parsing JSON
- fix: remove misleading bracket-notation hint from debug log (get_nested_value does not support it)

* Update tests/test_litellm/proxy/auth/test_handle_jwt.py

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>

* remove demo scripts and assets

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Ishaan Jaff <ishaan-jaff@users.noreply.github.com>
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
…riAI#22346)

* perf: raise aiohttp connection pool limits (300→1000, 50/host→500)

* perf: skip model_copy() on every chunk — only copy usage-bearing chunks

* perf: replace list+join O(n²) with str+= O(n) in async_data_generator

* perf: cache model-level guardrail lookup per request, not per chunk
Add 88 tests across 9 test files for the CostTrackingSettings component directory:
- provider_display_helpers.test.ts: 9 tests for helper functions
- how_it_works.test.tsx: 9 tests for discount calculator component
- add_provider_form.test.tsx: 7 tests for provider form validation
- add_margin_form.test.tsx: 9 tests for margin form with type toggle
- provider_discount_table.test.tsx: 12 tests for table editing and interactions
- provider_margin_table.test.tsx: 13 tests for margin table with sorting
- use_discount_config.test.ts: 11 tests for discount hook logic
- use_margin_config.test.ts: 12 tests for margin hook logic
- cost_tracking_settings.test.tsx: 15 tests for main component and role-based rendering

All tests passing. Coverage includes form validation, user interactions, API calls, state management, and conditional rendering.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
…itest

[Test] UI - CostTrackingSettings: Add comprehensive Vitest coverage
Add filtering capabilities to /key/list endpoint for project_id and access_group_id parameters. Both filters work globally across all visibility rules and stack with existing sort/pagination params. Added comprehensive unit tests for the new filters.

Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Add ProjectDetailsPage with header, details card, spend/budget progress,
  model spend bar chart, keys placeholder, and team info card
- Refactor CreateProjectModal into base form pattern (ProjectBaseForm)
  shared between Create and Edit flows
- Add EditProjectModal with pre-filled form data from backend
- Add useProjectDetails and useUpdateProject hooks
- Add duplicate key validation for model limits and metadata
- Wire project ID click in table to navigate to detail view
- Move pagination inline with search bar

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
[Feature] Key list endpoint: Add project_id and access_group_id filters
…ateProjectModal.tsx

Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
[Feature] UI - Projects: Add Project Details Page
…esAPIResponse

When providers return reasoning items without status/content/encrypted_content,
Pydantic's Optional defaults serialize them as null. This breaks downstream SDKs
(e.g., the OpenAI C# SDK crashes on status=null).

Add a field_serializer on ResponsesAPIResponse.output that removes null
status, content, and encrypted_content from reasoning items during
serialization. This mirrors the request-side filtering already done in
OpenAIResponsesAPIConfig._handle_reasoning_item().

Fixes BerriAI#16824
@vercel
Copy link

vercel bot commented Feb 28, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Feb 28, 2026 7:37am

Request Review

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Feb 28, 2026

Greptile Summary

This PR fixes a downstream SDK compatibility issue (#16824) where null status, content, and encrypted_content fields on reasoning output items in ResponsesAPIResponse caused crashes in SDKs like the OpenAI C# SDK.

  • Adds a @field_serializer("output", mode="wrap") on ResponsesAPIResponse that strips null values for status, content, and encrypted_content from reasoning items (type == "reasoning") during serialization
  • The filtering logic is identical to the existing request-side filtering in OpenAIResponsesAPIConfig._handle_reasoning_item() (fallback path at litellm/llms/openai/responses/transformation.py:146-151), creating symmetry between request and response handling
  • Non-reasoning output items (messages, function calls, etc.) and non-null values on reasoning items are unaffected
  • Includes 7 focused unit tests with no network calls

Confidence Score: 5/5

  • This PR is safe to merge — it's a targeted serialization fix with comprehensive test coverage and no side effects on non-reasoning output items.
  • The change is minimal and well-scoped: a single field_serializer that only affects reasoning items during serialization. The filtering logic is battle-tested (identical to the existing request-side filtering). The 7 new tests cover all critical paths including model_dump, model_dump_json, mixed output, and preservation of non-null values. No existing behavior is altered for non-reasoning items.
  • No files require special attention.

Important Files Changed

Filename Overview
litellm/types/llms/openai.py Adds a field_serializer on the output field of ResponsesAPIResponse to strip null status, content, and encrypted_content from reasoning items during serialization. The logic correctly mirrors the request-side filtering in _handle_reasoning_item().
tests/test_litellm/types/llms/test_types_llms_openai.py Adds 7 well-structured unit tests covering null field removal, non-null preservation, message items unaffected, mixed output, core field preservation, and top-level fields. All tests are mock-only with no network calls.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["ResponsesAPIResponse.model_dump() / model_dump_json()"] --> B["field_serializer('output', mode='wrap')"]
    B --> C["handler(value) — default serialization"]
    C --> D{"Is serialized output a list?"}
    D -- No --> E["Return as-is"]
    D -- Yes --> F["Iterate over items"]
    F --> G{"item is dict AND type == 'reasoning'?"}
    G -- No --> H["Keep item unchanged"]
    G -- Yes --> I["Filter out null status, content, encrypted_content"]
    I --> J["Return cleaned reasoning item"]
    H --> K["Return final output list"]
    J --> K
Loading

Last reviewed commit: 53350c8

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@krrishdholakia krrishdholakia changed the base branch from main to litellm_oss_staging_02_28_2026 February 28, 2026 08:00
@krrishdholakia krrishdholakia merged commit 8b0c3de into BerriAI:litellm_oss_staging_02_28_2026 Feb 28, 2026
26 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Responses API needs a Resoning status not null