Skip to content

Conversation

@jamesbraza
Copy link
Collaborator

@jamesbraza jamesbraza commented Oct 23, 2025

Ensuring reasoning effort is passed as expected, with tests.

With litellm<1.75.6 and GPT-5 reasoning effort, you will get blown up:

E               litellm.exceptions.UnsupportedParamsError: litellm.UnsupportedParamsError: openai does not support parameters: ['reasoning_effort'], for model=gpt-5-2025-08-07. To drop these, set `litellm.drop_params=True` or for proxy:
E               
E               `litellm_settings:
E                drop_params: true`
E               . 
E                If you want to use these params dynamically send allowed_openai_params=['reasoning_effort'] in your request.. Received Model Group=gpt-5-2025-08-07
E               Available Model Group Fallbacks=None LiteLLM Retried: 2 times, LiteLLM Max Retries: 3

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR updates the minimum version of litellm from 1.63.5 to 1.75.6 to ensure proper OpenAI reasoning effort support. The test suite has been refactored to validate reasoning effort functionality across both OpenAI (GPT-5) and Anthropic (Claude 3.7 Sonnet) models with different effort levels.

  • Refactored test from a single Anthropic-specific test to a parameterized test covering multiple models and effort levels
  • Added validation for reasoning effort by checking response length ranges and verifying provider-specific reasoning content
  • Added VCR cassettes capturing API responses for parameterized test scenarios

Reviewed Changes

Copilot reviewed 7 out of 8 changed files in this pull request and generated no comments.

File Description
packages/lmi/tests/test_llms.py Refactored test_anthropic_model to test_reasoning_effort with parameterized model and effort level testing
packages/lmi/pyproject.toml Updated minimum litellm version from 1.63.5 to 1.75.6 for reasoning effort support
packages/lmi/tests/cassettes/*.yaml Added VCR cassettes for parameterized test scenarios, deleted old cassette

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Copy link
Collaborator

@sidnarayanan sidnarayanan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice

@jamesbraza jamesbraza merged commit dc4bb3b into main Oct 23, 2025
7 checks passed
@jamesbraza jamesbraza deleted the reasoning-effort branch October 23, 2025 19:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants