Skip to content

Conversation

@devin-ai-integration
Copy link
Contributor

Fix Bedrock cross-region inference profile support (Issue #3791)

Summary

Fixes issue #3791 where Bedrock cross-region inference profiles stopped working after version 1.20.0. The issue occurred because the model_id parameter was being ignored in BedrockCompletion.__init__().

Changes:

  • Modified BedrockCompletion.__init__() to extract and use the model_id parameter from kwargs when provided
  • When model_id is specified, it's used for API calls instead of the base model name
  • This allows users to specify ARNs for cross-region inference profiles while keeping the base model name for internal identification
  • Added comprehensive tests covering cross-region inference profile usage
  • Fixed test fixture to properly import module before patching

Example usage:

llm = LLM(
    model="bedrock/anthropic.claude-sonnet-4-20250514-v1:0",
    model_id="arn:aws:bedrock:us-east-1:123456789012:inference-profile/us.anthropic.claude-sonnet-4-20250514-v1:0",
    temperature=0.3,
    max_tokens=4000,
)

Review & Testing Checklist for Human

  • CRITICAL: Review the uv.lock changes (4000+ lines). These appear to be from regenerating the lock file and include many dependency version bumps that are unrelated to the fix. Consider reverting these and regenerating separately if needed, or verify they don't introduce breaking changes.
  • Verify backward compatibility: Test that existing Bedrock usage without model_id parameter still works correctly
  • Test with actual AWS Bedrock cross-region inference profile (if possible) to verify the fix works with real API calls, not just mocks
  • Review the fix against the exact configuration mentioned in issue [BUG] LLMs using Bedrock Cross Region Inferencing break with 1.20.0. Works fine with 0.193.2. #3791 to ensure it solves the reported problem

Notes

devin-ai-integration bot and others added 2 commits October 24, 2025 18:21
- Add support for model_id parameter in BedrockCompletion
- When model_id is provided, use it instead of model for API calls
- This fixes issue #3791 where cross-region inference profiles
  (which require ARN as model_id) were not working
- Add comprehensive tests for cross-region inference profiles
- All existing bedrock tests pass

Co-Authored-By: João <[email protected]>
@devin-ai-integration
Copy link
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant