Add structured output with tools support for models with limited structured output

docs/models/litellm.md

@@ -1,90 +1,125 @@
-# Using any model via LiteLLM
-
-!!! note
-
-    The LiteLLM integration is in beta. You may run into issues with some model providers, especially smaller ones. Please report any issues via [Github issues](https://github.com/openai/openai-agents-python/issues) and we'll fix quickly.
-
-[LiteLLM](https://docs.litellm.ai/docs/) is a library that allows you to use 100+ models via a single interface. We've added a LiteLLM integration to allow you to use any AI model in the Agents SDK.
-
-## Setup
-
-You'll need to ensure `litellm` is available. You can do this by installing the optional `litellm` dependency group:
-
-```bash
-pip install "openai-agents[litellm]"
-```
-
-Once done, you can use [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel] in any agent.
-
-## Example
-
-This is a fully working example. When you run it, you'll be prompted for a model name and API key. For example, you could enter:
-
--   `openai/gpt-4.1` for the model, and your OpenAI API key
--   `anthropic/claude-3-5-sonnet-20240620` for the model, and your Anthropic API key
--   etc
-
-For a full list of models supported in LiteLLM, see the [litellm providers docs](https://docs.litellm.ai/docs/providers).
-
-```python
-from __future__ import annotations
-
-import asyncio
-
-from agents import Agent, Runner, function_tool, set_tracing_disabled
-from agents.extensions.models.litellm_model import LitellmModel
-
-@function_tool
-def get_weather(city: str):
-    print(f"[debug] getting weather for {city}")
-    return f"The weather in {city} is sunny."
-
-
-async def main(model: str, api_key: str):
-    agent = Agent(
-        name="Assistant",
-        instructions="You only respond in haikus.",
-        model=LitellmModel(model=model, api_key=api_key),
-        tools=[get_weather],
-    )
-
-    result = await Runner.run(agent, "What's the weather in Tokyo?")
-    print(result.final_output)
-
-
-if __name__ == "__main__":
-    # First try to get model/api key from args
-    import argparse
-
-    parser = argparse.ArgumentParser()
-    parser.add_argument("--model", type=str, required=False)
-    parser.add_argument("--api-key", type=str, required=False)
-    args = parser.parse_args()
-
-    model = args.model
-    if not model:
-        model = input("Enter a model name for Litellm: ")
-
-    api_key = args.api_key
-    if not api_key:
-        api_key = input("Enter an API key for Litellm: ")
-
-    asyncio.run(main(model, api_key))
-```
-
-## Tracking usage data
-
-If you want LiteLLM responses to populate the Agents SDK usage metrics, pass `ModelSettings(include_usage=True)` when creating your agent.
-
-```python
-from agents import Agent, ModelSettings
-from agents.extensions.models.litellm_model import LitellmModel
-
-agent = Agent(
-    name="Assistant",
-    model=LitellmModel(model="your/model", api_key="..."),
-    model_settings=ModelSettings(include_usage=True),
-)
-```
-
-With `include_usage=True`, LiteLLM requests report token and request counts through `result.context_wrapper.usage` just like the built-in OpenAI models.
+# Using any model via LiteLLM
+
+!!! note
+
+    The LiteLLM integration is in beta. You may run into issues with some model providers, especially smaller ones. Please report any issues via [Github issues](https://github.com/openai/openai-agents-python/issues) and we'll fix quickly.
+
+[LiteLLM](https://docs.litellm.ai/docs/) is a library that allows you to use 100+ models via a single interface. We've added a LiteLLM integration to allow you to use any AI model in the Agents SDK.
+
+## Setup
+
+You'll need to ensure `litellm` is available. You can do this by installing the optional `litellm` dependency group:
+
+```bash
+pip install "openai-agents[litellm]"
+```
+
+Once done, you can use [`LitellmModel`][agents.extensions.models.litellm_model.LitellmModel] in any agent.
+
+## Example
+
+This is a fully working example. When you run it, you'll be prompted for a model name and API key. For example, you could enter:
+
+-   `openai/gpt-4.1` for the model, and your OpenAI API key
+-   `anthropic/claude-3-5-sonnet-20240620` for the model, and your Anthropic API key
+-   etc
+
+For a full list of models supported in LiteLLM, see the [litellm providers docs](https://docs.litellm.ai/docs/providers).
+
+```python
+from __future__ import annotations
+
+import asyncio
+
+from agents import Agent, Runner, function_tool, set_tracing_disabled
+from agents.extensions.models.litellm_model import LitellmModel
+
+@function_tool
+def get_weather(city: str):
+    print(f"[debug] getting weather for {city}")
+    return f"The weather in {city} is sunny."
+
+
+async def main(model: str, api_key: str):
+    agent = Agent(
+        name="Assistant",
+        instructions="You only respond in haikus.",
+        model=LitellmModel(model=model, api_key=api_key),
+        tools=[get_weather],
+    )
+
+    result = await Runner.run(agent, "What's the weather in Tokyo?")
+    print(result.final_output)
+
+
+if __name__ == "__main__":
+    # First try to get model/api key from args
+    import argparse
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--model", type=str, required=False)
+    parser.add_argument("--api-key", type=str, required=False)
+    args = parser.parse_args()
+
+    model = args.model
+    if not model:
+        model = input("Enter a model name for Litellm: ")
+
+    api_key = args.api_key
+    if not api_key:
+        api_key = input("Enter an API key for Litellm: ")
+
+    asyncio.run(main(model, api_key))
+```
+
+## Tracking usage data
+
+If you want LiteLLM responses to populate the Agents SDK usage metrics, pass `ModelSettings(include_usage=True)` when creating your agent.
+
+```python
+from agents import Agent, ModelSettings
+from agents.extensions.models.litellm_model import LitellmModel
+
+agent = Agent(
+    name="Assistant",
+    model=LitellmModel(model="your/model", api_key="..."),
+    model_settings=ModelSettings(include_usage=True),
+)
+```
+
+With `include_usage=True`, LiteLLM requests report token and request counts through `result.context_wrapper.usage` just like the built-in OpenAI models.
+
+## Using tools with structured outputs
+
+Some models accessed via LiteLLM (particularly Google Gemini) don't natively support using tools and structured outputs simultaneously. For these models, enable prompt injection:
+
+```python
+from pydantic import BaseModel
+from agents import Agent, function_tool
+from agents.extensions.models.litellm_model import LitellmModel
+
+
+class Report(BaseModel):
+    summary: str
+    confidence: float
+
+
+@function_tool
+def analyze_data(query: str) -> dict:
+    return {"result": f"Analysis of {query}"}
+
+
+agent = Agent(
+    name="Analyst",
+    model=LitellmModel(
+        "gemini/gemini-2.5-flash",
+        enable_structured_output_with_tools=True,  # Required for Gemini
+    ),
+    tools=[analyze_data],
+    output_type=Report,
+)
+```
+
+The `enable_structured_output_with_tools` parameter on `LitellmModel` enables a workaround that injects JSON formatting instructions into the system prompt instead of using the native API. This allows models like Gemini to return structured outputs even when using tools.
+
+See the [prompt injection documentation](structured_output_with_tools.md) for complete details.