Skip to content

Adds Groq support #294

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
masegraye wants to merge 6 commits into
base: main
Choose a base branch
from
Open

Adds Groq support #294

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
513780e
Adds basic Groq support
masegraye Apr 17, 2025
bdc9d17
Doc and make tests conformant
masegraye Apr 17, 2025
a777179
Re-add env-based config
masegraye Apr 17, 2025
d7e5930
Test all supported models
masegraye Apr 17, 2025
1ba7747
Remove unused test server
masegraye Apr 17, 2025
129319e
Add script to interactively test ChatGroq
masegraye Apr 17, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@ Currently supported AI services:
- Anthropic Claude
- Google Gemini
- Google Vertex AI (Google's enterprise offering)
- Groq
- Ollama
- Mistral
- Bumblebee self-hosted models - including Llama, Mistral and Zephyr
Expand Down Expand Up @@ -99,6 +100,7 @@ config :langchain, openai_key: "YOUR SECRET KEY"
config :langchain, openai_org_id: "YOUR_OPENAI_ORG_ID"

config :langchain, :anthropic_key, System.fetch_env!("ANTHROPIC_API_KEY")
config :langchain, :groq_api_key, System.fetch_env!("GROQ_API_KEY")
```

It's possible to use a function or a tuple to resolve the secret:
Expand All @@ -118,6 +120,7 @@ For [fly.io](https://fly.io), adding the secrets looks like this:
```
fly secrets set OPENAI_API_KEY=MyOpenAIApiKey
fly secrets set ANTHROPIC_API_KEY=MyAnthropicApiKey
fly secrets set GROQ_API_KEY=MyGroqApiKey
```

A list of models to use:
Expand All @@ -127,6 +130,7 @@ A list of models to use:
- [OpenAI models](https://platform.openai.com/docs/models)
- [OpenAI models on Azure](https://learn.microsoft.com/en-us/azure/ai-services/openai/concepts/models)
- [Gemini AI models](https://ai.google.dev/gemini-api/docs/models/gemini)
- [Groq models](https://console.groq.com/docs/models)

## Prompt caching

Expand Down Expand Up @@ -258,6 +262,7 @@ mix test --include live_open_ai
mix test --include live_ollama_ai
mix test --include live_anthropic
mix test --include live_mistral_ai
mix test --include live_groq
mix test test/tools/calculator_test.exs --include live_call
```

Expand Down
Loading