-
Notifications
You must be signed in to change notification settings - Fork 618
Open
Labels
Description
Feature Request
Use the gRPC API for the Rig model providers that expose gRPC APIs (currently Google (Gemini Direct API) and xAI)
Docs + proto sources:
- Google Gemini Direct API (REST/gRPC) docs:
- Google proto source (Gemini / Generative Language API):
- xAI gRPC docs + proto sources:
Motivation
- Protobuf is strongly typed, more performant to de/serialize than JSON, smaller than JSON, easy to generate via the
toniccrate and backwards compatible
Proposal
- Use the gRPC backend for the existing
geminiandxaiproviders instead of the current REST backend - Implementation outline:
- Use
tonic+prost(or equivalent) to generate clients from the upstream protos. - Google:
- Generate a client for
google.ai.generativelanguage.v1beta.GenerativeServicefrom the googleapis proto. - Attach
x-goog-api-key(and optionallyx-goog-api-clientfor identification) metadata headers.
- Generate a client for
- xAI:
- Generate clients from
xai-org/xai-protoand connect toapi.x.aiwithAuthorization: Bearer <API_KEY>metadata.
- Generate clients from
- Map Rig’s existing request/response types to/from the proto messages; preserve existing streaming APIs (
stream_prompt, etc.) by adapting gRPC streaming responses into Rig’s stream types.
- Use
hghalebi, ertyurk and dxvid-pts