Skip to content

feat: Use gRPC APIs for Gemini + xAI #1201

@dxvid-pts

Description

@dxvid-pts

Feature Request

Use the gRPC API for the Rig model providers that expose gRPC APIs (currently Google (Gemini Direct API) and xAI)

Docs + proto sources:

Motivation

  • Protobuf is strongly typed, more performant to de/serialize than JSON, smaller than JSON, easy to generate via the tonic crate and backwards compatible

Proposal

  • Use the gRPC backend for the existing gemini and xai providers instead of the current REST backend
  • Implementation outline:
    • Use tonic + prost (or equivalent) to generate clients from the upstream protos.
    • Google:
      • Generate a client for google.ai.generativelanguage.v1beta.GenerativeService from the googleapis proto.
      • Attach x-goog-api-key (and optionally x-goog-api-client for identification) metadata headers.
    • xAI:
      • Generate clients from xai-org/xai-proto and connect to api.x.ai with Authorization: Bearer <API_KEY> metadata.
    • Map Rig’s existing request/response types to/from the proto messages; preserve existing streaming APIs (stream_prompt, etc.) by adapting gRPC streaming responses into Rig’s stream types.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions