Oracle Cloud Generative AI Service API

[federicoquaranta40]

Hello,

what do I need for calling the Oracle Generative AI Service from a python notebook?

Thanks,

Federico

[dosubot[bot]]

To call the Oracle Generative AI Service API from a Python notebook, you need to set up an OCI Gen AI client using the create_client function. Here's a code snippet to help you get started:

import oci
from oci.generative_ai_inference import GenerativeAiInferenceClient

def create_client(auth_type, auth_profile, auth_file_location, service_endpoint):
    client_kwargs = {
        "config": {},
        "signer": None,
        "service_endpoint": service_endpoint,
        "retry_strategy": oci.retry.DEFAULT_RETRY_STRATEGY,
        "timeout": (10, 240),  # default timeout config for OCI Gen AI service
    }

    if auth_type == "API_KEY":
        client_kwargs["config"] = oci.config.from_file(
            file_location=auth_file_location, profile_name=auth_profile
        )
        client_kwargs.pop("signer", None)
    elif auth_type == "SECURITY_TOKEN":
        def make_security_token_signer(oci_config):
            pk = oci.signer.load_private_key_from_file(
                oci_config.get("key_file"), None
            )
            with open(oci_config.get("security_token_file"), encoding="utf-8") as f:
                st_string = f.read()
            return oci.auth.signers.SecurityTokenSigner(st_string, pk)

        client_kwargs["config"] = oci.config.from_file(
            file_location=auth_file_location, profile_name=auth_profile
        )
        client_kwargs["signer"] = make_security_token_signer(
            oci_config=client_kwargs["config"]
        )
    elif auth_type == "INSTANCE_PRINCIPAL":
        client_kwargs["signer"] = oci.auth.signers.InstancePrincipalsSecurityTokenSigner()
    elif auth_type == "RESOURCE_PRINCIPAL":
        client_kwargs["signer"] = oci.auth.signers.get_resource_principals_signer()
    else:
        raise ValueError(f"Please provide valid value to auth_type, {auth_type} is not valid.")

    return GenerativeAiInferenceClient(**client_kwargs)

# Example usage
auth_type = "API_KEY"  # or "SECURITY_TOKEN", "INSTANCE_PRINCIPAL", "RESOURCE_PRINCIPAL"
auth_profile = "DEFAULT"
auth_file_location = "~/.oci/config"
service_endpoint = "https://inference.generativeai.us-chicago-1.oci.oraclecloud.com"

client = create_client(auth_type, auth_profile, auth_file_location, service_endpoint)

Ensure you have the oci Python package installed and configured properly with your OCI credentials. You can install the necessary packages with:

!pip install -U oci

If you are using LlamaIndex with OCI Generative AI, you can also initialize the OCIGenAI interface with your OCI endpoint, model ID, OCID, and authentication method ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other | Bug Report}

[federicoquaranta40]

@dosu i want to call it via Llama index

[dosubot[bot]]

To call the Oracle Generative AI Service API via LlamaIndex, you can use the OCIGenAI class from the llama_index.llms.oci_genai module. Here's how you can do it:

1. Install Required Packages:
   Make sure you have the necessary packages installed:

   pip install llama-index-llms-oci-genai
   pip install -U oci

2. Initialize the OCIGenAI Interface:
   Set up the OCIGenAI interface with your OCI endpoint, model ID, OCID, and authentication method.

   from llama_index.llms.oci_genai import OCIGenAI
   
   llm = OCIGenAI(
       model="MY_MODEL",
       service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",
       compartment_id="MY_OCID",
   )

3. Call the API:
   Use the complete method to send a prompt and receive a response.

   resp = llm.complete("Paul Graham is ")
   print(resp)

   Alternatively, use the chat method with a list of messages:

   from llama_index.core.llms import ChatMessage
   
   messages = [
       ChatMessage(role="system", content="You are a pirate with a colorful personality"),
       ChatMessage(role="user", content="Tell me a story"),
   ]
   
   resp = llm.chat(messages)
   print(resp)

Replace "MY_MODEL", "MY_OCID", and other placeholders with your actual model ID, compartment ID, and other necessary details. Ensure you have the required policies and authentication setup as per the OCI documentation ^[1][2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other | Bug Report}