Skip to content

Commit 0043272

Browse files
Add stream options to Gemini LLM for usage tracking (#4466)
* Add stream options to Gemini LLM for usage tracking * Update Gemini LLM to disable prompt token calculation --------- Co-authored-by: Timothy Carambat <[email protected]>
1 parent d6f0d30 commit 0043272

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

server/utils/AiProviders/gemini/index.js

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -416,9 +416,12 @@ class GeminiLLM {
416416
stream: true,
417417
messages,
418418
temperature: temperature,
419+
stream_options: {
420+
include_usage: true,
421+
},
419422
}),
420423
messages,
421-
true
424+
false
422425
);
423426

424427
return measuredStreamRequest;

0 commit comments

Comments
 (0)