Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion site/docs/bindings/node-js.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description: Node.js bindings provide JavaScript/TypeScript API.
OpenVINO GenAI provides Node.js bindings that enable you to use generative AI pipelines in JavaScript and TypeScript applications.

:::warning API Coverage
Node.js bindings currently provide a subset of the full OpenVINO GenAI API available in C++ and Python. The focus is on core text generation (`LLMPipeline`) and text embedding (`TextEmbeddingPipeline`) functionality.
Node.js bindings currently provide a subset of the full OpenVINO GenAI API available in C++ and Python. The focus is on core text generation (`LLMPipeline`), vision language models (`VLMPipeline`), and text embedding (`TextEmbeddingPipeline`) functionality.
:::

## Supported Pipelines and Features
Expand All @@ -23,6 +23,10 @@ Node.js bindings currently support:
- Multiple sampling strategies (greedy, beam search)
- Structured output
- ReAct agent support
- `VLMPipeline`: Vision Language Model inference for multimodal tasks
- Process images and videos with text prompts
- Chat mode with conversation history
- Streaming support
- `TextEmbeddingPipeline`: Generate text embeddings for semantic search and RAG applications
- `Tokenizer`: Fast tokenization / detokenization and chat prompt formatting
- Encode strings into token id and attention mask tensors
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import CodeBlock from '@theme/CodeBlock';

<CodeBlock language="javascript" showLineNumbers>
{`import { addon as ov } from "openvino-node";
import { VLMPipeline } from "openvino-genai-node";
import { stat, readdir } from "node:fs/promises";
import sharp from "sharp";
import path from "node:path";

async function readImage(imagePath) {
const img = sharp(imagePath);
const metadata = await img.metadata();
const { width, height, channels } = metadata;
const imageBuffer = await img.raw().toBuffer();
return new ov.Tensor(ov.element.u8, [height, width, channels], imageBuffer);
}

async function readImages(imagePath) {
const stats = await stat(imagePath);
if (stats.isDirectory()) {
const files = await readdir(imagePath);
return Promise.all(files.sort().map((file) => readImage(path.join(imagePath, file))));
}
return [await readImage(imagePath)];
}

const images = await readImages("./images");

const pipe = await VLMPipeline(modelPath, "${props.device || 'CPU'}");

const result = await pipe.generate(prompt, {
images,
generationConfig: { max_new_tokens: 100 },
});
console.log(result.texts[0]);

// To input videos frames, use 'videos' option, frames tensor layout = [Frame, H, W, C]
// const result = await pipe.generate(prompt, { videos: [frames], generationConfig: { max_new_tokens: 100 } });
`}
</CodeBlock>
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import CodeExampleCPP from './_code_example_cpp.mdx';
import CodeExamplePython from './_code_example_python.mdx';
import CodeExampleJS from './_code_example_js.mdx';

## Run Model Using OpenVINO GenAI

Expand Down Expand Up @@ -27,6 +28,16 @@ It can generate text from a text prompt and images as inputs.
</TabItem>
</Tabs>
</TabItemCpp>
<TabItemJS>
<Tabs groupId="device">
<TabItem label="CPU" value="cpu">
<CodeExampleJS device="CPU" />
</TabItem>
<TabItem label="GPU" value="gpu">
<CodeExampleJS device="GPU" />
</TabItem>
</Tabs>
</TabItemJS>
</LanguageTabs>

:::tip
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,28 @@ Similar to [text generation](/docs/use-cases/text-generation/#use-different-gene
}
```
</TabItemCpp>
<TabItemJS>
```javascript
import { VLMPipeline } from 'openvino-genai-node';

const pipe = await VLMPipeline(modelPath, "CPU", {});

// Create custom generation configuration
const config = {
max_new_tokens: 100,
temperature: 0.7,
top_k: 50,
top_p: 0.9,
repetition_penalty: 1.2
};

// Generate text with custom configuration
const output = await pipe.generate(prompt, {
images: images,
generationConfig: config
});
```
</TabItemJS>
</LanguageTabs>
</BasicGenerationConfiguration>

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
import Button from '@site/src/components/Button';
import { LanguageTabs, TabItemCpp, TabItemPython } from '@site/src/components/LanguageTabs';
import { LanguageTabs, TabItemCpp, TabItemPython, TabItemJS } from '@site/src/components/LanguageTabs';

import UseCaseCard from './UseCaseCard';

import CodeExampleCpp from '@site/docs/use-cases/image-processing/_sections/_run_model/_code_example_cpp.mdx';
import CodeExamplePython from '@site/docs/use-cases/image-processing/_sections/_run_model/_code_example_python.mdx';
import CodeExampleJS from '@site/docs/use-cases/image-processing/_sections/_run_model/_code_example_js.mdx';

export const ImageProcessing = () => (
<UseCaseCard>
Expand All @@ -27,6 +28,9 @@ export const ImageProcessing = () => (
<TabItemCpp>
<CodeExampleCpp />
</TabItemCpp>
<TabItemJS>
<CodeExampleJS />
</TabItemJS>
</LanguageTabs>
</UseCaseCard.Code>
<UseCaseCard.Actions>
Expand Down
2 changes: 2 additions & 0 deletions src/js/include/addon.hpp
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,10 @@ typedef Napi::Function (*Prototype)(Napi::Env);

struct AddonData {
Napi::FunctionReference core;
Napi::FunctionReference vlm_pipeline;
Napi::FunctionReference tokenizer;
Napi::FunctionReference perf_metrics;
Napi::FunctionReference vlm_perf_metrics;
Napi::FunctionReference chat_history;
Napi::ObjectReference openvino_addon;
};
Expand Down
Loading
Loading