Generate multiple embeddings with AI SDK
Example function to embed multiple chunks and return embeddings with content:
import { embedMany } from 'ai';
const embeddingModel = 'openai/text-embedding-ada-002';
export const generateEmbeddings = async (
value: string,
): Promise<Array<{ embedding: number[]; content: string }>> => {
const chunks = generateChunks(value);
const { embeddings } = await embedMany({
model: embeddingModel,
values: chunks,
});
return embeddings.map((e, i) => ({ content: chunks[i], embedding: e }));
};
Embeddings for semantic similarity
Embeddings are a way to represent words, phrases, or images as vectors in a high-dimensional space. Similar words are plotted close to each other in vector space. Cosine similarity is used to calculate similarity between two vectors, where a value of 1 indicates high similarity and -1 indicates high opposition. Larger inputs to embeddings result in lower quality embeddings.
Generate single embedding with AI SDK
Example function to generate a single embedding from input string:
import { embed } from 'ai';
const embeddingModel = 'openai/text-embedding-ada-002';
export const generateEmbedding = async (value: string): Promise<number[]> => {
const input = value.replaceAll('\\n', ' ');
const { embedding } = await embed({
model: embeddingModel,
value: input,
});
return embedding;
};
embedMany function from AI SDK
The embedMany function from the 'ai' package takes an embedding model and an array of values, and returns embeddings for all provided values. Usage: await embedMany({ model: 'openai/text-embedding-ada-002', values: chunks }) returns an object with an embeddings array.
embed function from AI SDK
The embed function from the 'ai' package generates a single embedding for a string value. Usage: await embed({ model: 'openai/text-embedding-ada-002', value: input }) returns an object with a single embedding array.
RAG embedding generation with AI SDK
Use the embedMany() function from the AI SDK to generate embeddings for multiple text chunks at once. Specify the model parameter with an embedding model like 'openai/text-embedding-3-small'. The function returns an embeddings array in the same order as the input values.
RAG single embedding with embed function
Use the embed() function from the AI SDK to generate an embedding for a single query string. Specify the model parameter with an embedding model like 'openai/text-embedding-3-small'. The function returns a single embedding vector.
Embeddings enable practical RAG applications
Embeddings are used in Retrieval-Augmented Generation (RAG) systems for document retrieval by enabling semantic search based on the mathematical meaning of text rather than keyword matching.
embed() function basic usage
The embed() function from the ai SDK converts text into embeddings. It accepts a model parameter (e.g., 'openai/text-embedding-3-small') and a value parameter containing the text to embed. It returns an object with an embedding property containing the high-dimensional vector and a usage property with token usage information.
embed() code example
```ts
import { embed } from 'ai';
import 'dotenv/config';
async function main() {
const { embedding, usage } = await embed({
model: 'openai/text-embedding-3-small',
value: 'sunny day at the beach',
});
console.log(embedding);
console.log(usage);
}
main().catch(console.error);
```
This example demonstrates how to convert text into embeddings using the AI SDK embed() function.
Text embeddings as semantic representations
Text embeddings are numerical vector representations of text that capture semantic meaning, enabling machines to understand and process language mathematically. These high-dimensional vectors are crucial for AI applications including semantic search, document similarity comparison, and content recommendation.
embedMany function for batch embedding
Use the embedMany function from the AI SDK to embed multiple text inputs in a single operation. The function accepts an object with a model parameter (e.g., 'openai/text-embedding-3-small') and a values parameter containing an array of text strings to embed. The function returns an object with embeddings (array of embedding vectors) and usage (API usage statistics).
Batch embedding example with embedMany
import { embedMany } from 'ai';
import 'dotenv/config';
async function main() {
const { embeddings, usage } = await embedMany({
model: 'openai/text-embedding-3-small',
values: [
'sunny day at the beach',
'rainy afternoon in the city',
'snowy night in the mountains',
],
});
console.log(embeddings);
console.log(usage);
}
main().catch(console.error);
This example demonstrates how to embed multiple text strings using the AI SDK's embedMany function with OpenAI's text-embedding-3-small model.
Performance benefit of batch embeddings
Batch embedding significantly improves performance and reduces API calls when processing large datasets or multiple pieces of text. This approach is particularly useful when processing documents, chat messages, or any collection of text that needs to be vectorized, as it converts multiple text inputs into embeddings simultaneously rather than one at a time.