rag/backend

3 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Server action to create resource with embeddings

Example server action that creates a resource and generates embeddings for its content: 'use server'; import { NewResourceParams, insertResourceSchema, resources, } from '@/lib/db/schema/resources'; import { db } from '../db'; import { generateEmbeddings } from '../ai/embedding'; import { embeddings as embeddingsTable } from '../db/schema/embeddings'; export const createResource = async (input: NewResourceParams) => { try { const { content } = insertResourceSchema.parse(input); const [resource] = await db .insert(resources) .values({ content }) .returning(); const embeddings = await generateEmbeddings(content); await db.insert(embeddingsTable).values( embeddings.map(embedding => ({ resourceId: resource.id, ...embedding, })), ); return 'Resource successfully created and embedded.'; } catch (error) { return error instanceof Error && error.message.length > 0 ? error.message : 'Error, please try again.'; } };

RAG backend with AI SDK functions

The AI SDK provides three core functions for RAG: embed() for single value embedding, embedMany() for batch embeddings, cosineSimilarity() for comparing embedding vectors, and generateText() for the final generation step with context.

Knowledge base setup script example with Upstash

This example shows how to read an essay from a file, split it into paragraph chunks, and upsert them to Upstash Search: ```ts import fs from 'fs'; import path from 'path'; import 'dotenv/config'; import { Search } from '@upstash/search'; type KnowledgeContent = { text: string; section: string; title?: string; }; const search = new Search({ url: process.env.UPSTASH_SEARCH_REST_URL!, token: process.env.UPSTASH_SEARCH_REST_TOKEN!, }); const index = search.index<KnowledgeContent>('knowledge-base'); async function setupKnowledgeBase() { const content = fs.readFileSync(path.join(__dirname, 'essay.txt'), 'utf8'); const chunks = content .split(/\n\s*\n/) .map(chunk => chunk.trim()) .filter(chunk => chunk.length > 50); const batchSize = 100; for (let i = 0; i < chunks.length; i += batchSize) { const batch = chunks.slice(i, i + batchSize).map((chunk, j) => ({ id: `chunk-${i + j}`, content: { text: chunk, section: `section-${Math.floor((i + j) / 10)}`, title: chunk.split('\n')[0] || `Chunk ${i + j + 1}`, }, })); await index.upsert(batch); } } setupKnowledgeBase().catch(console.error); ```

Give your agent this brain