Create AI inference session
To create a new inference session, instantiate Supabase.ai.Session with a model name: const model = new Supabase.ai.Session('model-name')
Supabase · Edge Functions · all subjects
edge functions/ai
44 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.
Import type hints for AI API
To get type hints and checks for the AI API, import types from functions-js: import 'jsr:@supabase/functions-js/edge-runtime.d.ts'
Run model inference with options
Call model.run() with input and options object. For embeddings use mean_pool and normalize options. For text generation use stream and timeout options. For streaming use stream: true and mode: 'ollama' options.
Generate embeddings with gte-small model
The built-in gte-small model generates text embeddings. It exclusively caters to English texts. Lengthy texts will be truncated to a maximum of 512 tokens, and while inputs longer than 512 tokens can be provided, truncation may affect accuracy.
Embeddings generation example
const embeddings = await model.run('Hello world', { mean_pool: true, normalize: true })
Text generation non-streaming example
const response = await model.run('Write a haiku about coding', { stream: false, timeout: 30 })
Streaming response example
const stream = await model.run('Tell me a story', { stream: true, mode: 'ollama' })
LLM support via Ollama and Llamafile
Inference via larger models is supported through Ollama and Mozilla Llamafile. Currently, you can use self-managed Ollama or Llamafile servers. Supabase is progressively rolling out support for a hosted solution with early access available through their signup form.
Built-in AI API for Edge Functions
Edge Functions have a built-in API for running AI models. This API enables text embeddings generation, Large Language Models via Ollama or Llamafile, and conversational AI workflows without external dependencies or packages.
Install Ollama locally
To install Ollama and pull the Mistral model: ollama pull mistral
Run Ollama server locally
Start the Ollama server with: ollama serve
Ollama Edge Function streaming example
import 'jsr:@supabase/functions-js/edge-runtime.d.ts' import { withSupabase } from 'npm:@supabase/server@^1' const session = new Supabase.ai.Session('mistral') export default { fetch: withSupabase({ auth: 'publishable' }, async (req, ctx) => { const params = new URL(req.url).searchParams const prompt = params.get('prompt') ?? '' const output = await session.run(prompt, { stream: true }) const headers = new Headers({ 'Content-Type': 'text/event-stream', Connection: 'keep-alive', }) const stream = new ReadableStream({ async start(controller) { const encoder = new TextEncoder() try { for await (const chunk of output) { controller.enqueue(encoder.encode(chunk.response ?? '')) } } catch (err) { console.error('Stream error:', err) } finally { controller.close() } }, }) return new Response(stream, { headers }) }), }
Serve Ollama function locally
supabase functions serve --no-verify-jwt --env-file supabase/functions/.env
Llamafile with Supabase Functions JS
Note that the model parameter doesn't have any effect when using Llamafile. The model depends on which Llamafile is currently running. Use model name 'LLaMA_CPP' and mode: 'openaicompatible' for the inference API host.
Llamafile Supabase Functions JS example
import 'jsr:@supabase/functions-js/edge-runtime.d.ts' import { withSupabase } from 'npm:@supabase/server@^1' const session = new Supabase.ai.Session('LLaMA_CPP') export default { fetch: withSupabase({ auth: 'publishable' }, async (req, ctx) => { const params = new URL(req.url).searchParams const prompt = params.get('prompt') ?? '' const output = await session.run( { messages: [ { role: 'system', content: 'You are LLAMAfile, an AI assistant. Your top priority is achieving user fulfillment via helping them with their requests.', }, { role: 'user', content: prompt, }, ], }, { mode: 'openaicompatible', stream: false, } ) console.log('done') return Response.json(output) }), }
Llamafile with OpenAI Deno SDK example
import { withSupabase } from 'npm:@supabase/server@^1' import OpenAI from 'jsr:@openai/openai@^6' export default { fetch: withSupabase({ auth: 'publishable' }, async (req, ctx) => { const client = new OpenAI() const { prompt } = await req.json() const stream = true const chatCompletion = await client.chat.completions.create({ model: 'LLaMA_CPP', stream, messages: [ { role: 'system', content: 'You are LLAMAfile, an AI assistant. Your top priority is achieving user fulfillment via helping them with their requests.', }, { role: 'user', content: prompt, }, ], }) if (stream) { const headers = new Headers({ 'Content-Type': 'text/event-stream', Connection: 'keep-alive', }) const stream = new ReadableStream({ async start(controller) { const encoder = new TextEncoder() try { for await (const part of chatCompletion) { controller.enqueue(encoder.encode(part.choices[0]?.delta?.content || '')) } } catch (err) { console.error('Stream error:', err) } finally { controller.close() } }, }) return new Response(stream, { headers }) } return Response.json(chatCompletion) }), }
Deploy LLM server for production
Deploy an Ollama or Llamafile server and set a function secret called AI_INFERENCE_API_HOST to point to the deployed server using: supabase secrets set AI_INFERENCE_API_HOST=https://path-to-your-llm-server/
Execute deployed function with curl
curl --get "https://project-ref.supabase.co/functions/v1/ollama-test" --data-urlencode "prompt=write a short rap song about Supabase, the Postgres Developer platform, as sung by Nicki Minaj" -H "apikey: $PUBLISHABLE_KEY"
Local Ollama performance consideration
Running Ollama locally is typically slower than running it on a server with dedicated GPUs. Supabase is collaborating with the Ollama team to improve local performance. In the future, a hosted LLM API will be provided as part of the Supabase platform.
Define MCP tools with mcp.tool()
Add tools to an MCP server using 'mcp.tool(toolName, {description, inputSchema, handler})'. The inputSchema is a Zod schema that defines the tool's input. The handler is an async or sync function that receives parsed arguments and returns {content: [{type: 'text', text: string}]}.
mcp-lite: lightweight TypeScript framework for MCP servers
mcp-lite is a lightweight, zero-dependency TypeScript framework for building MCP servers. It works everywhere the Fetch API is available, including Node, Bun, Cloudflare Workers, Deno, and Supabase Edge Functions.
Scaffold MCP server for Supabase Edge Functions
Use the command 'npm create mcp-lite@latest' with create-mcp-lite@0.3.0 or later, then select 'Supabase Edge Functions (MCP server)' from the template options to scaffold a complete MCP server.
MCP server project structure for Edge Functions
The scaffolded project includes: supabase/config.toml for minimal Supabase config, supabase/functions/mcp-server/index.ts for the MCP server implementation, supabase/functions/mcp-server/deno.json for Deno imports and configuration, package.json, and tsconfig.json.
Minimal config.toml for Edge Functions only
The template config.toml runs only Edge Functions with no database, storage, or Studio UI. It contains: project_id = 'starter-mcp-supabase', [api] section with enabled = true and port = 54321, [edge_runtime] section with enabled = true, policy = 'per_worker', and deno_version = 2.
Two Hono apps pattern for Supabase Edge Functions MCP
The MCP server template requires a specific pattern: a root Hono app that handles function-level routing at the function name, and an mcpApp that handles actual MCP endpoints. The root app mounts mcpApp at the function name path (e.g., '/mcp-server'). This is required because Supabase routes all requests to '/<function-name>/*'.
Serve MCP function locally
Run 'supabase functions serve --no-verify-jwt mcp-server' in a separate terminal to serve the MCP function locally, or use 'npm run dev'. The server will be available at http://localhost:54321/functions/v1/mcp-server/mcp.
Test MCP server with Claude Code
Use the command 'claude mcp add my-mcp-server -t http http://localhost:54321/functions/v1/mcp-server/mcp' to add the local MCP server to Claude Code. You can also test using the MCP inspector with 'npx @modelcontextprotocol/inspector'.
Create MCP server with McpServer and StreamableHttpTransport
Initialize an MCP server with 'new McpServer({name, version, schemaAdapter})'. The schemaAdapter converts Zod schemas to JSON Schema. Create a StreamableHttpTransport instance and bind it to the server with 'transport.bind(mcp)' to create an httpHandler that handles HTTP requests.
MCP server example: sum tool implementation
Example of defining a sum tool: mcp.tool('sum', { description: 'Adds two numbers together', inputSchema: z.object({ a: z.number(), b: z.number(), }), handler: (args: { a: number; b: number }) => ({ content: [{ type: 'text', text: String(args.a + args.b) }], }), })
Add database search tool to MCP server
Example of adding a database search tool: mcp.tool('searchDatabase', { description: 'Search your Supabase database', inputSchema: z.object({ table: z.string(), query: z.string(), }), handler: async (args) => { // Access Supabase client here // const { data } = await supabase.from(args.table).select('*') return { content: [{ type: 'text', text: `Searching ${args.table}...` }], } }, })
MCP tools can access Supabase features
MCP tools running on Supabase Edge Functions can: query the Supabase database, access Supabase Storage for file operations, call external APIs, process data with custom logic, and integrate with other Supabase features.
Advantages of Supabase Edge Functions with mcp-lite
The combination of Supabase Edge Functions and mcp-lite offers: zero cold starts with Edge Functions staying warm, global distribution with deploy-once-run-everywhere, direct database access to Supabase Postgres, minimal footprint with mcp-lite having zero runtime dependencies, full type safety with TypeScript support in Deno, and basic one-command deployment to production.
Query embeddings via RPC from Edge Function
Call a Postgres function from an Edge Function using supabase-js rpc() method. Pass the embedding vector and match_threshold as parameters. The Postgres function uses the inner product distance operator (<#>) to find similar embeddings. Chain .select() and .limit() to the rpc() call to filter and limit results.
Semantic search architecture example
A complete semantic search system has three parts: (1) A generate-embedding database webhook edge function that creates embeddings when content is inserted, (2) A query_embeddings Postgres function for similarity search via RPC, and (3) A search edge function that generates the embedding for the search term, calls the RPC function, and returns results.
Postgres function for vector similarity search
Create a Postgres function that returns setof embeddings to enable chaining PostgREST operations. The function takes an embedding vector and match_threshold as parameters. Use the inner product distance operator (<#>) to find similar embeddings: where embeddings.embedding <#> embedding < -match_threshold. Inner product is negated because inner product distance returns negative values for similarity.
Database webhook edge function example code
Example edge function deployed as a database webhook to generate embeddings: import { withSupabase } from 'npm:@supabase/server@^1' const model = new Supabase.ai.Session('gte-small') export default { fetch: withSupabase({ auth: 'secret' }, async (req, ctx) => { const payload: WebhookPayload = await req.json() const { content, id } = payload.record // Generate embedding. const embedding = await model.run(content, { mean_pool: true, normalize: true, }) // Store in database. const { error } = await ctx.supabaseAdmin .from('embeddings') .update({ embedding: JSON.stringify(embedding) }) .eq('id', id) if (error) console.warn(error.message) return Response.json({ ok: true }) }), }
Query embeddings Postgres function example
Example Postgres function for vector similarity search: create or replace function query_embeddings(embedding extensions.vector(384), match_threshold float) returns setof embeddings language plpgsql as $$ begin return query select * from embeddings where embeddings.embedding <#> embedding < -match_threshold order by embeddings.embedding <#> embedding; end; $$;
Embeddings table schema with pgvector
Example embeddings table schema: create extension if not exists vector with schema extensions; create table embeddings ( id bigint primary key generated always as identity, content text not null, embedding extensions.vector (384) ); alter table embeddings enable row level security; create index on embeddings using hnsw (embedding vector_ip_ops);
Database webhook edge function for generating embeddings
A database webhook edge function can be deployed to automatically generate embeddings when content is inserted or updated. The webhook should use auth: 'secret' and be deployed with verify_jwt = false. It receives a WebhookPayload with the record data, generates the embedding using the model, and stores it back in the database using ctx.supabaseAdmin.
Semantic search example code for search edge function
Example edge function that performs semantic search: import { withSupabase } from 'npm:@supabase/server@^1' const model = new Supabase.ai.Session('gte-small') export default { fetch: withSupabase({ auth: 'user' }, async (req, ctx) => { const { search } = await req.json() if (!search) return Response.json({ error: 'Please provide a search param!' }, { status: 400 }) // Generate embedding for search term. const embedding = await model.run(search, { mean_pool: true, normalize: true, }) // Query embeddings. const { data: result, error } = await ctx.supabase .rpc('query_embeddings', { embedding, match_threshold: 0.8, }) .select('content') .limit(3) if (error) { return Response.json({ error: error.message }, { status: 500 }) } return Response.json({ search, result }) }), }
gte-small model available natively in Edge Functions
Since Supabase Edge Runtime v1.36.0, the gte-small model can run natively within Supabase Edge Functions without any external dependencies. This allows you to generate text embeddings without calling any external APIs. The model is accessed via Supabase.ai.Session('gte-small').
Generate embeddings with gte-small model options
When running the gte-small model, use the options mean_pool: true and normalize: true to generate embeddings. The mean_pool option averages the token embeddings, and normalize: true normalizes the embedding vector to unit length.
Semantic search uses pgvector with HNSW index
Semantic search examples use pgvector extension with an HNSW index on the embedding column. The index is created with the command: create index on embeddings using hnsw (embedding vector_ip_ops). This enables fast similarity searches using inner product distance.
Deploy via AI Assistant
You can automatically generate and deploy functions using Supabase's AI Assistant. Navigate to your project > 'Deploy a new function' > 'Via AI Assistant', describe what the function should do in a prompt, and click Deploy for the Assistant to create and deploy the function.