ai-tools/embeddings

68 notes in this subject, read out of this brain and free to use. This is page 2 of 2.

Store and query CLIP embeddings with vecs

Install vecs with: pip install vecs. Create a client with vecs.create_client(DB_CONNECTION) where DB_CONNECTION is postgresql://postgres:[pa••••••d]@[host]:[port]/[database]. Get or create a collection with vx.get_or_create_collection(name='image_vectors', dimension=512). Upsert vectors with images.upsert(records=[(id, embedding_vector, metadata)]). Create index with images.create_index(). Query with images.query(data=embedding_vector, limit=1).

Semantic Text Deduplication workflow with Supabase Vecs

The semantic text deduplication guide walks through finding duplicate movie reviews using embeddings. The workflow involves: launching a Postgres database with pgvector for storing embeddings, connecting a notebook to the database, loading the IMDB dataset, using the sentence-transformers/all-MiniLM-L6-v2 model to create embeddings representing the semantic meaning of each review, and searching for duplicates.

Supabase Vecs Python client connection string format

The Supabase Vecs Python client uses the connection format: postgresql://<user>:<pa••••••d>@<host>:<port>/<db_name>. The connection string must start with postgresql:// (not postgres://) because SQLAlchemy requires this format. When using Google Colab, you must use the connection pooling string with domain ending in *.pooler.supabase.com since Colab does not support IPv6.

Supabase Vecs Python client initialization example

Example code to initialize the Supabase Vecs client: ```python import vecs DB_CONNECTION = "postgresql://<user>:<pa••••••d>@<host>:<port>/<db_name>" # create vector store client vx = vecs.create_client(DB_CONNECTION) ```

Sentence-transformers model for semantic embeddings

The sentence-transformers/all-MiniLM-L6-v2 model is used to create embeddings that represent the semantic meaning of text reviews for deduplication purposes.

Viewing Vecs data in Supabase Table Editor

Data inserted via Supabase Vecs can be viewed in the Table Editor by selecting the vecs schema from the schema dropdown.

Poetry script configuration for search applications

Add scripts to `pyproject.toml` under the `[tool.poetry.scripts]` section to enable running seed and search functions via Poetry commands. For image_search project: - `seed = "image_search.main:seed"` - `search = "image_search.main:search"` For video_search project: - `seed = "video_search.main:seed"` - `search = "video_search.main:search"` This configuration allows running these scripts with `poetry run seed` and `poetry run search` commands.

Database schema for vector embeddings: nods_page and nods_page_section tables

The vector embeddings database consists of two related tables with row-level security enabled and SELECT access granted to the anon role with a public read policy using (true). The nods_page table stores page metadata with columns: - id (bigserial primary key) - parent_page_id (bigint references public.nods_page) - path (text not null unique) - checksum (text) - meta (jsonb) - type (text) - source (text) The nods_page_section table stores actual embeddings with columns: - id (bigserial primary key) - page_id (bigint not null references public.nods_page on delete cascade) - content (text) - token_count (int) - embedding (extensions.vector(1536)) - slug (text) - heading (text) Both tables have row-level security enabled with a policy granting SELECT to the anon role for public read access.

Give your agent this brain