deep agents & deployment

49 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

Managed Deep Agent deployment process

Deploying a Managed Deep Agent compiles a code-first project into a managed LangGraph app, syncs deploy-owned context to Context Hub, uploads the compiled source, and triggers a LangSmith hosted deployment build.

Prerequisites for Managed Deep Agent deployment

Before deploying a Managed Deep Agent, you need: a workspace with Managed Deep Agents public beta access, a LangSmith API key for that workspace in .env or shell environment, the mda CLI installed from managed-deepagents, project dependencies installed with uv sync for Python projects or npm install for TypeScript projects, and model provider credentials such as OPENAI_API_KEY in .env, shell environment, or LangSmith workspace secrets.

mda deploy command for Managed Deep Agents

Deploy a local Managed Deep Agent project using the command: mda deploy . This can be customized with the --name flag to set deployment name explicitly, --deployment-type prod for production deployments, and --no-wait to trigger the build without polling for completion.

mda deploy routes local project inputs

The mda deploy command routes local project inputs as follows: instructions.md and skills/** go to Context Hub deploy-owned context, .env goes to deploy auth and non-reserved hosted secrets (not archived), project source files go to .mda/build source archive and hosted deployment, and schedules/** go to LangSmith cron jobs after deployment is live.

Environment file handling in mda deploy

mda deploy reads project .env values before shell environment variables. The .env file should contain the LangSmith API key that authenticates the deploy and runtime secrets the hosted deployment needs, such as OPENAI_API_KEY, model provider tokens, and custom tool credentials.

Reserved platform variables in .env

LANGSMITH_API_KEY, LANGGRAPH_HOST_API_KEY, LANGCHAIN_API_KEY, and other platform variables are reserved. They can authenticate the deploy, but they are not uploaded as user-managed deployment secrets. They are not copied into the compiled build archive.

Non-reserved .env entries in mda deploy

Non-reserved .env entries such as model provider keys, MCP tokens, and custom tool credentials are forwarded as hosted deployment secrets when mda deploy creates or updates the deployment. If the configured model requires a provider key, deploy fails before upload unless that key is available from .env, shell environment, or LangSmith workspace secrets.

What gets excluded from compiled build archive

Reserved platform variables, empty values, .env, and .env.* files are not copied into the compiled build archive.

Handling shell environment secrets in mda deploy

When a provider key is only in the shell environment and not in .env or LangSmith workspace secrets, mda deploy forwards it as a secret for that deploy.

CLI targets US LangSmith Cloud by default

The mda CLI targets US LangSmith Cloud by default.

Schedule reconciliation with --no-wait flag

When --no-wait is set in mda deploy, schedule reconciliation is skipped for that deploy invocation because the CLI exits before the deployment reaches DEPLOYED state.

Deploy failure troubleshooting for Managed Deep Agents

If a deployment reaches BUILD_FAILED or DEPLOY_FAILED state, open the printed deployment URL in LangSmith and inspect the revision logs to troubleshoot the deployment.

Managed Deep Agents evals are Harbor evals

Managed Deep Agents evals use Harbor evals. The canonical Harbor dataset is located at evals/tasks/. The mda evals commands package the managed agent for Harbor and can optionally compile a minimal starter task from evals/scaffold/ into a complete task under evals/tasks/.

Project structure for Managed Deep Agents evals

For Python projects, the project structure should have a top-level evals/ directory containing tasks/ (canonical Harbor dataset) and an optional scaffold/ directory (optional starter tasks). For example: my-agent/ with agent.py and evals/ containing evals/tasks/<task>/ and evals/scaffold/<task>/.

Scaffold to canonical Harbor task workflow

A scaffold under evals/scaffold/<task>/ has a one-way relationship with its canonical Harbor task under evals/tasks/<task>/, using the workflow: evals/scaffold/<task>/ → mda evals compile → evals/tasks/<task>/

evals/scaffold/ is not a separate eval system

evals/scaffold/ is not a second eval system. Harbor runs tasks under evals/tasks/. Scaffolding should only be used when you want MDA to create a minimal starting point.

Two authoring workflows for Managed Deep Agents evals

There are two ways to populate the canonical Harbor dataset: (1) Author a Harbor task directly by creating a complete task under evals/tasks/ and managing it with Harbor when you need the full Harbor task format, or (2) Start from an optional scaffold by running mda evals init <name> to create a minimal task under evals/scaffold/, then compile it into evals/tasks/ with the agent artifact and Harbor adapter.

Prerequisites for Managed Deep Agents evals

Prerequisites include: a Managed Deep Agents project created with mda init or an existing project with an agent entry; Docker running locally when using Harbor's default docker environment; the mda CLI from managed-deepagents; Harbor on PATH or uv installed to run uv run --with harbor; model and tool credentials exported in the shell that runs Harbor.

Harbor does not load .env values automatically

Harbor does not load values from the project .env file. When MDA generates a Harbor job config, it writes ${VAR} placeholders for eligible .env keys, not their values. Required variables must be exported in the shell before running Harbor.

Harbor task structure and grading paths

A Harbor task can define its instruction, environment, verifier, metadata, and other configuration. Harbor runs the agent in the task environment, then runs tests/test.sh to grade the result. During grading, the main paths are: /app (agent working directory and task output), /tests (task verifier files), /logs/verifier/ (verifier reward output).

Harbor verifier reward output requirements

The verifier must write a numeric reward to /logs/verifier/reward.txt or numeric metrics to /logs/verifier/reward.json.

Files authored directly under evals/tasks/ are preserved

Files you author directly under evals/tasks/ are preserved when MDA compiles scaffolds with other names.

mda evals init command creates scaffold task

Running mda evals init <name> from the Managed Deep Agents project root creates a minimal scaffold task. The task name can contain ASCII letters, numbers, underscore, and hyphen. The command creates evals/scaffold/<name>/instruction.md and evals/scaffold/<name>/tests/test_answer.py for Python projects. The starter task asks the agent to write answer.txt containing PONG.

Python scaffold test template

For Python scaffolded tasks, the starter test is: from pathlib import Path; def test_answer_is_pong(): assert Path("/app/answer.txt").read_text().strip() == "PONG"

mda evals compile command

Run mda evals compile . to compile every scaffold under evals/scaffold/. Use --task <name> flag to refresh specific scaffolds (repeat flag for multiple tasks). For each selected scaffold, MDA replaces the matching directory under evals/tasks/, copies the complete scaffold, and adds tests/test.sh when the scaffold does not provide one.

mda evals compile --task flag examples

To refresh specific scaffolds, repeat the --task flag. Example: mda evals compile . --task smoke --task regression

Treat evals/scaffold/ as source of truth

Treat evals/scaffold/<name>/ as the source of truth for a scaffolded task. Compiling that scaffold replaces the entire matching evals/tasks/<name>/ directory, including changes made only to the canonical copy.

Harbor files can be added to scaffolds

You can add Harbor files such as task.toml, environment/, or a custom tests/test.sh to a scaffold under evals/scaffold/<name>/. MDA copies them into the canonical Harbor task during compilation.

Compilation output paths for Managed Deep Agents evals

Compilation writes or updates the Harbor workspace with the following paths: evals/artifact/ (compiled managed agent and artifact manifest), evals/harbor-adapter/ (embedded mda_harbor adapter that Harbor imports to run the agent), evals/tasks/ (canonical Harbor dataset including compiled scaffolds and directly authored tasks), evals/harbor-job.json (ready-to-edit Harbor job config), evals/harbor-jobs/<id>/ (local trial results for this compile).

mda evals compile repeatable flags

mda evals compile supports the following repeatable flags: --task <name> (select one task; repeat to select more tasks; if a selected task has a source under evals/scaffold/, MDA refreshes its canonical copy; omit to select all tasks and refresh every scaffold), --model <provider:model> (record a model in the artifact manifest; the generated job config uses the first model; if omitted, MDA uses the agent's model when available).

Running trials with Harbor for Managed Deep Agents

After mda evals compile, export the listed variables and run the printed command from the project root. Example: export OPENAI_API_KEY="<key>"; PYTHONPATH=evals/harbor-adapter uv run --with harbor harbor run --config evals/harbor-job.json --yes. If harbor is already on PATH, the command uses harbor run directly instead of uv run --with harbor.

Harbor job configuration and editing

Edit evals/harbor-job.json to change the task dataset, model, environment, concurrency, or attempts. Harbor owns trial orchestration, environments, and reporting.

Resuming or starting new Harbor runs

Running the same Harbor command again resumes the jobs directory referenced by the config. Recompile or pass Harbor a fresh --job-name to start a new run.

Version control guidance for Managed Deep Agents evals

Check in the Harbor definitions and configuration under evals/ that your project uses. Keep local run output under evals/harbor-jobs/ out of version control. The evals/ directory is not included in the deployed agent build.

Default identity authentication for Managed Deep Agents

By default, Managed Deep Agents uses LangSmith API key authentication. The `mda init` command scaffolds this secure default. Callers must present a valid LangSmith workspace API key, which Managed Deep Agents verifies with LangSmith Cloud.

Define identity with LangSmith API key in JavaScript

Use `defineIdentity({ auth: auth.langsmithApiKey() })` from the managed-deepagents module to configure LangSmith API key authentication.

LangSmith API key passed as x-api-key header

Clients send the LangSmith API key as the `x-api-key` header. No additional verification endpoint or tenant settings in the project `.env` are needed, as LangSmith Cloud supplies those.

Identity file location in Managed Deep Agents project

The identity declaration lives at the project root in a file named `identity.py` (Python) or `identity.ts` (TypeScript), alongside the main agent file.

Authenticate end users with Supabase in Managed Deep Agents

Use Supabase authentication when a browser or client calls the deployment as a signed-in person. Each user gets private threads. Managed Deep Agents configures ownership automatically. Configure it by enabling an auth provider in the Supabase dashboard, copying the project reference (subdomain before `.supabase.co`), and passing it to `define_identity(auth=auth.supabase(project_ref="your-project-ref"))`.

Define identity with Supabase in JavaScript

Use `defineIdentity({ auth: auth.supabase({ projectRef: "your-project-ref" }) })` from the managed-deepagents module to configure Supabase authentication, where projectRef is the subdomain before `.supabase.co` in your project URL.

Send Supabase access token as Bearer token

In the client app, sign the user in with Supabase, then send the Supabase access token on every deployment request as `Authorization: Bearer <supabase_access_token>` header. Do not send a LangSmith API key in this mode.

Supabase JWT verification in Managed Deep Agents

Managed Deep Agents verifies the JWT against the project's JWKS URL derived from the project reference: `https://<project-ref>.supabase.co/auth/v1/.well-known/jwks.json`.

Supabase identity migration warning for existing deployments

Adding Supabase identity to an existing deployment does not add owner metadata to existing threads. Plan and test a migration before relying on identity-based access for those threads.

instructions.md file location in Managed Deep Agents project

The instructions.md file lives at the project root directory, alongside the agent.py or agent.ts file. The project structure is: my-agent/ containing agent.py (or agent.ts) and instructions.md.

Instructions define Managed Deep Agent system prompt

Instructions define the agent's behavior and make up the core of the agent's system prompt. They are defined in a simple markdown file and are picked up automatically by the agent.

How instructions are used in Managed Deep Agents

Instructions are inserted into the agent's system prompt on every run. They are always present and help guide the agent's behavior.

Instructions synced to Context Hub on deploy

When you run mda deploy to deploy the agent, instructions are automatically synced to the agent's Context Hub repo. You can then edit the instructions in the LangSmith UI and have your changes automatically propagated to the agent.

Relationship between instructions, skills, and memory in Managed Deep Agents

Instructions define general behavior and role. Skills are for task-specific procedures that the agent loads only when relevant. Memory is for knowledge the agent learns and retains across threads.

Example instructions.md content for Managed Deep Agent

An instructions.md file typically starts with a heading like '# Assistant' followed by a description of the agent's role and behavior. The file can define the agent's role, behavior, constraints, and guidance for using its tools.

Give your agent this brain