new·The score now tells you which way it movedA brain's exam only ever grows: its own material writes questions, and so does every question a real caller asked and did not get answered. The score is a percentage over that growing set, so a brain that learned more could post a smaller number — and this week three did. One of them answered two MORE questions than the week before and showed eighteen points less. Printed as a single percentage, that reads as decline to a reader and as punishment to anyone who contributes material.all news →
mozg.beta
Sign in

AI SDK · Cookbook · all subjects

multimodal/overview

13 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

TextPart message structure

TextPart has type='text' and text=string properties. It represents a text portion of a message.

ImagePart message structure

ImagePart has type='image', image (string|Uint8Array|Buffer|ArrayBuffer|URL), and optional mediaType (string) properties. The image field accepts base64 encoded content, base64 data URLs, or http(s) URLs.

FilePart message structure

FilePart has type='file', data (string|Uint8Array|Buffer|ArrayBuffer|URL), and mediaType (string, required) properties. The data field accepts base64 encoded content, base64 data URLs, or http(s) URLs.

UserModelMessage type definition

UserModelMessage has a role field set to 'user' and a content field that accepts UserContent. UserContent can be either a string or an array containing TextPart, ImagePart, or FilePart elements. The Zod schema for UserModelMessage can be accessed via the userModelMessageSchema export.

AssistantModelMessage type definition

AssistantModelMessage has a role field set to 'assistant' and a content field that accepts AssistantContent. AssistantContent can be either a string or an array containing TextPart, CustomPart, or ToolCallPart elements. The Zod schema for AssistantModelMessage can be accessed via the assistantModelMessageSchema export.

ToolModelMessage type definition

ToolModelMessage has a role field set to 'tool' and a content field that accepts ToolContent. ToolContent is an array of ToolResultPart elements. The Zod schema for ToolModelMessage can be accessed via the toolModelMessageSchema export.

Message part types supported by AI SDK

The AI SDK supports the following message part types: TextPart (text content), FilePart (files with mediaType), CustomPart (provider-specific content), ToolCallPart (tool calls with toolCallId, toolName, and args), and ToolResultPart (tool results with toolCallId, toolName, output, and optional providerOptions). ImagePart is deprecated in favor of FilePart with mediaType 'image' or specific image subtypes.

FilePart structure

FilePart has a type field set to 'file', a data field accepting FileData (which can be {type: 'data', data: DataContent}, {type: 'url', url: URL}, {type: 'reference', reference: ProviderReference}, or {type: 'text', text: string}), an optional filename field, and a required mediaType field which can be either a full IANA media type (e.g., 'image/png') or a top-level segment (e.g., 'image', 'audio', 'video', 'text').

CustomPart structure

CustomPart has a type field set to 'custom', a kind field containing a string in format '{provider}.{provider-type}' that identifies the content type, and an optional providerOptions field for additional provider-specific metadata.

ImagePart deprecation

ImagePart is deprecated. Use FilePart with mediaType set to 'image' or a more specific image subtype instead.

TextPart structure

TextPart has a type field set to 'text' and a text field containing a string of text content.

ModelMessage types overview

ModelMessage encompasses four message types: SystemModelMessage (role 'system' with string content), UserModelMessage (role 'user' with UserContent which can be string or array of TextPart, ImagePart, or FilePart), AssistantModelMessage (role 'assistant' with AssistantContent which can be string or array of TextPart, CustomPart, or ToolCallPart), and ToolModelMessage (role 'tool' with ToolContent which is an array of ToolResultPart).

SystemModelMessage type definition

SystemModelMessage has a role field set to 'system' and a content field that accepts a string. The Zod schema for SystemModelMessage can be accessed via the systemModelMessageSchema export.

Give your agent this brain