multi-modal/ui

5 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

useChat hook setup with DefaultChatTransport

Use the useChat hook from '@ai-sdk/react' with DefaultChatTransport to configure the API endpoint. The hook provides messages (array of objects with id, role, and parts properties) and sendMessage function. Each message contains a parts array that can include text, images, PDFs, and other content types. Files are converted to data URLs before being sent to maintain compatibility.

Basic chat UI component code example

```tsx 'use client'; import { useChat } from '@ai-sdk/react'; import { DefaultChatTransport } from 'ai'; import { useState } from 'react'; export default function Chat() { const [input, setInput] = useState(''); const { messages, sendMessage } = useChat({ transport: new DefaultChatTransport({ api: '/api/chat', }), }); return ( <div className="flex flex-col w-full max-w-md py-24 mx-auto stretch"> {messages.map(m => ( <div key={m.id} className="whitespace-pre-wrap"> {m.role === 'user' ? 'User: ' : 'AI: '} {m.parts.map((part, index) => { if (part.type === 'text') { return <span key={`${m.id}-text-${index}`}>{part.text}</span>; } return null; })} </div> ))} <form onSubmit={async event => { event.preventDefault(); sendMessage({ role: 'user', parts: [{ type: 'text', text: input }], }); setInput(''); }} className="fixed bottom-0 w-full max-w-md mb-8 border border-gray-300 rounded shadow-xl" > <input className="w-full p-2" value={input} placeholder="Say something..." onChange={e => setInput(e.target.value)} /> </form> </div> ); } ``` This code creates a basic chat interface with message display and text input, using useChat hook with DefaultChatTransport to communicate with the /api/chat endpoint.

useChat hook return values and message structure

The useChat hook returns an object with messages and sendMessage properties. Messages is an array of message objects, each containing an id, role ('user' or 'assistant'), and parts array. The parts array can contain multiple parts of different types: text parts with text property, and file parts with type 'file', mediaType (MIME type string), and url (data URL string).

Image part structure in useChat messages

Image parts in useChat messages are structured with: type set to 'file' (as const), mediaType set to the image MIME type (e.g., 'image/png'), and url set to the image URL string.

Render message parts in chat UI

When displaying messages in the chat interface, iterate through message.parts array and render based on part.type. For text parts, display the part.text content. The parts structure allows rendering different content types within a single message.

Give your agent this brain