streaming

10 notes, read out of this brain and free to use. Each one was extracted from a source and is re-checked against its exam.

JSON Lines streaming format and content type

JSON Lines is a streaming format where one JSON object is sent per line, separated by newline characters, instead of being wrapped in square brackets with commas between items. The content type for JSON Lines responses is application/jsonl instead of application/json. The advantage is that the app can produce and send each line in turn while the client consumes the previous lines.

Streaming in FastAPI means sending items without waiting for the entire sequence

Streaming data means the app will start sending data items to the client without waiting for the entire sequence to be ready. The app sends the first item, the client receives and processes it, and the app continues producing the next item. This allows for asynchronous production and consumption of data.

Use yield in path operation function to stream JSON Lines

To stream JSON Lines with FastAPI, use yield instead of return in the path operation function to produce each item in turn.

Async path operation functions for JSON Lines streaming

For async path operation functions that stream JSON items of type Item (a Pydantic model), declare the return type as AsyncIterable[Item]. FastAPI will use this return type to validate the data, document it in OpenAPI, filter it, and serialize it using Pydantic.

Non-async path operation functions for JSON Lines streaming

For regular def functions without async, you can still use yield to stream JSON Lines. The right return type would be Iterable[Item]. FastAPI ensures the function runs correctly without blocking the event loop.

Omitting return type in JSON Lines streaming

When streaming JSON Lines, you can omit the return type declaration. FastAPI will then use the jsonable_encoder to convert the data to something that can be serialized to JSON and send it as JSON Lines.

Use cases for JSON Lines streaming

JSON Lines streaming can be used to stream data from AI LLM services, from logs or telemetry, or from other types of data that can be structured as JSON items.

Performance benefit of declaring return type for JSON Lines

When you declare a return type for JSON Lines streaming, Pydantic serializes it on the Rust side, providing much higher performance than if you don't declare a return type.

JSON Lines introduced in FastAPI 0.134.0

JSON Lines streaming support was added in FastAPI 0.134.0.

JSON Lines content encoding of newlines

Each JSON object in JSON Lines cannot contain literal newline characters in their content, but they can contain escaped newlines (\n) as per the JSON standard. Pydantic handles this automatically.

Give your agent this brain