Streaming AI chat responses responsibly.

What streaming changes

Many model providers can send incremental output over an event stream. Your app can render text as it arrives instead of waiting for a complete response. The transport and event schema vary between providers.

A safe streaming lifecycle

1. Start a request with a correlation ID and a time budget.
2. Open the provider stream and validate the initial response.
3. Accumulate text or structured events in order; escape displayed content.
4. Handle tool calls and refusals according to the provider protocol.
5. Complete only when a terminal event or documented stop condition arrives.
6. Recover gracefully from cancellation, disconnects and rate limiting.

UI and transport considerations

A partial assistant response is not necessarily a final or verified answer. Preserve the distinction in the UI. Bound the accumulated text size, avoid exposing private prompts through logs, and make cancellation visible to the user.

Server-sent events (SSE) are commonly used for incremental server-to-client delivery. WebSockets can also be appropriate, but they introduce different lifecycle concerns.

See request design and chat API cost controls.