Streaming
Streaming reduces time to first token and sends the response as text/event-stream events.
01
Enable streaming
Add "stream": true. The client must read complete SSE events separated by a blank line.
text/event-stream
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":"Hel"}}]}
data: {"id":"chatcmpl-...","object":"chat.completion.chunk","choices":[{"delta":{"content":"lo"}}]}
data: [DONE]02
Terminal semantics
| Format | Terminal signal |
|---|---|
| Chat Completions | data: [DONE] |
| Responses | response.completed or terminal error |
| Anthropic Messages | message_stop |
03
Usage
Usage normally arrives in terminal events. Do not finalize local accounting before processing the terminal event.
04
Disconnects and cancellation
- Close the upstream reader when the client cancels.
- Retry only requests for which the client received no useful bytes.
- Keep
X-GYW-Request-IDfor diagnostics.