Streaming and citations¶
Retrieved sources¶
Every RAG completion returns the retrieved sources as chunks. Each chunk has these fields:
| Field | Meaning |
|---|---|
reference_index |
The source number used in citation markers, counted from 0 |
file_id |
ID of the file the chunk comes from |
filename |
Original file name |
collection_id |
Collection of the file |
text |
The chunk's text |
metadata |
Source details. type is paginated (with page_numbers and page_count), website (with url and hierarchy), image (with url) or plain |
Without streaming, the chunks are in the chunks field of the completion
response. POST /v1/rag/augmentation returns them in the same way.
Citation markers¶
The model is instructed to cite sources with citation markers: [cit-N]
directly after a statement, where N is the reference_index of a chunk. A
statement with several sources gets several markers, for example [cit-0][cit-3].
Markers are part of the model's text, so treat them as untrusted:
- Match them with a pattern such as
\[cit-(\d+)\]. - Check
Nagainst thereference_indexvalues of the response's chunks, and ignore markers without a matching chunk. - Render valid markers as links or footnotes to the source, for example with the file name and page numbers.
RAG Web doesn't validate or remove markers.
Streaming¶
POST /v1/rag/completions with "stream": true responds with
200 text/event-stream. The data chunks arrive in this order:
-
First, the retrieved sources, as one data chunk, before any model output:
data: {"chunks": [{"reference_index": 0, "file_id": "…", "filename": "handbook.pdf", "…": "…"}], "object": "chat.rag.retrieval"}You have all valid source numbers before the first
[cit-N]arrives. -
Then the completion chunks as the provider sends them, in OpenAI's streaming format (
object: "chat.completion.chunk"), ending withdata: [DONE].
Errors during a stream¶
Once the stream has started, the HTTP status 200 is already sent. A failure after
that point arrives as a final data chunk:
Handle this shape on every streamed completion. Errors before the stream starts are ordinary HTTP errors. See Errors.
No server-side conversation state¶
The API stores no conversation. Your application keeps the message history and
sends the whole history in messages on every request. Retrieval uses the last
user message.