Skip to content

Streaming and citations

Retrieved sources

Every RAG completion returns the retrieved sources as chunks. Each chunk has these fields:

Field Meaning
reference_index The source number used in citation markers, counted from 0
file_id ID of the file the chunk comes from
filename Original file name
collection_id Collection of the file
text The chunk's text
metadata Source details. type is paginated (with page_numbers and page_count), website (with url and hierarchy), image (with url) or plain

Without streaming, the chunks are in the chunks field of the completion response. POST /v1/rag/augmentation returns them in the same way.

Citation markers

The model is instructed to cite sources with citation markers: [cit-N] directly after a statement, where N is the reference_index of a chunk. A statement with several sources gets several markers, for example [cit-0][cit-3].

Markers are part of the model's text, so treat them as untrusted:

  • Match them with a pattern such as \[cit-(\d+)\].
  • Check N against the reference_index values of the response's chunks, and ignore markers without a matching chunk.
  • Render valid markers as links or footnotes to the source, for example with the file name and page numbers.

RAG Web doesn't validate or remove markers.

Streaming

POST /v1/rag/completions with "stream": true responds with 200 text/event-stream. The data chunks arrive in this order:

  1. First, the retrieved sources, as one data chunk, before any model output:

    data: {"chunks": [{"reference_index": 0, "file_id": "…", "filename": "handbook.pdf", "…": "…"}], "object": "chat.rag.retrieval"}
    

    You have all valid source numbers before the first [cit-N] arrives.

  2. Then the completion chunks as the provider sends them, in OpenAI's streaming format (object: "chat.completion.chunk"), ending with data: [DONE].

Errors during a stream

Once the stream has started, the HTTP status 200 is already sent. A failure after that point arrives as a final data chunk:

data: {"detail": "An error occurred."}

Handle this shape on every streamed completion. Errors before the stream starts are ordinary HTTP errors. See Errors.

No server-side conversation state

The API stores no conversation. Your application keeps the message history and sends the whole history in messages on every request. Retrieval uses the last user message.