Run a local instance¶
This page runs the whole platform on your own machine with Docker Compose. It's the same stack an operator installs, with throwaway passwords.
What you need¶
- Docker with the Compose plugin
bash,curl,opensslandpython3for the helper scripts- Access to an OpenAI-compatible inference provider, for example KI:Connect. You will need:
- the base URL (e.g.
https://chat.kiconnect.nrw/api/v1/) - the token
- an embedding model that returns 4096-dimensional vectors
- a generation model. Choose a generation model that can read images, because the platform uses it to process PDF pages and images.
- the base URL (e.g.
Get the deployment files¶
git clone --branch 1.1.0 https://gitlab.git.nrw/ki-expertisezentrum.nrw/apps/rag/rag-docker.git
cd rag-docker
cp example.env .env
Run every command on this page in the rag-docker directory.
Configure¶
Set these in .env. The credentials are throwaway values that only have to
match each other. Replace the uppercase placeholders with your provider's values:
POSTGRES_USER=rag
POSTGRES_PASSWORD=rag
POSTGRES_DB=gateway
DB_USER=rag
DB_PASSWORD=rag
MB_USER=rag
MB_PASSWORD=rag
FilesOptions__Token=local-dev
EndpointOptions__Base=PROVIDER_BASE_URL
EndpointOptions__Token=PROVIDER_TOKEN
EndpointOptions__ChatModelVersion=GENERATION_MODEL
DEFAULT_EMB_BASE_URL=PROVIDER_BASE_URL
DEFAULT_EMB_MODEL_VERSION=EMBEDDING_MODEL
DEFAULT_EMB_TOKEN=PROVIDER_TOKEN
DEFAULT_GEN_BASE_URL=PROVIDER_BASE_URL
DEFAULT_GEN_MODEL_VERSION=GENERATION_MODEL
DEFAULT_GEN_TOKEN=PROVIDER_TOKEN
PROVIDER_BASE_URL must end with /, for example https://provider.example/api/v1/.
Use your provider's /models endpoint to find the model versions (=model-id).
RAG Web ignores EndpointOptions__ChatModelVersion, but the smoke test below
uses it as the model name. The configuration reference
explains every variable.
RAG Web listens on port 8000 of your machine. If your own application already
uses that port, change WEB_GATEWAY_HOST_PORT.
Start¶
The first start pulls the images, which takes a minute or two.
Check that it works¶
The smoke test creates its own tenant and API key, uploads a small text file,
waits until it's processed and asks your generation model a question about it.
It ends with smoke test passed. Its tenant and file stay in the databases,
which doesn't matter on a local instance.
Create your API key¶
Locally you can choose the API key yourself. The tenant ID can be any UUID. For convenience, you can just execute this:
TENANT=3f6c1a52-8d0e-4b7a-9c1d-2e5f8a7b6c40
docker compose exec -T gateway-database psql -U rag -d gateway \
-c "INSERT INTO identity_tenants (id, name) VALUES ('$TENANT', 'local-dev');"
utilities/create-api-key.sh --tenant "$TENANT" --description local-dev \
--sql-only local-dev-key-change-me \
| docker compose exec -T gateway-database psql -U rag -d gateway
Now follow the API quickstart with:
BASE_URL:http://localhost:8000API_KEY:local-dev-key-change-me
RAG Web finds a key by its first 12 characters. If you create more keys this way, make them differ in those characters.
Keep this instance to yourself
The example key is printed on this page, and Docker publishes the port on
all network interfaces. Anyone who can reach your machine can then use
your provider token through this instance. If other machines can reach
yours, leave out --sql-only and the key, and follow
Tenants and API keys to create a random one.
Stop and reset¶
The data stays in the directories that .env names, so the next
docker compose up -d still has your tenant, key and files. To start from
scratch, delete them after docker compose down and create them again empty.
On Linux, Docker creates their contents as root, so deleting needs sudo:
sudo rm -rf postgres/data postgres-rag/data rabbitmq-rag/data temp_data
mkdir -p postgres/data postgres-rag/data rabbitmq-rag/data temp_data
docker compose up -d
Then create your API key again.