Skip to content

Run a local instance

This page runs the whole platform on your own machine with Docker Compose. It's the same stack an operator installs, with throwaway passwords.

What you need

  • Docker with the Compose plugin
  • bash, curl, openssl and python3 for the helper scripts
  • Access to an OpenAI-compatible inference provider, for example KI:Connect. You will need:
    • the base URL (e.g. https://chat.kiconnect.nrw/api/v1/)
    • the token
    • an embedding model that returns 4096-dimensional vectors
    • a generation model. Choose a generation model that can read images, because the platform uses it to process PDF pages and images.

Get the deployment files

git clone --branch 1.1.0 https://gitlab.git.nrw/ki-expertisezentrum.nrw/apps/rag/rag-docker.git
cd rag-docker
cp example.env .env

Run every command on this page in the rag-docker directory.

Configure

Set these in .env. The credentials are throwaway values that only have to match each other. Replace the uppercase placeholders with your provider's values:

POSTGRES_USER=rag
POSTGRES_PASSWORD=rag
POSTGRES_DB=gateway
DB_USER=rag
DB_PASSWORD=rag
MB_USER=rag
MB_PASSWORD=rag
FilesOptions__Token=local-dev

EndpointOptions__Base=PROVIDER_BASE_URL
EndpointOptions__Token=PROVIDER_TOKEN
EndpointOptions__ChatModelVersion=GENERATION_MODEL
DEFAULT_EMB_BASE_URL=PROVIDER_BASE_URL
DEFAULT_EMB_MODEL_VERSION=EMBEDDING_MODEL
DEFAULT_EMB_TOKEN=PROVIDER_TOKEN
DEFAULT_GEN_BASE_URL=PROVIDER_BASE_URL
DEFAULT_GEN_MODEL_VERSION=GENERATION_MODEL
DEFAULT_GEN_TOKEN=PROVIDER_TOKEN

PROVIDER_BASE_URL must end with /, for example https://provider.example/api/v1/.

Use your provider's /models endpoint to find the model versions (=model-id).

RAG Web ignores EndpointOptions__ChatModelVersion, but the smoke test below uses it as the model name. The configuration reference explains every variable.

RAG Web listens on port 8000 of your machine. If your own application already uses that port, change WEB_GATEWAY_HOST_PORT.

Start

docker compose up -d

The first start pulls the images, which takes a minute or two.

Check that it works

utilities/smoke.sh --with-completion

The smoke test creates its own tenant and API key, uploads a small text file, waits until it's processed and asks your generation model a question about it. It ends with smoke test passed. Its tenant and file stay in the databases, which doesn't matter on a local instance.

Create your API key

Locally you can choose the API key yourself. The tenant ID can be any UUID. For convenience, you can just execute this:

TENANT=3f6c1a52-8d0e-4b7a-9c1d-2e5f8a7b6c40
docker compose exec -T gateway-database psql -U rag -d gateway \
  -c "INSERT INTO identity_tenants (id, name) VALUES ('$TENANT', 'local-dev');"
utilities/create-api-key.sh --tenant "$TENANT" --description local-dev \
  --sql-only local-dev-key-change-me \
  | docker compose exec -T gateway-database psql -U rag -d gateway

Now follow the API quickstart with:

  • BASE_URL: http://localhost:8000
  • API_KEY: local-dev-key-change-me

RAG Web finds a key by its first 12 characters. If you create more keys this way, make them differ in those characters.

Keep this instance to yourself

The example key is printed on this page, and Docker publishes the port on all network interfaces. Anyone who can reach your machine can then use your provider token through this instance. If other machines can reach yours, leave out --sql-only and the key, and follow Tenants and API keys to create a random one.

Stop and reset

docker compose down

The data stays in the directories that .env names, so the next docker compose up -d still has your tenant, key and files. To start from scratch, delete them after docker compose down and create them again empty. On Linux, Docker creates their contents as root, so deleting needs sudo:

sudo rm -rf postgres/data postgres-rag/data rabbitmq-rag/data temp_data
mkdir -p postgres/data postgres-rag/data rabbitmq-rag/data temp_data
docker compose up -d

Then create your API key again.