RAG (retrieval-augmented generation) lets an inference endpoint answer questions using your own documents. Before each request, Infersec searches a vector database for content relevant to the user's message and injects it into the prompt, so the model grounds its answer in your data instead of guessing.

The flow has two halves:

• Ingest - a Conduit agent reads local files, splits them into chunks, generates embeddings using one of your inference sources, and stores everything in a vector database collection.
• Retrieval - at request time, the API embeds the user's latest message, searches the same collection, and prepends the top matches to the prompt.

Prerequisites

RAG needs an embeddings model exposed through an online inference source. This is a model with the embeddings task type (for example nomic-ai/nomic-embed-text-v1.5-GGUF), not a chat model. You need:

• An embeddings model registered in the console (Inferencing - Models, task type embeddings)
• An inference source running that model, online via a Conduit agent
• For ingest: an inference endpoint attached to that source - the ingest job references this endpoint

Keep the embedding model consistent between ingest and retrieval. The collection's vectors are only comparable if queries are embedded by the same model (and quantization) that produced the indexed chunks.

Vector database

Infersec speaks the Qdrant REST API, and Qdrant is the recommended vector database. It is simple to self-host next to your stack:

docker run -d --name qdrant -p 6333:6333 -v qdrant-data:/qdrant/storage qdrant/qdrant

Register it in the console under Data - Vector Databases: give it a name and set the endpoint to your Qdrant URL (for example http://localhost:6333).

Reachability matters: the API service performs the search at request time, so the endpoint must be reachable from the API containers, not just from your browser. When the API runs in Docker and Qdrant runs on the host, use http://host.docker.internal:6333 (Linux hosts need extra_hosts: host.docker.internal:host-gateway in the API's compose service, and a 127.0.0.1 host.docker.internal entry in the host's /etc/hosts if a host-run Conduit also uses the same URL).

If the collection requires authentication, store the API key as a secret and attach it to the vector database record.

Ingesting files

Create an ingest job in the console under Data - Ingest Jobs. The job defines:

Then run the ingest from the machine holding the files, using the command shown on the ingest job's detail page:

npx @infersec/conduit ingest files \
  --api-url <api-url> \
  --key <your-api-key> \
  --accept "*.{pdf,html,htm,txt,md,json,csv}" \
  --recursive \
  --path <path-to-your-corpus> \
  --target <ingest-job-id>

Useful flags:

• --accept - glob of file types to pick up (the job's configured file types also filter the discovery list)
• --watch - keep the agent running and re-ingest files as they change
• --no-ocr - skip OCR fallback for scanned PDFs

Re-running an ingest is safe: chunk point IDs are deterministic per file, so unchanged chunks are overwritten rather than duplicated.

Supported file types

Attaching RAG to an endpoint

Open an inference endpoint's edit page and fill in the RAG Retrieval panel:

Each retrieval- augmented request records a rag_query charge under pay-as-you-go billing.