Conduit is the self-hostable agent that bridges your local hardware to the Infersec cloud. It downloads model files, manages the LLM engine lifecycle, and proxies inference requests. Conduit runs as a lightweight Node.js process or as a Docker container on your machine.

The quickest way to run Conduit is via npx:

npx @infersec/conduit inference start \
  --engine <engine> \
  --key <your-api-key> \
  --source <source-id>

On Linux (x64) and Apple Silicon Macs you can also install Conduit as a standalone binary - no NodeJS required:

curl -fsSL https://infersec.ai/conduit.sh | bash

The installer detects your platform, downloads the binary from files.infersec.ai, verifies its checksum, and runs models fit to check your hardware. Extra flags pass straight through to the command (e.g. bash -s -- --json). The binary is cached under ~/.cache/infersec/conduit/<version>/ and reused on later runs. Unsupported platforms (Intel Macs, Linux arm64) exit with an error directing you to npx @infersec/conduit instead.

For Docker use, see Docker below. For pre-built images that bundle Conduit with vLLM, see Engine Docker Images.

Commands

Conduit exposes four top-level commands:

inference start

Starts the inference agent against a single Infersec source.

tool connect <toolID>

Connects a local tool to Infersec so it can be invoked by server-side tool calling. The tool type is resolved from the server, so the same command connects either a Local Filesystem or Local MySQL tool. See Tools for the full list and what each tool exposes.

models list / models clear

Manage cached model files on disk (the models/ directory under --root).

models fit

Detects your hardware and suggests models that will run on it. This is the command behind the installer at https://infersec.ai/conduit.sh.

benchmark run

Runs benchmarks from a benchmarks.json config file. Typically used by the Infersec team to populate Recommended Models; most users do not need this.

Environment variables

Every CLI flag has a corresponding environment variable. Flags override env vars when both are provided.

Inference

Tool connect

Docker

When running Conduit in Docker, pass the same environment variables via -e flags (or an env file). The model cache and engine state should be mounted as a volume so they survive container restarts.

docker run -d \
  --name infersec-conduit \
  -p 9600:9600 \
  -v ~/.cache/infersec/iagent:/data/iagent \
  -e ENGINE=llama.cpp \
  -e API_KEY=your-api-key \
  -e SOURCE=your-source-id \
  -e ROOT_DIRECTORY=/data/iagent \
  node:24 \
  npx @infersec/conduit inference start

The relevant env vars in a container context are:

For pre-built images that bundle Conduit with vLLM for specific hardware (AMD ROCm, NVIDIA CUDA, NVIDIA DGX Spark), see Engine Docker Images.