Infersec

The Resource Management API lets you manage your Infersec infrastructure programmatically. Create models, configure engines, deploy inference sources, and expose endpoints - all via a REST API authenticated with your Infersec API key.

For the full interactive API reference, see the API documentation.

Authentication

All Resource Management endpoints require an API key. Pass it via the Authorization header or the x-api-key header:

Authorization: Bearer <your-api-key>
x-api-key: <your-api-key>

Create API keys in the console under Administration - API Keys.

Base URL

https://api.infersec.ai

Models

Models are LLM definitions sourced from HuggingFace. They define which weights are available for inference.

List models

GET /api/v1/models

Returns all models in your account.

Create a model

POST /api/v1/models

Register a new model from HuggingFace:

{
    "name": "My Qwen Model",
    "provider": "huggingface",
    "slug": "barozp/qwen3.6-28b-reap20-a3b-gguf",
    "format": "gguf"
}

The slug must be a fully qualified HuggingFace repo (owner/repo). The format determines which engines can serve the model: gguf -> llama.cpp; exl2/exl3/gptq -> exllamav3; mlx -> mlx-lm; awq/gptq/pytorch/safetensors -> vllm/sglang/tensorrt-llm.

Retrieve a model

GET /api/v1/models/:modelID

Update a model

PATCH /api/v1/models/:modelID

Delete a model

DELETE /api/v1/models/:modelID

Engines

An engine holds the execution configuration for a model family. Engine types: llama.cpp, vllm, sglang, tensorrt-llm, exllamav3, and mlx-lm. Runtime settings are provided as extraArgs - raw CLI arguments passed straight to the underlying engine (for example --n-gpu-layers 99 for llama.cpp) - and inference sources reference an engine by ID. Create an engine before creating a source that uses it.

List engines

GET /api/v1/engines

Create an engine

POST /api/v1/engines

Required fields are name and type; optional extraArgs holds raw CLI arguments passed to the engine:

{
    "name": "My llama.cpp engine",
    "type": "llama.cpp",
    "extraArgs": ["--n-gpu-layers 99", "--flash-attn on"]
}

Retrieve an engine

GET /api/v1/engines/:engineID

Update an engine

PATCH /api/v1/engines/:engineID

Delete an engine

DELETE /api/v1/engines/:engineID

Inference Sources

An inference source represents a single compute instance (your hardware) running one model on one engine. Sources are connected to your machine via Conduit. Create the model and engine first, then reference them here by ID.

List sources

GET /api/v1/sources

Create a source

POST /api/v1/sources

{
    "name": "My GPU Server",
    "modelID": "<model-id>",
    "engineId": "<engine-id>",
    "contextLength": 32768,
    "quantizationLabel": "Q4_K_M"
}
  • engineId - the engine to run (created via the Engines endpoints above)
  • modelID - the model to serve
  • contextLength - max context window (model-dependent)
  • quantizationLabel - quantization variant (GGUF only)

Retrieve a source

GET /api/v1/sources/:sourceID

Update a source

PATCH /api/v1/sources/:sourceID

Delete a source

DELETE /api/v1/sources/:sourceID

Inference Endpoints

An inference endpoint exposes a public URL that routes requests to one or more inference sources.

List endpoints

GET /api/v1/endpoints

Create an endpoint

POST /api/v1/endpoints

{
    "name": "Production API",
    "sourceIDs": ["<source-id-1>", "<source-id-2>"],
    "routingMethod": "round-robin",
    "enabled": true
}

Routing methods:

  • first-available - routes to the first online source
  • round-robin - distributes across all online sources

Retrieve an endpoint

GET /api/v1/endpoints/:endpointID

Update an endpoint

PATCH /api/v1/endpoints/:endpointID

Delete an endpoint

DELETE /api/v1/endpoints/:endpointID

Using endpoints

Once an endpoint is created and enabled, it exposes OpenAI- and Anthropic-compatible APIs. The model name is always default.

# OpenAI compatible
POST /api/inferencing/:endpointID/oai/v1/chat/completions
POST /api/inferencing/:endpointID/oai/v1/completions
POST /api/inferencing/:endpointID/oai/v1/embeddings
GET  /api/inferencing/:endpointID/oai/v1/models

# Anthropic compatible
POST /api/inferencing/:endpointID/anthropic/v1/messages

See Getting Started for full usage examples with various clients.

MCP tool service

Tool services are configured separately from inference endpoints (an endpoint may reference a tool service). Individual MCP tool invocations are executed through a dedicated route, scoped to the tool service rather than the endpoint:

POST /api/tool/service/:toolServiceID/mcp

The request and response bodies are passthrough JSON; the API key authorizes the call.

Full reference

For complete request/response schemas and the interactive try-it-out console, visit the API documentation.