[02/A] Cloud
Managed cloud
The managed platform. Your machines run the models; we run the routing, metering and control plane. Pay only for what flows through.
01
Bring-Your-Own Sources
Keep hardware wherever it lives; conduit connects outward, so no inbound firewall holes needed.
02
Linux + macOS
Conduit ships prebuilt Linux x64 and Apple Silicon macOS binaries, with npm and Docker images covering everything else.
03
Source Groups
Auto-grouped compute configs: pools of sources — including ephemeral ones — routed and scaled as a single unit.
04
Multi-Engine Support
Conduit drives vLLM, llama.cpp, MLX-LM, SGLang, TensorRT-LLM, ExLlamaV3 and custom OpenAI-compatible engine servers.
05
Model Fit Check
Conduit probes your GPUs, VRAM and RAM to show which models fit your machines, with AI-powered suggestions for your hardware.
06
Open Weight Model Catalog
A curated catalog of open weight models, sized against real hardware — from a 230M phone-class model to 284B flagships.
07
Endpoint Authentication
Per-endpoint API keys with scoped authentication: separate credentials for every consumer, revocable at any time.
08
Compatible APIs
OpenAI- and Anthropic-compatible REST surfaces. Existing agents, SDKs and CLIs work unchanged.
09
REST Management API
Scriptable CRUD over endpoints, sources, models and engines — manage the platform from scripts, pipelines or any HTTP client.
10
Decision Models
Serve decision models alongside LLMs: one state in, calibrated typed answers out — choices, scores and booleans in a single forward pass, zero generated tokens.
11
MCP Endpoints
Expose your own MCP tool services: agents connect once and call tools per request, metered individually.
12
MCP Proxying
Proxy third-party MCP servers through your account with authentication, quotas and per-call accounting.
13
Server-Side Tool Calling
The platform intercepts and executes tool calls server-side, so even basic clients get full agentic capabilities.
14
Document Ingestion
Parse PDF, DOCX, HTML, Markdown, CSV and JSON documents into RAG attachments — with OCR fallback and watched directories that ingest as files change.
15
Server-Side RAG
Retrieval-augmented generation over your own documents, executed inside the request path.
16
Hardware Telemetry
Live machine telemetry in the console: CPU, RAM, GPU model, VRAM usage, temperature and driver versions for every connected machine.
17
Open Telemetry
Open Telemetry metrics and traces for prompts, tokens and tool calls — feed them straight into your observability stack.
18
Auditing
Attach audits to inference endpoints and capture requests, tool calls, auth failures and source state. The cloud retains 90 days; self-hosted exports daily extracts to a volume you control.
19
Console
The full control panel: sources, models, endpoints, tool services and spend, from one place.
20
Credit Billing
Top up via Stripe, deduct in real time. No subscriptions, no minimums — metered infrastructure rates.
Pay for what flows
Credit-based and real time. Connected sources bill per hour; inference bills per token and per tool call.
Connect your first source
Install conduit on a machine with an engine server, register it in the console and point your stack at the API.