Cloud
Inference hosting
Deploy your custom models as autoscaling endpoints.
Supported model formats
- HuggingFace transformers (safetensors, pytorch_model.bin)
- ONNX
- TensorRT engines (for maximum throughput)
- Custom Docker images with your own runtime
Deploy an endpoint
bash`xobriq deploy \ --name my-llm \ --source hf://mistralai/Mistral-7B-v0.3 \ --gpu h200-1x \ --autoscale 1-8`
Autoscaling behaviour
Endpoints scale by concurrency and queue depth. Cold-start for common LLMs is ~15s. Set min-replicas to 1 to avoid cold starts entirely.