Batch Inference API
Run large offline batch inference jobs at half the per-token rate, with results delivered within 24 hours.
What it does
Run large offline batch inference jobs at half the per-token rate, with results delivered within 24 hours.
- Production GPU inference
- OpenAI-compatible LLM serving
- GPU compute rental
- Self-hosted GitHub Actions runners with GPU access
- Category
- Model hosting and inference
- Users
- Teams running AI inference workloads, ML teams needing GPU-enabled CI/CD pipelines, Developers using OpenAI-compatible LLM APIs
- Public API
- Available
Not yet disclosed: deployment options, platforms, languages and support channels.
Pricing
Pricing isn't published
OpenRelay doesn't list prices publicly. Ask them for a quote.
Security and compliance
As reported by OpenRelay. Ask for audit reports before you buy.
- SOC 2 Type IIIn progress
- SOC 2Compliant
- PCI DSSCompliant
- TLS 1.3 encryption in transit
- Encrypted volumes at rest
- VM-level isolation with separate guest kernel
- GPU passthrough via VFIO with IOMMU-enforced memory isolation
- Complete GPU reset between workloads
- WireGuard VPN tunnels for node-to-control-plane communication
- Egress filtering
- DDoS protection at edge proxy layer
- HSTS enforcement
- Zero-access architecture (providers cannot access workloads)
- No data persistence on provider nodes by default
- Secure wipe of workload data on termination
- Role-based access control (Owner, Admin, Member)
- Scoped and rotatable API keys
- One-time use provisioning tokens with hashed storage
- Audit logging of authentication and authorization events
- Network segmentation and firewalls
- Intrusion detection and monitoring
- Regular penetration testing
- Incident response procedures
- Certificate pinning for control plane communications
Reviews
No reviews yet
Reviews come from people who use Batch Inference API at work. Share how your team uses it to help other buyers decide.
Alternatives
MAGIC
by Pipeshift
Proprietary framework (Modular Architecture for GPU Inference Clusters) that compiles workload-specific inference pipelines, controlling layers from model to silicon including K...
Coding Platform (Kimchi)
by Cast AI
Enterprise AI coding and inference platform.
Red Hat AI Inference
by Red Hat
Optimize model performance with an integrated stack for fast, consistent, and cost-effective inference at scale.
Inference Endpoints
by VESSL AI
Coming soon. OpenAI & Anthropic-compatible APIs; pay only for the tokens you use.
QHexRT
by RunAnywhere, Inc.
An inference engine designed to run LLMs, VLMs, STT, TTS, and embeddings entirely on Qualcomm Hexagon NPUs.
Dedicated Model Inference
by Together AI
Deploy models on dedicated infrastructure.
Is this your product?
Claim it to keep details accurate and submit evidence for the next ranking review.
Claim Batch Inference API