List your company

Batch Inference API

Model hosting and inference · by OpenRelay

Run large offline batch inference jobs at half the per-token rate, with results delivered within 24 hours.

Visit website

What it does

Run large offline batch inference jobs at half the per-token rate, with results delivered within 24 hours.

  • Production GPU inference
  • OpenAI-compatible LLM serving
  • GPU compute rental
  • Self-hosted GitHub Actions runners with GPU access
Category
Model hosting and inference
Users
Teams running AI inference workloads, ML teams needing GPU-enabled CI/CD pipelines, Developers using OpenAI-compatible LLM APIs
Public API
Available

Not yet disclosed: deployment options, platforms, languages and support channels.

Product insights

Evidence for this product in each industry and task.

#5 · classify documentsView ranking →

Documented workflow

“Each workload page ships a runnable JSONL example, model picks with batch pricing, and the prompt patterns that separate a good run from a re-run. LLM-as-a-Judge Batch Evaluation API Run LLM-as-a-judge evaluation over…”

Read product documentation
  • Each workload page ships a runnable JSONL example, model picks with batch pricing, and the prompt patterns that separate a good run from a re-run. LLM-as-a-Judge Batch Evaluation API Run LLM-as-a-judge evaluation over…
    Read source · openrelay.inc ↗

Pricing

Pricing not available

We don't have pricing details for Batch Inference API yet. Ask OpenRelay for a quote.

Security and compliance

As reported by OpenRelay. Ask for audit reports before you buy.

  • SOC 2 Type IIIn progress
  • SOC 2Compliant
  • PCI DSSCompliant
  • TLS 1.3 encryption in transit
  • Encrypted volumes at rest
  • VM-level isolation with separate guest kernel
  • GPU passthrough via VFIO with IOMMU-enforced memory isolation
  • Complete GPU reset between workloads
  • WireGuard VPN tunnels for node-to-control-plane communication
  • Egress filtering
  • DDoS protection at edge proxy layer
  • HSTS enforcement
  • Zero-access architecture (providers cannot access workloads)
  • No data persistence on provider nodes by default
  • Secure wipe of workload data on termination
  • Role-based access control (Owner, Admin, Member)
  • Scoped and rotatable API keys
  • One-time use provisioning tokens with hashed storage
  • Audit logging of authentication and authorization events
  • Network segmentation and firewalls
  • Intrusion detection and monitoring
  • Regular penetration testing
  • Incident response procedures
  • Certificate pinning for control plane communications

Alternatives

See the full ranking

Is this your product?

Claim it to keep details accurate and submit evidence for the next ranking review.

Claim Batch Inference API