Model Inference Hosting
A hosted API that runs a machine learning model (open-weights or custom) and returns predictions on demand — the layer between "I have a model" and "my product can call it."
Inference hosting providers run models (often popular open-weights LLMs, but also custom fine-tuned or non-LLM models) behind an API, handling the GPU provisioning, batching, and scaling that would otherwise be significant infrastructure work to run in-house. The main differentiators are which models are available out of the box, latency/throughput (especially for real-time applications), and whether custom model weights can be deployed rather than only pre-hosted ones. Pricing is typically per-token or per-compute-second, and self-hosting the same open-weights model remains a viable alternative for teams with the GPU capacity and ops capability to run it themselves.
Last verified: