vLLM

The leading open-source LLM serving engine, known for continuous batching and paged KV-cache management. The default answer to 'how do we self-host this model'.