Google Cloud has published a detailed guide on the four different ways developers can serve open large language models including Llama, DeepSeek, Mistral, and Qwen on its platform, ranging from fully serverless managed APIs to fully custom, self-built containers.
Why Choice of Serving Option Matters
Gemini Enterprise Agent Platform offers multiple ways to serve open models, and each option provides high availability along with Google Cloud’s security best practices by default. The right choice depends on factors like how much infrastructure control a team needs, data residency requirements, cost sensitivity, and whether a model needs custom pre- or post-processing logic.
1. Model as a Service (MaaS)
The simplest option is Model as a Service, where open models are served using serverless, managed APIs, meaning there’s no infrastructure to provision or manage. Requests continue to go to standard Gemini Enterprise Agent Platform endpoints, and the underlying models can be discovered and deployed directly through Model Garden. This is the fastest path to production for teams that want to call an open model the same way they’d call any other managed API.
2. Self-Deployed Models in Model Garden
For teams that need more control, self-deployed models let developers deploy open models with one-click deployment or custom weights. Unlike MaaS offerings, which are serverless and don’t require manual deployment, self-deployed models run securely within a developer’s own Google Cloud project and VPC network, giving full control over the deployment environment.
Model Garden also helps developers purchase and manage licenses for proprietary partner models available as a self-deploy option through Cloud Marketplace, with the option to deploy on-demand hardware or use existing Compute Engine reservations and committed-use discounts to manage cost. This option suits use cases with strict data residency or compliance requirements that rule out serverless, multi-tenant infrastructure.
3. Prebuilt Container Images
The third option covers prebuilt container images for popular open-source serving frameworks. Gemini Enterprise Agent Platform provides prebuilt containers for frameworks like vLLM, Hex-LLM, and TGI, letting teams serve open models without having to build and maintain their own serving stack from scratch, while still deploying within their own project environment.
4. Custom vLLM Container
The most flexible and most hands-on option is building a fully custom vLLM container. This route is aimed at situations where existing serving options and prebuilt containers aren’t sufficient, giving developers full control over the container image, including all dependencies and configurations, along with support for custom pre-processing or post-processing logic that prebuilt containers don’t support.
Choosing the Right Option
Google frames the four options as a spectrum rather than a hierarchy: MaaS trades control for speed and simplicity, self-deployed models and prebuilt containers sit in the middle by balancing control with reduced setup effort, and fully custom containers offer maximum flexibility for teams with specialized infrastructure or compliance needs. Regardless of which path a team picks, they can rely on Google Cloud’s security best practices being applied by default.
