Google Vertex AI LLM
Google Vertex AI (opens in a new tab) lets you run Gemini (and other Model Garden) models through your own Google Cloud project. Usage is billed to your GCP project and is covered by Google Cloud's data governance terms (opens in a new tab) - your prompts and responses are not used to train models.
Vertex AI vs Google Gemini?
Both providers serve the same Gemini models. Use Vertex AI if your organization runs on Google Cloud and you want billing, quotas, and access control managed through your GCP project. If you just want a simple API key with a free tier, use the Google Gemini provider instead.
Connecting to Google Vertex AI
Valid Google Cloud project required!
You must have a Google Cloud (opens in a new tab) project with billing enabled to use Vertex AI - there is no free tier for this service.
Obtain an API key for Vertex AI
AnythingLLM connects to Vertex AI with a project-bound Vertex AI API key (opens in a new tab).
- Create or select a project in the Google Cloud console (opens in a new tab) and make sure billing is enabled (opens in a new tab) for it.
- Enable the Vertex AI API (opens in a new tab) for the project.
- Go to APIs & Services → Credentials (opens in a new tab) and click Create credentials → API key.
- (Recommended) Restrict the key (opens in a new tab) to the Vertex AI API so it cannot be used for anything else.
Paste this key into the Vertex AI API Key field in AnythingLLM.
API key creation blocked?
Some Google Cloud organizations disable API key creation by policy. If the API key option is missing or errors, ask your organization admin to allow it via the organization policy for API keys (opens in a new tab), or create the key from a project outside the restricted organization.
GCP Project ID
The Project ID of the Google Cloud project your API key belongs to. You can find it on the console dashboard (opens in a new tab) or in the project picker - note that the project ID is not always the same as the project name.
Region
The Google Cloud region (opens in a new tab) your requests are processed in. The default global endpoint is recommended - it offers the best availability and supports all Gemini models. Pick a specific region only if your organization requires data processing to stay in a particular location, and confirm your model is available in that region (opens in a new tab).
Chat Model Selection
The model dropdown is pre-populated with the Gemini models available on Vertex AI. If the model you want is not listed - for example a Model Garden (opens in a new tab) partner model - choose the manual entry option and type the full model ID (eg: google/gemini-2.5-pro or a publisher-prefixed ID like meta/llama-4-maverick-17b-128e-instruct-maas).
Model context window
Only needed for manually entered models. This is the maximum number of tokens for a single request - consult the model's documentation for its limit. Models selected from the dropdown have their context window detected automatically.