Things to know about MCP on AnythingLLM Docker

MCP Server support

AnythingLLM Docker supports Tools loading via MCP Servers. We do not support Resources, Prompts, or Sampling.

Startup sequence

AnythingLLM does not automatically start MCP servers when the container starts to prevent any overloading of resources on boot.

AnythingLLM will automatically start MCP servers when you open the "Agent Skills" page in the AnythingLLM UI or invoke the @agent directive. All MCP servers will be started in the background - subsequent "boots" will then be much faster since the MCP servers will already be running.

If you mark a tool as anythingllm.autoStart: false in your configuration file, it will not be started automatically - you will need to start it manually from the UI.

Command availability

The majority of commands that are required to run a MCP server are available in the AnythingLLM Docker container already.

The base image of AnythingLLM Docker is ubuntu:jammy-20240627.1, so generic Ubuntu commands will be available as the user running the services inside of the container.

Additionally - we have pre-installed the following commands:

• npx
• uv or uvx
• node
• bash

Where is the MCP Server configuration file?

The MCP Server configuration file is located in the plugins/anythingllm_mcp_servers.json file in the AnythingLLM storage directory.

The storage directory is defined by the STORAGE_LOCATION environment variable when you start the AnythingLLM Docker container - see example

Reloading MCP Servers

You can reload MCP Servers on the fly by clicking the "Refresh" button in the "Agent Skills" page. This will reload the MCP Servers from the configuration file and restart them. This does not require you to restart the AnythingLLM Docker container - the currently running MCPs will be killed and rebooted with whatever changes you made to the configuration file.

You can also click "Refresh" to reload the MCP Servers if you are debugging a specific MCP Server.

Starting and stopping MCP Servers

You can start and stop MCP Servers on the fly by clicking the "Start" or "Stop" action via the gear icon in the "Agent Skills" page while selecting the MCP Server you want to start or stop.

This does not require you to restart the AnythingLLM Docker container - the target MCP Server will be started or stopped immediately.

If you wish to stop an MCP Server from automatically starting - see the Autostart prevention section.

How do I add/remove an MCP Server?

Adding an MCP Server

Adding an MCP Server is as simple as adding a new tool to the mcpServers object in the anythingllm_mcp_servers.json file in your AnythingLLM storage directory.

Removing an MCP Server

You can remove an MCP Server by clicking on an MCP Server in the "Agent Skills" page, select the gear icon, and clicking "Delete". Deleting the MCP Server from the UI will remove the MCP Server from the file and kill the process running that MCP Server.

You can also manually remove an MCP Server by removing the object from the mcpServers object in the anythingllm_mcp_servers.json file and clicking "Refresh" in the "Agent Skills" page on the UI afterwards.

Viewing the status of an MCP Server

On the "Agent Skills" page, you can view the status of an MCP Server by clicking on the MCP Server in the list - if there is an error, it will be displayed in the card.

Additionally, you can quickly see the status of all MCP Servers by clicking the "Agent Skills" page and looking at the MCP Servers list.

Debugging MCP Servers

If you are having issues with an MCP Server, you can best debug these by looking at the docker container logs.

Issues installing a tool

Sometimes, an MCP Server will require a tool to be installed via uv tool install xyz. The easiest way to do this is to open a shell into the container and run the command manually. Then you can click "Refresh" in the "Agent Skills" and see if the tool now boots successfully.

Autostart prevention

This property is specific to AnythingLLM only and will have no effect on other tools.

Sometimes, you may want to optionally start an MCP server manually to prevent it from starting automatically and consuming resources.

To do this, AnythingLLM respects the anythingllm.autoStart property in the MCP Server configuration file.

For example, if you want to prevent the face-generator MCP Server from starting automatically, you can set the autoStart property to false in the configuration file.

Any tool that does not have autoStart: false explicitly set will be started automatically when the rest of the MCP servers are started.

{
  "mcpServers": {
    "face-generator": {
      "command": "npx",
      "args": [
        "@dasheck0/face-generator"
      ],
      "anythingllm": {
        "autoStart": false
      }
    },
    "mcp-youtube": {
      "command": "mcp-youtube",
      "args": []
    }
  }
}

Tool persistence

If you stop or delete the AnythingLLM Docker container the libraries cached for the MCP servers will be lost and need to be re-downloaded on first start. Typically, this takes much longer for MCP servers that have a large number of dependencies or build steps and can increase boot times when starting MCP servers.

The also applies to any tools that you may have manually installed to get an MCP server to work - your changes are within the container and will be lost when the container is deleted.

Writing files to the host machine

Often, you may want to write or even read files from the host machine - since the MCP Server runs within the context of the container - you must use this path:

/app/server/storage/...

This path will then be using the STORAGE_LOCATION directory that you defined when you started the AnythingLLM Docker container. From here you can then write and read files to the host machine.

My LLM is not calling my MCP Server!

First, ensure that the MCP Server is running and that the tool is available in the "Agent Skills" page.

Next, your issue is probably the model you are using - this is especially true if you are using a small local model with a limited context window.

Learn more about LLMs with Agent Skills →