Windows Installation

Install using the installation file

1. Download the installer for your hardware

Downloading the installation file

Here is the download link for the latest version of Anything LLM Windows.

Installing the application

After downloading the windows exe installer for AnythingLLM, you can double-click the installer and it will display the installation process.

After which, now the regular installer can run to install AnythingLLM Desktop!

Click on the application name "AnythingLLM" on your desktop to boot up AnythingLLM!

your first boot may take a minute or two to complete on some systems, but subsequent boots will be near instant.

Troubleshooting

"AnythingLLM cannot be closed" when trying to install

If you are trying to install AnythingLLM and you are getting an error that says "AnythingLLM cannot be closed" during installation the following steps will solve your issue.

1. You previously installed AnythingLLM and it is still running in the background.

• Since v1.11.0, AnythingLLM minimizes to the system tray when closed. Check your system tray for the AnythingLLM icon and click on it. "Quit AnythingLLM" to close it.
• You can then click on "Retry" and the installation will continue.

2. You previously installed AnythingLLM for all users instead of just the current user.

• Prior to v1.11.0, AnythingLLM was allowed to be installed for all users. This is no longer the case - this can cause issues during installation since you must run the installer as administrator to uninstall programs from your primary drive.
• To fix this, you must uninstall AnythingLLM for all users and then reinstall for the current user only. Right- Click on the AnythingLLM Desktop icon and click "Open File Location" to open the installation folder.
• Click on the "Uninstall AnythingLLM.exe" file and run it. You may or may not have to run it as administrator.
• After uninstalling, you can then reinstall for the current user only.

Performing the above steps WILL NOT remove your data or settings. You can do this safely without losing any of your data.

Failed to install/extract GPU support from CDN

During installation, we pull in the vendor specific GPU support from our CDN so that you can leverage your GPU for local LLM inferencing for an optimal experience.

Depending on your network connection, firewall, device settings, or other factors, you may encounter an error during installation or extraction of the GPU support where it fails to download or extract the files. Our CDN is globally accessible, so it is extremely unlikely that the issue is with our CDN.

Before attempting to fix the issue, be sure to check your network connection, firewall, device settings, or other factors that may be blocking the download or extraction of the files. You can relaunch the installer and try again. If the error contines, open the ollama_install.log file in the %APPDATA%\Local\Programs\AnythingLLM folder and check for any errors but also the version of Ollama that is being installed.

From the ollama_install.log file, you will see the version of Ollama that is being installed. Keep this version number handy for the next steps.

If you encounter this error, you can try the following steps to fix it:

1. Download the GPU support files manually

• Required base binaries: https://cdn.anythingllm.com/support/ollama/<version>/bins.7z

if you have an NVIDIA GPU, you can download the CUDA support from the following link:

• Ollama Cudav13: https://cdn.anythingllm.com/support/ollama/<version>/cudav13.7z
• Ollama CUDAv12: https://cdn.anythingllm.com/support/ollama/<version>/cudav12.7z

if you have an AMD GPU, you can download the ROCm support from the following link:

• Ollama ROCm: https://cdn.anythingllm.com/support/ollama/<version>/rocm_vulkan.7z

2. Extract the bins.7z folder

• Open the storage location for AnythingLLM, typically %APPDATA%\Roaming\anythingllm-desktop\storage\engines\ollama
• In this folder you should see llm.exe
• Create and open the following folder: %APPDATA%\Roaming\anythingllm-desktop\storage\engines\ollama\lib\ollama
• Unzip the bins.7z file into this folder.
• You should now see several DLL files directly in this folder.

3. Extract relevant GPU support files

• If you have an NVIDIA GPU, you can extract the cudav13 or cudav12 folder into the %APPDATA%\Roaming\anythingllm-desktop\storage\engines\ollama\lib\ollama folder.
• If you have an AMD GPU, you can extract the rocm_vulkan.7z file into the %APPDATA%\Roaming\anythingllm-desktop\storage\engines\ollama\lib\ollama folder.

Your explorer should now look like this:

%APPDATA%\Roaming\anythingllm-desktop\storage\engines\ollama
├── lib
│   └── ollama
│       ├── ggml-base.dll
│       ├── ggml-cpu-**.dll (several DLL files from bins.7z)
│       ├── cudav13/ (if you have an NVIDIA GPU)
│       │   └── cublas64_11.dll (several DLL files from cudav13.7z)
│       ├── cudav12/ (if you have an NVIDIA GPU)
│       │   └── cublas64_11.dll (several DLL files from cudav12.7z)
│       ├── rocm/ (if you have an AMD GPU)
│       │   └── **.dll (files and folders)
│       └── vulkan/ (if you have an AMD GPU)
│           └── vulkan_layer.dll (several DLL files from rocm_vulkan.7z)
└── llm.exe

4. Create a new dot-file in the %APPDATA%\Roaming\anythingllm-desktop\storage\engines\ollama folder called .ollama-version (no extension!) and add the following content:

• <version> eg: (0.13.0, 0.12.0, etc.)

5. Restart the AnythingLLM desktop app.

• Delete all the 7z files from the %APPDATA%\Roaming\anythingllm-desktop\storage\engines\ollama folder as they are no longer needed.
• You should now be able to use your GPU for local LLM inferencing.