@agent Overhaul & streaming ⚡️️

When anythingllm first launched, the word "agent" was not in the vocabulary of the LLM world. Agents are quickly becoming the standard for building AI applications and also the core expierience for interacting with LLMs.

For too long, due to the complexity of building agents, spotty tool call support, models that cant even use tools and more nerd stuff we often had to settle an experience that was not really fun to use since 99% of the time you were just looking at at loading spinners waiting for the response.

The new agent experience is now here

• Streams tool calls and responses in real time (all providers, all models, any hardware)
• Agents can now real-time download and ingest files from the web (eg: link to PDF, excel, csv). Anything you would use a document can be read in real time by the agent.

Upcoming:

• Agent real-time API calling without agent flows
• Agent image understanding
• Agent system prompt passthrough + user context awareness
• Realtime file searching cross-platform default skill

Microsoft Local Foundry support 🤖

Are you using Microsoft Foundry Local (opens in a new tab)? We have you covered.

AnythingLLM Desktop now supports a deep integration with Microsoft Foundry Local.

This means you can now use AnythingLLM Desktop to chat with your data on your Microsoft Foundry Local device.

• AnythingLLM will automatically start Microsoft Foundry Local when you start AnythingLLM Desktop, if installed.
• AnythingLLM will automatically unload models for you to keep your system resources free.
• Can pull optimized models based on your system hardware (CPU, GPU, NPU, etc.)

btw, Foundry Local is free and runs on Apple Silicon, Windows (x64 & ARM64), and Linux (x64 & ARM64)! Its worth checking out if you are looking for a local LLM solution.

currently the model selection in AnythingLLM only shows currently downloaded models. So pulling of models still needs to be done via foundry cli

You can download the latest version of Microsoft Foundry Local here (opens in a new tab).

Linux improvements & ARM64 support 🖥️

Linux ARM64 is quickly becoming the most popular architecture for Linux devices and even personal compute devices like the upcoming NVIDIA DGX Spark (opens in a new tab), Framework desktops, and even people who tinker around with ARM based Raspberry Pi devices.

Additionally, we overhauled our Linux Installation guide to make it more user friendly and easier to install.

• Auto created the apparmor rule to allow the app to run without any additional configuration. (Ubuntu only)
• Auto created the .desktop file so the app can be launched from the desktop and pinned to the launcher. (GNOME based desktops only)

Linux x64 and ARM64 now ships with Ollama 🚀

Linux for a long time has been lacking a local LLM support. We are happy to announce that we have now shipped Ollama (0.11.4) with Linux.

This does increase the size of the Linux AppImage, but it is a small price to pay for local LLM support with zero setup or installation required.

Happy chatting!

Major Improvements: 🚀

• All models and providers now support agentic streaming
• Microsoft Foundry Local integration
• Ephemerally scrape any web-resource via agent or uploader

Other still cool, but not major improvements

• Workspace/Thread Tooltips
• Resize chat area on paste in main chat UI
• Web-scraper can now handle URLs with no protocol
• Generic OpenAI Embedder allow artificial delay
• Anthropic computer-use tool updated to newest model and tool version.
• Ollama and LMStudio automatic model context window size detection
• Render HTML live in chat responses
• Update how chats are rendered in chat history viewer
• Youtube transcript improvements for ASR
• Custom HTTP Response timeout for ollama
• New System Prompt variables (workspace.name, workspace.id)
• Generic OpenAI Embedder allow artificial delay
• Report sources in API responses on last chunk in stream via developer API
• Add user agent to Generic OpenAI requests
• Patch folder GET request response code for developer API
• CometAPI integration
• Portuguese translations
• Export JSON/JSONL with attachments from Workspace Chats viewer

Bug Fixes:

• Upgraded core Electron version
• Migrated OpenAI inteface to Responses API
• Fixed orphan docs bug with filenames that have spaces being pruned
• Update UI icons to be normalized in spacing under chat messages
• PGVector metadata sanitization to prevent bad byte in jsonb vector metadata field
• Fix Dell Pro AI Studio default URL

Deprecated Feature Notices:

• NVIDIA NIM is being phased out of AnythingLLM Desktop starting with v.1.9.0 and will be removed in the next version or patch.

Pinned Download Links

Revision 1.9.0:

• Mac (x86_64) Download (opens in a new tab)
• Mac (Apple Silicon) Download (opens in a new tab)
• Windows Download (opens in a new tab)
• Windows (ARM) Download (opens in a new tab)
• Linux (x86_64) Download (opens in a new tab)
• Linux (ARM64) Download (opens in a new tab)