Running Hermes Agent Locally with Lemonade Server

Overview

Hermes Agent is a self-improving AI agent built by Nous Research. It has a built-in learning loop, it creates skills from experience, builds a persistent memory of who you are across sessions, and can run scheduled automations on your behalf. Unlike a simple chat assistant, Hermes takes real actions: running shell commands, writing files, browsing the web, and delegating parallel workstreams to subagents.

Lemonade Server is the local inference backend that powers it. It is an open-source server that runs GenAI models directly on your AMD hardware and exposes them through the industry-standard OpenAI API.

Together they form a fully local AI agent stack: Lemonade handles model inference on your GPU, and Hermes provides the agent loop, memory, skills, and messaging gateway.

Before you continue: Hermes Agent is a highly autonomous AI agent. Giving any AI agent access to your system may result in unpredictable or unintended outcomes. Proceed only if you understand the risks and are comfortable with autonomous software acting on your behalf.

What You’ll Learn

By the end of this playbook you will be able to:

• Install Hermes Agent and point it at Lemonade Server as its AI backend.
• (Recommended) Enable Docker/Podman sandboxing to isolate the agent’s actions from your host.
• Start the Hermes gateway and confirm your agent is ready.
• Connect a communication channel (Discord or Telegram) so you can chat with your agent from any device.

Setting the Memory Configuration

sudo apt install pipx
pipx ensurepath

pipx install amd-debug-tools

amd-ttm

amd-ttm --set <NUM>

Installing Software Prerequisites

sudo apt-get install -y podman`

sudo apt-get install -y podman

msiexec /i lemonade-server-minimal.msi /qn

sudo add-apt-repository ppa:lemonade-team/stable
sudo apt install lemonade-server

yay -S lemonade-server

lemonade --version

lemonade version x.y.z

Pull and Load the Recommended Model

The recommended model for this playbook is Qwen3.6-35B-A3B-GGUF from Unsloth, a strong MoE model with a 263k-token context window that is well-suited to agent workloads. This model uses UD-Q4_K_XL quantization. Pull it now:

lemonade pull Qwen3.6-35B-A3B-GGUF

Then load it with a large context window and save that setting for future runs:

lemonade unload
lemonade load Qwen3.6-35B-A3B-GGUF --ctx-size 262144 --save-options

The model has a default context length of 262,144 tokens. If you encounter out-of-memory (OOM) errors, consider reducing the context window.

Tip: Disable thinking for faster agent responses: Qwen3.6-35B-A3B runs in thinking mode by default, which adds latency before each response. For agent loops this overhead accumulates quickly. The lemonade-sdk/recipes repo provides a ready-made config that disables thinking. To use it, download the file and import it:

curl -LO https://raw.githubusercontent.com/lemonade-sdk/recipes/main/coding-agents/Qwen3.6-35B-A3B-NoThinking.json
lemonade import Qwen3.6-35B-A3B-NoThinking.json

wsl --install --no-distribution

wsl --install -d Ubuntu-24.04

sudo tee /etc/wsl.conf > /dev/null <<'EOF'
[boot]
systemd=true
EOF

wsl --shutdown
wsl

ip route show default | awk '{print $3}' | head -1

netsh interface portproxy add v4tov4 listenaddress=<WSL-Gateway-IP> listenport=13305 connectaddress=127.0.0.1 connectport=13305

New-NetFirewallRule -DisplayName "Lemonade-WSL" -Direction Inbound -Protocol TCP -LocalPort 13305 -Action Allow

WINDOWS_HOST=$(ip route show default | awk '{print $3}' | head -1)
curl -s "http://$WINDOWS_HOST:13305/api/v1/models"

{
  "data": [
    {
      "checkpoint": "unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL",
      "checkpoints": {
        "main": "unsloth/Qwen3.6-35B-A3B-GGUF:UD-Q4_K_XL"
      },
      "mmproj": "unsloth/Qwen3.6-35B-A3B-GGUF:mmproj-F16.gguf",
      ....
    }
  ],
  "object": "list"
}

Install Hermes Agent

curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash -s -- --skip-setup

The --skip-setup flag skips the interactive setup wizard so you can configure the model backend manually in the next step.

Reload your shell:

source ~/.bashrc

Confirm the installation:

hermes --version

Run a self-diagnostic to check all dependencies:

hermes doctor

Configure Hermes to Use Lemonade

Hermes stores its model configuration in ~/.hermes/config.yaml. You can either use the interactive hermes model picker or can write the config directly.

Option 1: Interactive picker

hermes model

hermes model

When prompted:

1. Select Custom endpoint (enter URL manually)

3. API key: lemonade
4. API compatibility mode: 1 (Auto-detect)
5. Select model: choose Qwen3.6-35B-A3B-GGUF from the list
6. Context length in tokens: 262144
7. Display name: local-lemonade (or any name you prefer)

hermes model saves both the active model selection and a named custom_providers entry that stores the context length alongside the endpoint. The result in ~/.hermes/config.yaml looks like this:

model:
  default: Qwen3.6-35B-A3B-GGUF
  provider: custom
  base_url: http://127.0.0.1:13305/api/v1
  api_key: lemonade
custom_providers:
  - name: local-lemonade
    base_url: http://127.0.0.1:13305/api/v1
    api_key: lemonade
    model: Qwen3.6-35B-A3B-GGUF
    models:
      Qwen3.6-35B-A3B-GGUF:
        context_length: 262144

Option 2: Write config directly

mkdir -p ~/.hermes
cat >> ~/.hermes/config.yaml <<'EOF'
model:
  default: Qwen3.6-35B-A3B-GGUF
  provider: custom
  base_url: http://127.0.0.1:13305/api/v1
  api_key: lemonade
custom_providers:
  - name: local-lemonade
    base_url: http://127.0.0.1:13305/api/v1
    api_key: lemonade
    model: Qwen3.6-35B-A3B-GGUF
    models:
      Qwen3.6-35B-A3B-GGUF:
        context_length: 262144
EOF

WINDOWS_HOST=$(ip route show default | awk '{print $3}' | head -1)

mkdir -p ~/.hermes
cat >> ~/.hermes/config.yaml <<EOF
model:
  default: Qwen3.6-35B-A3B-GGUF
  provider: custom
  base_url: http://$WINDOWS_HOST:13305/api/v1
  api_key: lemonade
custom_providers:
  - name: local-lemonade
    base_url: http://$WINDOWS_HOST:13305/api/v1
    api_key: lemonade
    model: Qwen3.6-35B-A3B-GGUF
    models:
      Qwen3.6-35B-A3B-GGUF:
        context_length: 262144
EOF

(Recommended) Enable Podman Sandboxing

Hermes Agent can route all agent shell and file operations through an isolated container rather than running them directly on your host. This limits the blast radius of any unintended action to the sandbox, leaving your host filesystem and network untouched.

Build a lightweight sandbox image:

podman build -t hermes-sandbox:bookworm-slim - <<'DOCKERFILE'
FROM debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
  bash ca-certificates curl git jq python3 ripgrep \
  && rm -rf /var/lib/apt/lists/*
RUN useradd --create-home --shell /bin/bash sandbox
USER sandbox
WORKDIR /home/sandbox
CMD ["sleep", "infinity"]
DOCKERFILE

wsl -d Ubuntu-24.04

podman build -t hermes-sandbox:bookworm-slim - <<'DOCKERFILE'
FROM debian:bookworm-slim
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y --no-install-recommends \
  bash ca-certificates curl git jq python3 ripgrep \
  && rm -rf /var/lib/apt/lists/*
RUN useradd --create-home --shell /bin/bash sandbox
USER sandbox
WORKDIR /home/sandbox
CMD ["sleep", "infinity"]
DOCKERFILE

Then configure Hermes to use Podman as the container runtime and set the terminal backend:

echo "HERMES_DOCKER_BINARY=/usr/bin/podman" >> ~/.hermes/.env

cat >> ~/.hermes/config.yaml <<'EOF'
terminal:
  backend: docker
  docker_image: hermes-sandbox:bookworm-slim
EOF

The terminal.backend is still docker. HERMES_DOCKER_BINARY is what tells Hermes to use Podman as the runtime instead.

Hermes will now spin up a persistent sandbox container and route all terminal and file-tool calls through it. The container shares the life of the Hermes process, is reused across all tool calls, and is destroyed when Hermes exits.

Verify the sandbox is working: Start Hermes (hermes) and ask it to run hostname - you should see a short container ID instead of your machine’s hostname. You can also ask it to rm -rf <path-to-a-dummy-file/folder>: Hermes will confirm the deletion, but the folder will still be on your host. The command ran inside the container’s isolated $HOME, not yours.

Need stronger isolation? Hermes also provides an official Docker image (nousresearch/hermes-agent) that runs the entire agent process inside a container - gateway, tools, and all. See the Hermes Docker documentation for setup details.

cd ~/.config/systemd/user

nano firecrawl.service

[Unit]
Description=Firecrawl
After=podman.service
Requires=podman.service

[Service]
Type=oneshot
RemainAfterExit=yes
WorkingDirectory=${HOME}/firecrawl

# Optional: Validate config before starting
ExecStartPre=/usr/bin/podman -f hermes-compose.yaml config --quiet

# Start containers in detached mode
ExecStart=/usr/bin/podman compose -f hermes-compose.yaml up -d --remove-orphans

# Stop containers when the service stops
ExecStop=/usr/bin/podman compose -f hermes-compose.yaml down

[Install]
WantedBy=default.target

systemctl --user daemon-reload
systemctl --user enable firecrawl.service

git clone https://github.com/firecrawl/firecrawl.git

# ===== Required ENVS ======
PORT=3002
HOST=0.0.0.0

# ===== Firecrawl =====
# FIRECRAWL_API_KEY=""

# ===== Proxy =====
# PROXY_SERVER can be a full URL (e.g. http://0.1.2.3:1234) or just an IP and port combo (e.g. 0.1.2.3:1234)
# Do not uncomment PROXY_USERNAME and PROXY_PASSWORD if your proxy is unauthenticated
# PROXY_SERVER=
# PROXY_USERNAME=
# PROXY_PASSWORD=

# This key lets you access the queue admin panel. Change this if your deployment is publicly accessible.
BULL_AUTH_KEY=CHANGEME

# ===== System Resource Configuration =====
# Maximum CPU usage threshold (0.0-1.0). Worker will reject new jobs when CPU usage exceeds this value.
# Default: 0.8 (80%)
# MAX_CPU=0.8

# Maximum RAM usage threshold (0.0-1.0). Worker will reject new jobs when memory usage exceeds this value.
# Default: 0.8 (80%)
# MAX_RAM=0.8

podman pull docker.io/nousresearch/hermes-agent:latest

podman compose -f hermes-compose.yaml up -d

podman compose -f hermes-compose.yaml down

systemctl --user start firecrawl.service

systemctl --user stop firecrawl.service

Hermes Native

Start an interactive CLI session directly:

hermes

Congratulations, you’ve built a fully local AI agent stack.

Web Dashboard

Hermes includes a browser-based UI for managing config, API keys, models, sessions, memory, and cron jobs. Open a second terminal while the gateway or CLI is running and launch it with:

hermes dashboard

This starts a local server and opens http://127.0.0.1:9119 in your browser. See the dashboard documentation for the full feature reference.

Optional: Connect a Communication Channel

Once the gateway is running you can reach your local agent from any device. Hermes supports Discord, Telegram, and others

Discord

Discord requires a server where you have administrator access to add a bot. If you share servers but don’t own one, use Telegram instead.

Create a Discord application and bot

1. Go to the Discord Developer Portal and click New Application. Give it a name (e.g. “hermes-bot”).
2. In the sidebar, click Bot. Set a username for the bot.
3. Still on the Bot page, scroll to Privileged Gateway Intents and enable:
   • Message Content Intent (required)
   • Server Members Intent (recommended)
4. Scroll back up and click Reset Token to generate your bot token. Copy it.

Add the bot to your server

1. In the sidebar, click OAuth2 / URL Generator.
2. Under Scopes, enable bot and applications.commands.
3. Under Bot Permissions, enable: View Channels, Send Messages, Read Message History, Embed Links, Attach Files.
4. Copy the generated URL, paste it in your browser, select your server, and confirm.

Collect your IDs and allow DMs

Enable Developer Mode in Discord (User Settings / Advanced / Developer Mode), then:

• Right-click your server icon: Copy Server ID
• Right-click your own avatar: Copy User ID

Right-click your server icon / Privacy Settings / toggle on Direct Messages. This is required for the pairing step.

Configure Hermes for Discord

Add the following to ~/.hermes/.env:

# Required
DISCORD_BOT_TOKEN=your-bot-token
DISCORD_ALLOWED_USERS=your-discord-user-id

Then start the gateway:

hermes gateway

The bot should come online in Discord within a few seconds. Send it a message, either a DM or in a channel it can see.

Telegram

Create a Telegram bot

1. Open Telegram and message @BotFather.
2. Send /newbot and follow the prompts. Save the bot token it gives you.

Configure Hermes for Telegram

Add the following to ~/.hermes/.env:

TELEGRAM_BOT_TOKEN=your-bot-token
TELEGRAM_ALLOWED_USERS=your-telegram-user-id   # comma-separated for multiple users

Don’t know your Telegram user ID? Message @userinfobot in Telegram, it will reply with your numeric ID.

Then start the gateway:

hermes gateway

Send your bot any message in Telegram to test. You can now chat with your agent via Telegram DM. See the full Telegram setup guide for webhook mode and advanced options.

Next Steps

Now that your agent can receive commands from your phone and act on your local machine, here are three directions worth exploring:

1. Automated research digest: Schedule Hermes to search the web for topics you care about each morning, summarize the findings with your local model, and push a digest to your phone via Telegram or Discord, all running on your own hardware with no cloud costs.

2. Code review on demand: Point Hermes at a GitHub repository, ask it to review open pull requests, and have it post comments or a summary back to your chat. With the Docker terminal backend, all git operations run inside the sandbox, keeping your host clean.

3. Local file assistant: Give Hermes access to a working directory and ask it to organize, rename, summarize, or transform files on demand from your phone. Because the Docker terminal backend confines all writes to the sandbox workspace, accidental destructive operations are contained.