Getting Started with Lemonade

Overview

🍋 Lemonade is an open-source local AI server that lets you run large language models (LLMs), image generators, and audio models directly on your own hardware. It exposes the models through the industry-standard OpenAI API, so any app that works with OpenAI can instantly work with Lemonade. By the end of the playbook, you’ll be using Lemonade to run models locally on your machine.

What You’ll Learn

By the end of this playbook you will be able to:

• Install Lemonade Server and verify it is running.
• Download and chat with an LLM using a single command.
• Explore the web UI and try different modalities such as vision, speech-to-text, and image generation.
• Switch GPU backends between Vulkan and AMD ROCm™ software.
• Build a Python app powered by a local LLM using the OpenAI-compatible API.
• Run models on the AMD Neural Processing Unit (NPU) using Hybrid and FLM execution modes on AMD Ryzen™ AI hardware.

Setting the Memory Configuration

sudo apt install pipx
pipx ensurepath

pipx install amd-debug-tools

amd-ttm

amd-ttm --set <NUM>

Installing Software Prerequisites

Before you begin, make sure you have:

• A PC running Windows 11 or a supported Linux distribution (Ubuntu 24.04+, Fedora, Debian)
• 16 GB of RAM is recommended for the runtime model used in Steps 1–7 (Gemma-4-E2B-it-GGUF, ~3 GB). 32 GB+ is recommended if you want to use the larger code-generation model in Step 6 (Qwen3.5-35B-A3B-GGUF, ~20 GB).
• ~4–30 GB of free disk space, depending on the models you download. The largest model in this guide is about 20 GB.
• Python 3.10–3.13 (used in the Python app section)
• An internet connection (wired or wireless)
• [Optional] An AMD XDNA 2 NPU (Ryzen AI 300/400/Max 300 series or Z2 Extreme) with the latest driver installed from Ryzen AI Software Installation Instructions if you want to run a model on the NPU.

msiexec /i lemonade-server-minimal.msi /qn

sudo add-apt-repository ppa:lemonade-team/stable
sudo apt install lemonade-server

yay -S lemonade-server

lemonade --version

lemonade version x.y.z

Core Concepts — How Local AI Servers Work

Before we run a model, it is worth understanding why things are set up this way. Lemonade is a local model server, a process that loads AI models into memory and exposes them to applications over HTTP, just like a cloud AI service would.

Why a Server?

The OpenAI API Standard

Lemonade implements the OpenAI API, the same interface used by ChatGPT, Azure OpenAI, and dozens of other services. The conversation model is simple:

This means any library or app that supports OpenAI can talk to Lemonade by pointing it at http://localhost:13305/api/v1 while Lemonade Server is running.

Main Activity — Your First Local AI Chat

Let’s download an LLM and have a conversation with it, running the AI entirely on your own machine.

Step 1: Download and Run a Model

Lemonade ships with a curated model library. Let’s start with Gemma-4-E2B-it, a capable and compact model that includes vision support. Open a terminal and run:

lemonade run Gemma-4-E2B-it-GGUF

This single command does three things:

1. Downloads the model (~3 GB) from Hugging Face, if it is not already downloaded. (May take some time)
2. Starts the Lemonade Server process on port 13305.
3. Opens Lemonade App so you can start chatting with the model.

Try typing a question:

What are three fun facts about lemons?

The model will respond directly in the chat window. Congratulations! You are running a large language model locally.

In the Server Logs pane in the Lemonade App, you can find telemetry data about the model’s performance after each response. For example:

 === Telemetry ===
Input tokens:  24
Output tokens: 527
TTFT (s):      0.052
TPS:           95.99
=================

Step 2: Explore the Web Interface and Different Modalities

Lemonade includes a built-in web interface where you can:

• Interact with the loaded model in a familiar chat window
• Browse models in the Model Manager tab
• Download new models with one click

Try switching between different modalities using the Model Manager tab in the web UI where you can browse models by Recipe or by Category:

1. Vision: The Gemma-4-E2B-it-GGUF model you already have loaded supports vision. Paste an image into the chat box and ask the model to describe it.
2. Image generation: In the Image category, download an image model such as SDXL-Turbo from the Model Manager, then use the Lemonade Image Generator to type a prompt and generate an image locally.
3. Audio: In the Audio category, download an audio model such as Whisper-Tiny, which can do speech-to-text. Provide a recording of audio to transcribe it locally. For text-to-speech, try one of the models in the Speech category, such as kokoro-v1.

Step 3: Try a Model with a Different Backend

If you hover over a model in the Lemonade App, you’ll see a gear icon. Clicking this allows you to select options for the model, including choosing your desired backend.

By default, Lemonade uses Vulkan for GPU acceleration. If you have a supported AMD discrete GPU, you can switch to ROCm.

To manage your installed backends, click the backend button in the leftmost column.

Alternatively, you can specify the backend using the following command:

lemonade run Gemma-4-E2B-it-GGUF --llamacpp rocm

You can also set your default backend using the environment variable LEMONADE_LLAMACPP with the values: vulkan, rocm, or cpu.

Going Deeper — Build an AI-Powered App with Python

The real power of a local AI server is that any application can connect to it using just a few lines of code. To prove it, let’s build a small but functional study flashcard generator where you give it a topic, it generates flashcards, and you can quiz yourself interactively.

Step 4: Start the Server

Verify that the Lemonade server is running. It typically starts automatically in the background after installation. To verify, run:

lemonade status

You should see a message like: Server is running on port 13305.

If the server isn’t running, start it by opening the Lemonade app. Use the default port 13305 (you can confirm or select this from the tray icon).

Step 5: Install the OpenAI Python Client

In a terminal, create a venv and install the OpenAI Python Client using the following commands:

sudo apt update
sudo apt install -y python3-venv
python3 -m venv lemonade-env
source lemonade-env/bin/activate
pip install openai

python -m venv lemonade-env
lemonade-env\Scripts\activate
pip install openai

Step 6: Build the Flashcard App

Let’s download a different model to generate code: Qwen3.5-35B-A3B-GGUF. This is a large (~20 GB) and performant model best suited to systems with 32 GB+ of RAM. If you have less RAM available, try Qwen3.5-9B-GGUF (~6 GB) instead.

You can download it from the UI or run the following:

lemonade run Qwen3.5-35B-A3B-GGUF

Feed the following prompt into Lemonade Chat UI to generate code for a simple Flashcard app.

We’ll use Qwen3.5-35B-A3B-GGUF (a larger model better at writing code) to generate our Python app, and the app itself will call Gemma-4-E2B-it-GGUF (the smaller model you already downloaded) at runtime. The code can then be copied to a file of your choice to be run in Python.

Generate a Python script that uses the OpenAI Python library to call a local LLM and create an interactive flashcard study tool.

Connection details:
- Base URL: http://localhost:13305/api/v1
- API key: "lemonade"
- Model to use: "Gemma-4-E2B-it-GGUF"

Structure:

1. A `generate_flashcards(topic, count=5)` function that:
   - Sends a system message instructing the LLM to return ONLY a JSON array of objects with "question" and "answer" fields.
   - Handles malformed JSON gracefully.
   - Returns the parsed list of cards, or an empty list if parsing fails.

2. A `quiz(cards)` function that shuffles the cards and, for each card:
   - Prints `--- Card i/N ---`.
   - Prints `Q: <question>`.
   - Waits for the user to press Enter ("Press Enter to reveal the answer...").
   - Prints `A: <answer>`.
   - Asks "Did you get it right? (y/n): " and tracks the score.
   - At the end, prints `🏆 Score: <score>/<total>`.

3. A main loop that:
   - Prints a `🍋 Lemonade Flashcard Generator` banner on startup.
   - Asks the user for a topic (typing "quit" exits).
   - Prints `✨ Generating N flashcards on: <topic>`.
   - Calls `generate_flashcards` and lists the generated questions as an indented numbered list (`  1. ...`).
   - Offers to start the quiz.

For your convenience, we have provided sample output in flashcards.py . Feel free to download it to your directory. Either way, you should now have a Python file that can be run.

Step 7: Run The Generated Code

# Ensure the virtual environment is running
python flashcards.py # replace with your file name

Here’s what you should see:

🍋 Lemonade Flashcard Generator
================================
Powered by a local LLM running on your own hardware.

Enter a topic (or "quit" to exit): the solar system

✨ Generating 5 flashcards on: the solar system

Generated 5 cards!

  1. Which planet is closest to the Sun?
  2. What is the largest planet in our solar system?
  3. Which planet is known as the "Red Planet"?
  4. How many moons does Earth have?
  5. What separates the inner planets from the outer planets?

Start quiz? (y/n): y

--- Card 1/5 ---
Q: What is the largest planet in our solar system?

Press Enter to reveal the answer...
A: Jupiter is the largest planet, with a diameter of about 139,820 km.

Did you get it right? (y/n): y

...

🏆 Score: 4/5

In about 150 lines of code you have built a fully functional study tool powered by a local LLM. There is no API key to manage, no usage costs, and no data ever leaves your machine.

Key insight: Notice the client = OpenAI(base_url=...) line is the only thing tying this app to Lemonade instead of OpenAI’s cloud. The rest of the code is identical to what you would write against any OpenAI-compatible service. If you have ever used the OpenAI Python library, you already know how to build apps with Lemonade.

What This Demonstrates

This small app exercises several real-world integration patterns:

These same patterns scale to any application such as chatbots, code assistants, content generators, automation tools.

Bonus Challenge

• For an added challenge, try updating the app to have the flashcards read to the user by referencing the example provided here.

Running Models on the NPU (Optional)

If you have a Ryzen AI 300/400/Max 300 series or Z2 Extreme, your device has a built-in Neural Processing Unit (NPU), a dedicated chip designed specifically for AI workloads. Running models on the NPU is more power-efficient than using the GPU, which makes it ideal for background AI tasks, longer sessions, and battery-powered use.

Lemonade supports three NPU execution modes, all transparent behind the same OpenAI API:

Requirements

• AMD Ryzen AI 300/400 series or Z2 series processor
• For FLM models: The FLM runtime can be installed from within the Lemonade app or Lemonade will automatically install the FLM runtime when running an FLM model. To learn more about FastFlowLM, see here.

Step 8: Run a Hybrid Model

Hybrid models split work between the NPU and iGPU for a good balance of speed and efficiency. In the Lemonade App, select a model from the Ryzen AI LLM list, for example, Qwen3-4B-Hybrid, or run it using the following command:

lemonade run Qwen3-4B-Hybrid

Lemonade detects your NPU automatically and installs the Ryzen AI LLM backend.

What is happening under the hood? When you send a message, the NPU processes your entire prompt in parallel (this is called “prefill”). Then, the iGPU takes over to generate the response one token at a time (this is called “decode”). This hybrid approach plays to each chip’s strengths.

Step 9: Run an FLM Model

FastFlowLM (FLM) models are specifically optimized for AMD’s XDNA2 NPU architecture and can be very fast for their size. For example, select qwen3.5-4b-FLM from the FastFlowLM NPU list or use the following command:

fastflowlm_0.9.X_debian13_amd64.deb

fastflowlm_0.9.X_ubuntuY.Z_amd64.deb

After a successful installation, you should see that flm:npu completed in the Download Manager inside the Lemonade Desktop App. You can then select any of the available FFLM models and start using the NPU backend.

For specific model, download desired model from models page and validate it using the Shell command provided in the documentation.

flm run qwen3.5-4b-FLM

or via

lemonade run qwen3.5-4b-FLM

FLM models include some of the most popular architectures (Gemma 3, Qwen 3, Llama 3, and DeepSeek R1) and range from under 1 GB to over 13 GB. Lemonade detects your NPU automatically and installs the FastFlowLM NPU backend.

Switching Models

The flashcard app from Step 6 works on NPU models too, just change the model name:

# In flashcards.py, swap the model to run on NPU instead of GPU
response = client.chat.completions.create(
    model="Qwen3-4B-Hybrid",  # swap in any NPU/Hybrid/FLM model
    messages=messages,
)

Next Steps

You have a local AI server running on your own hardware, here is where to go next:

1. Connect your favorite apps: Lemonade works out of the box with VS Code Copilot, Open WebUI, Continue, n8n, and many more.

2. Browse more models: Explore the full model library to find models optimized for coding, reasoning, vision, and more. Use the Lemonade App or lemonade list to see what is available.

3. Unlock ROCm GPU acceleration: If you have a supported AMD GPU, switch to the ROCm backend: lemonade config set llamacpp.backend=rocm. See supported AMD GPUs.

4. Read the full API spec: Lemonade supports chat completions, embeddings, audio transcription, image generation, text-to-speech, and more. See the Server Spec for every endpoint.

5. Contribute: Lemonade is open source. Check out the contribution guide and look for Good First Issues.