Vision MCP

Simple mcp server in go hallucinated by AI for me to talk to llama-cpp api and provide 2 tools to: ocr_image, describe_image, purely for models that don't do vision but are great otherwise.

Tools

ocr_image

Extracts text from an image.

• path (required): filesystem path to the image
• prompt (optional): custom prompt; defaults to an OCR-focused prompt

describe_image

Describes the contents of an image.

• path (required): filesystem path to the image
• prompt (optional): custom prompt; defaults to a description prompt

Architecture

MCP client (Hermes, Claude Desktop, etc.)
  ↕ stdio (MCP protocol)
vision-mcp (Go binary)
  ↕ HTTP
llama-server (localhost:5999, systemd user service "llama")
  → LFM2.5-VL GGUF

The server is a thin HTTP client over a local llama-server endpoint. It does not own the model or manage its process lifecycle — it only auto-starts the systemd service if it's not already running.

Prerequisites

1. llama-server binary from llama.cpp — build and install it
2. LFM2.5-VL GGUF model file — download from HuggingFace or your preferred source
3. systemd user service — copy deploy/llama.service to ~/.config/systemd/user/, edit the paths, then:

   systemctl --user daemon-reload
   systemctl --user enable llama.service
   

Installation

make install
# or
go install git.ayushm.com/ayush/vision-mcp@latest

Configuration

All configurable via environment variables or CLI flags (flags override env):

Config	Env var	Flag	Default
llama-server URL	LLAMA_SERVER_URL	-llama-url	http://localhost:5999
systemd service	LLAMA_SERVICE_NAME	-service	llama
startup timeout	LLAMA_STARTUP_TIMEOUT	-timeout	60s

MCP Client Setup

Hermes

Add to your MCP servers config:

{
  "vision-mcp": {
    "command": "vision-mcp",
    "env": {
      "LLAMA_SERVER_URL": "http://localhost:5999"
    }
  }
}

Claude Desktop

Add to claude_desktop_config.json:

{
  "mcpServers": {
    "vision-mcp": {
      "command": "/home/ayush/.local/bin/vision-mcp",
      "env": {
        "LLAMA_SERVER_URL": "http://localhost:5999"
      }
    }
  }
}

Development

make test      # run unit tests
make build     # build binary to bin/
make fmt vet   # format and lint
go test -tags=integration ./internal/vision/  # integration test (needs live llama-server)

Lifecycle Behavior

• Before each tool call, the server health-checks llama-server (GET /health)
• If unreachable, it runs systemctl --user start llama and polls every 2s until ready (or timeout)
• It never stops the service — that's the user's job
• If the server can't be started within the timeout, the tool returns a clear error to the MCP client