Skip to content
SS·

Case study

claude-telegram-bot

A self-hosted Telegram bot powered by the Claude Agent SDK — text, voice, photos, and video in; analyzed media and generated files out.

Context

I wanted the full power of an AI agent — not just a chat window — available from my phone. The result is a self-hosted Telegram bot built on the Claude Agent SDK, the same agent runtime behind Claude Code. It accepts text, voice messages, photos, and video, lets the agent actually process media with local tools, and delivers any files the agent produces straight back into the chat.

The problem

Most chatbot integrations stop at text. The interesting engineering starts when the conversation includes media and the model needs a way to send results back: a transcribed voice note, an edited image, an extracted video frame. That requires a real pipeline on both sides of the model — not just an API call.

Architecture

  • A long-polling Telegram listener receives updates and normalizes them into agent input.
  • Incoming media is saved to a local media/ folder, and the file path is handed to the agent — Claude reads images directly and uses CLI tools (ffmpeg, ImageMagick) for processing.
  • Voice notes are transcribed with ElevenLabs Scribe before entering the conversation.
  • An in-process MCP tool named send_media gives the agent an explicit, deterministic way to push any produced file back to the chat — no fragile parsing of model output for file paths.
  • A persistent conversation session carries context across messages; /new resets it.

Technical decisions worth noting

  • A tool, not a convention, for the return path. Exposing send_media as an MCP tool means the harness stays deterministic: the agent calls it when it wants to deliver a file, instead of the host guessing from text.
  • Instant feedback. The bot posts a "Thinking…" message immediately and then edits it in place with the answer — a small detail that makes long agent turns feel responsive.
  • Single-user lock. A self-hosted bot with shell access must not serve strangers: only the owner's chat ID is answered; everything else is ignored.
  • Runtime control. /model switches between Claude models with inline buttons; /mode switches the agent's permission mode (plan / auto / bypass).
  • Portability. Runs on macOS, Linux, and Windows, with an optional Docker setup.

What it demonstrates

The same pattern I built professionally — a Telegram bot in front of a data pipeline — now with an AI agent at the core: agent orchestration, MCP tool design, media processing, and the operational details (sessions, permissions, safety locks) that make an agent usable day to day.