---
title: "Deploying a Stateful Autonomous Agent on Cloud Run: Hermes Agent with Telegram & Dual-Engine Fallback"
date: "2026-09-16T10:00:00+02:00"
tags: ["Cloud Run", "AI Agent", "Hermes", "Telegram", "Vertex AI", "GCS", "Secret Manager"]
categories: ["AI & LLM Ops", "Google Cloud"]
---

![Deploying a Stateful Autonomous Agent on Cloud Run with Hermes Agent][1]

Serverless platforms like Google Cloud Run are fundamentally designed to be stateless: containers can spin up, restart, or scale down to zero at any moment. But how do you run a long-running, stateful autonomous AI Agent that maintains persistent conversation history, chat database files, and a live Telegram bot connection?

In this article, I share the architecture behind deploying **Hermes Agent** on Cloud Run with persistent state backed by Google Cloud Storage and a dual-engine LLM backend.

<!--more-->

## High-Level Architecture

```text
                               ┌────────────────────────────────────────┐
                               │           TELEGRAM USERS               │
                               │         @WojciechHermesBot             │
                               └──────────────────┬─────────────────────┘
                                                  │ (Long-polling / Webhooks)
                                                  ▼
                               ┌────────────────────────────────────────┐
                               │       CLOUD RUN: hermes-agent          │
                               │     (europe-west4, Always-On)          │
                               │                                        │
                               │   ┌────────────────────────────────┐   │
                               │   │   gcs_sync.py (State Manager)  │   │
                               │   │   • Pulls state.db on startup  │   │
                               │   │   • Syncs state.db periodically│   │
                               │   └────────────────┬───────────────┘   │
                               └────────────────────┼───────────────────┘
                                                    │
                      ┌─────────────────────────────┴─────────────────────────────┐
                      ▼                                                           ▼
       ┌──────────────────────────────┐                            ┌──────────────────────────────┐
       │   PRIMARY LLM BACKEND        │                            │   FALLBACK / NIGHT BACKEND   │
       │   vLLM (RTX PRO 6000 96GB)   │                            │   Vertex AI (Gemini 3.7 Flash│
       │   • 08:00 - 23:00 Active     │                            │   • 23:00 - 08:00 Active     │
       └──────────────────────────────┘                            └──────────────────────────────┘
```

---

## 1. Stateful Persistence on Serverless (GCS State Bridge)

Hermes Agent maintains SQLite databases (`state.db`) for conversation memory, user sessions, and tool states. To survive container revisions and deployments without losing a single chat:

1. **Startup Hook:** When the container initializes, a lightweight Python wrapper runs GCS API sync to restore `/app/data/state.db` from `gs://wojciech-genai-hermes-state/`.
2. **Periodic Daemon & SIGTERM Catch:** A background thread flushes SQLite WAL checkpoints and uploads snapshots to GCS every 5 minutes and on graceful container shutdown.

---

## 2. Secure Secret Management (GCP Secret Manager)

Hardcoding tokens or passwords in container environment variables is an anti-pattern. We integrate **GCP Secret Manager** with native Cloud Run references:

```bash
gcloud run services update hermes-agent \
  --set-secrets="HERMES_DASHBOARD_PASSWORD=hermes-dashboard-password:latest,\
                 HERMES_DASHBOARD_SECRET=hermes-dashboard-secret:latest,\
                 TELEGRAM_BOT_TOKEN=hermes-telegram-bot-token:latest" \
  --service-account=cloud-run-executor@wojciech-genai.iam.gserviceaccount.com
```

---

## 3. Dual-Engine LLM Fallback (vLLM + Vertex AI)

To maximize reasoning power while keeping GPU FinOps tightly controlled:
- **Daytime (vLLM GPU):** Between 08:00 and 23:00, when our dedicated NVIDIA RTX PRO 6000 (96 GB VRAM) instance is active, Hermes routes primary inference and tool-calling requests to our private BF16 model with custom Jinja tool-calling templates.
- **Nighttime & Failover (Vertex AI):** At 23:00, Cloud Scheduler scales the GPU service down to `min-instances=0` (cutting monthly GPU spend from **$944/month** down to **~$550/month**). Whenever the GPU instance is scaled to zero or undergoing maintenance, Hermes transparently falls back to `google/gemini-3.7-flash` on Vertex AI with zero downtime.

In the next post, we will look at how we route and sanitize tool-calling payloads using a custom Python middleware proxy!

[1]: /images/uploads/2026/09/stateful-ai-agent-cloud-run-hermes.webp