Deploying a Stateful Autonomous Agent on Cloud Run: Hermes Agent with Telegram & Dual-Engine Fallback

Deploying a Stateful Autonomous Agent on Cloud Run with Hermes Agent

Serverless platforms like Google Cloud Run are fundamentally designed to be stateless: containers can spin up, restart, or scale down to zero at any moment. But how do you run a long-running, stateful autonomous AI Agent that maintains persistent conversation history, chat database files, and a live Telegram bot connection?

In this article, I share the architecture behind deploying Hermes Agent on Cloud Run with persistent state backed by Google Cloud Storage and a dual-engine LLM backend.

High-Level Architecture

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24


                               ┌────────────────────────────────────────┐
                               │           TELEGRAM USERS               │
                               │         @WojciechHermesBot             │
                               └──────────────────┬─────────────────────┘
                                                  │ (Long-polling / Webhooks)
                                                  ▼
                               ┌────────────────────────────────────────┐
                               │       CLOUD RUN: hermes-agent          │
                               │     (europe-west4, Always-On)          │
                               │                                        │
                               │   ┌────────────────────────────────┐   │
                               │   │   gcs_sync.py (State Manager)  │   │
                               │   │   • Pulls state.db on startup  │   │
                               │   │   • Syncs state.db periodically│   │
                               │   └────────────────┬───────────────┘   │
                               └────────────────────┼───────────────────┘
                                                    │
                      ┌─────────────────────────────┴─────────────────────────────┐
                      ▼                                                           ▼
       ┌──────────────────────────────┐                            ┌──────────────────────────────┐
       │   PRIMARY LLM BACKEND        │                            │   FALLBACK / NIGHT BACKEND   │
       │   vLLM (RTX PRO 6000 96GB)   │                            │   Vertex AI (Gemini 3.7 Flash│
       │   • 08:00 - 23:00 Active     │                            │   • 23:00 - 08:00 Active     │
       └──────────────────────────────┘                            └──────────────────────────────┘

1. Stateful Persistence on Serverless (GCS State Bridge)

Hermes Agent maintains SQLite databases (state.db) for conversation memory, user sessions, and tool states. To survive container revisions and deployments without losing a single chat:

Startup Hook: When the container initializes, a lightweight Python wrapper runs GCS API sync to restore /app/data/state.db from gs://wojciech-genai-hermes-state/.
Periodic Daemon & SIGTERM Catch: A background thread flushes SQLite WAL checkpoints and uploads snapshots to GCS every 5 minutes and on graceful container shutdown.

2. Secure Secret Management (GCP Secret Manager)

Hardcoding tokens or passwords in container environment variables is an anti-pattern. We integrate GCP Secret Manager with native Cloud Run references:

1
2
3
4
5


gcloud run services update hermes-agent \
  --set-secrets="HERMES_DASHBOARD_PASSWORD=hermes-dashboard-password:latest,\
                 HERMES_DASHBOARD_SECRET=hermes-dashboard-secret:latest,\
                 TELEGRAM_BOT_TOKEN=hermes-telegram-bot-token:latest" \
  --service-account=cloud-run-executor@wojciech-genai.iam.gserviceaccount.com

3. Dual-Engine LLM Fallback (vLLM + Vertex AI)

To maximize reasoning power while keeping GPU FinOps tightly controlled:

Daytime (vLLM GPU): Between 08:00 and 23:00, when our dedicated NVIDIA RTX PRO 6000 (96 GB VRAM) instance is active, Hermes routes primary inference and tool-calling requests to our private BF16 model with custom Jinja tool-calling templates.
Nighttime & Failover (Vertex AI): At 23:00, Cloud Scheduler scales the GPU service down to min-instances=0 (cutting monthly GPU spend from $944/month down to ~$550/month). Whenever the GPU instance is scaled to zero or undergoing maintenance, Hermes transparently falls back to google/gemini-3.7-flash on Vertex AI with zero downtime.

In the next post, we will look at how we route and sanitize tool-calling payloads using a custom Python middleware proxy!