Skip to content

L10 — Backend Bottlenecks

Backend-only view annotating every known scaling ceiling on the Cloud Run estate. Red = single-point limits with no horizontal path today. Yellow = scales, but into a cost or quota cliff. Green = horizontally scalable today.

Cross-cutting hot spots

Horizontally scalable today

Scales but cost / quota cliff

Single-point / non-horizontally-scalable

every create_game

every game tx

every AI game

every chat turn

per-game execution

writes sessions/queue

writes tasks/campaigns

platform checks

Imagine seconds

executions.create

executions.create

Game matchmaker authority

1 keypair in Secret Manager

create_game throughput = signing rate

+ RPC sendTransaction quota

mcp-server MCP session state

sessions pinned via client-IP affinity

redeploy/restart drops in-flight sessions

Firestore read storm to restore N sessions

grok-agent keypair pool

one funded treasury keypair per

concurrent AI game

pool size caps concurrent AI opponents

Attestation signing key

single holder (shillbot-attester)

deliberate: one key, one auditable signer

Solana + EVM RPC

per-API-key req/sec tiers

burst → 429s or unbudgeted spend

used by every backend

xAI Grok API

per-org rate limits

Imagine $-priced per second of video

YouTube Data API v3

fixed daily unit quota, per project

adding instances does NOT help

Google Workflows

per-step pricing

every game + task = several steps

Firestore

per-collection write throughput

single-doc write ceiling on hot keys

(daily subsidy doc · tournament leaderboard)

Cloud Run cold starts

min-0 Jobs pay startup latency per execution

(AI-opponent match latency is UX-managed,

not warm-pool-mitigated)

Subsidy drip budget

daily cap + anomaly halt

intentional brake on agent acquisition

game-api [Cloud Run Service]

stateless other than WS

request-based autoscaling

shillbot-api · verifier · shorts-api

x-post-guard [Cloud Run Services]

stateless · Firestore-backed

grok-agent · shillbot-worker ·

short-video-generator [Cloud Run Jobs]

scale by execution count · $0 idle

lean-runner [Cloud Run Service]

stateless sandbox · scales per proof check

Frontends · GCS + Cloud CDN

bandwidth scales transparently

Game commit-reveal preimages

MCP memory + Firestore mirror

restart = N sessions × restore reads

Per-tournament hot keys

one live tournament today

splitting tournaments shards naturally

Legend:

red = single-point ceiling

yellow = quota / cost cliff

green = horizontally scalable

gray = cross-cutting concern

  • The Cloud Run migration removed several former ceilings outright: there is no NAT port pool to exhaust, no single-replica pinned deployments, and the video generator’s old in-memory job queue is gone (each render is its own Job execution, retried by the owning workflow).
  • The remaining red items are key-shaped, not compute-shaped: one matchmaker signer, one attestation signer, a finite funded-keypair pool for AI opponents. Scaling them means key management work, not more instances.
  • MCP session stickiness is a managed trade-off: client-IP affinity keeps Streamable HTTP sessions on one instance; the preimage mirror in Firestore bounds the blast radius of a restart to a re-read, not lost games.
  • Quota cliffs are watched via structured logs + log-based metrics (Cloud Logging); the subsidy cap is an intentional economic brake, not an accident.