Real cloud labs.
Clear system design.

React 18 + TypeScript, Tailwind CSS, Zustand (state), Axios (HTTP), React Hook Form + Zod (validation), Framer Motion (UI animation). Built with Vite. Served as static files from an Nginx pod inside AKS.

• React 18 concurrent rendering handles real-time lab status updates without UI freezing
• Zustand is lightweight vs Redux — no boilerplate for auth + lab state
• Tailwind keeps styles co-located with components — critical for a large team
• Vite gives sub-second hot reload, keeping dev velocity high

Nginx 1.25 running as a Kubernetes deployment. Acts as the single entry point for all API traffic from the frontend. All service-to-service internal calls bypass it and go direct on the internal Docker/K8s network.

Azure API Management / AWS API Gateway are costly at scale and create cloud lock-in. Nginx is free, runs in AKS, and the config is version-controlled in the same repo. Migrating clouds = copy one nginx.conf.

• FastAPI — async, OpenAPI auto-docs, Pydantic validation
• SQLAlchemy 2.x async + asyncpg — non-blocking DB calls
• Alembic — schema migrations per service
• structlog — structured JSON logs (Loki-ready)
• python-jose — JWT validation via auth-service
• Each service calls /internal/verify on auth-service to validate tokens

Single AKS cluster, namespace-separated by environment:

• gl-dev — developer sandbox
• gl-staging — pre-prod, full stack
• gl-prod — production

For world-record events: second prod cluster in a second Azure region, fronted by Azure Traffic Manager.

• AKS cluster (node pools, autoscaler config)
• ACR (Azure Container Registry)
• Azure Key Vault (all secrets)
• Virtual Network + subnets
• Load Balancer + public IP
• Storage accounts (PostgreSQL PVC backing)

Each microservice and agent has its own Helm chart. A monorepo charts/ directory contains:

• Deployment (image, replicas, resource limits)
• Service (ClusterIP / LoadBalancer)
• HPA (Horizontal Pod Autoscaler)
• ConfigMap + Secret references to Key Vault
• ServiceMonitor (Prometheus scrape config)

• LangGraph — agent state machine and tool-call orchestration
• Claude API (Anthropic) — primary LLM for lab design, analysis, commentary
• Azure OpenAI GPT-4o — fallback / cost routing for high-volume scoring
• FastAPI — same framework as app services, consistent tooling
• Temporal Activity Workers — each agent registers as a Temporal activity worker, so Temporal workflows can call them reliably with retries

• During a 25K-user event, scoring-agent needs 300+ replicas. Lab-architect needs 3. Bundled = you scale both to 300.
• An LLM timeout in hint-dispenser-agent doesn't crash provisioning-agent
• Independent versioning — you can hotfix scoring logic without redeploying the whole agent fleet
• KEDA autoscaling per agent based on RabbitMQ queue depth

A workflow orchestration engine. You write a Python function (@workflow.defn) with steps (@activity.defn). Temporal persists every step's result. If the server crashes between step 3 and 4, Temporal replays from step 3's saved result — transparently, automatically.

Without: You write a background job calling Terraform → Graph API → DB update → notification. Server crashes after Terraform but before Graph API. You have a live VM and no license — leaked cost, broken state. You need manual retry logic, dead-letter queues, compensating transactions.

With Temporal: The workflow function replays from the last checkpoint. Terraform step already completed? Skip it. Resume at Graph API. Zero data loss, zero manual recovery.

• ProvisionLabWorkflow — AI design → dataset → VM/container spin-up → license assign → dataset inject → URL return. 2–8 min.
• GradeLabWorkflow — run scoring scripts against live cloud → AI commentary → 5-dimension score → store results.
• DestroyLabWorkflow — revoke licenses → delete Entra users → Terraform destroy → log cost. Atomic — half-teardown impossible.

Carries events between services and agents. Services don't call each other directly for async operations — they publish to a queue and move on.

Redis-compatible, BSD-licensed. All ephemeral stateful data lives here — not in PostgreSQL.

Azure Service Bus and Azure Cache for Redis are excellent managed services — but they tie you to Azure. RabbitMQ and Valkey run identically on AWS EKS, GCP GKE, or bare metal.

Production note: In Phase 4, RabbitMQ gets clustered (3 nodes) for HA. Valkey gets a primary + replica pair. Both managed via Helm charts in AKS.

Single PostgreSQL 16 instance in AKS, with one database per microservice. Not five separate Postgres pods — one pod, multiple logical databases inside it.

• Simpler ops — one backup, one upgrade, one monitoring target
• Cheaper — one pod with 16 logical DBs vs 16 separate pods
• Logical isolation is still strict — no service can query another's DB (enforced by separate DB users + passwords per service)
• Future split is easy if one service's DB needs to scale independently — just migrate that one DB to its own instance

Managed Postgres services are fine but cloud-specific. Running Postgres in AKS means:

• Identical setup on Azure, AWS, GCP, on-prem
• No extra monthly cost per managed instance
• Full control over Postgres config (connection limits, extensions, WAL)
• Backup is handled by a scheduled Kubernetes CronJob running pg_dump → MinIO

Deployed inside AKS. Scrapes metrics from every pod via ServiceMonitor CRDs. All FastAPI services expose /metrics (prometheus-fastapi-instrumentator). Key metrics per service:

• Request rate, latency (p50/p95/p99), error rate
• Active lab count, provisioning queue depth
• RabbitMQ queue depth per queue
• Valkey hit/miss rate, memory usage
• Agent LLM token usage + latency
• PostgreSQL connections, query time

Also in AKS. Pre-built dashboards for:

• Event Operations — active participants, labs in each status, provisioning queue, time-to-ready
• Agent Fleet — pod count per agent, LLM latency, queue drain rate
• Infrastructure — node CPU/memory, pod restarts, PVC usage
• Business — token burn rate, revenue snapshot, tenant activity

All containers write structured JSON logs via structlog. Loki (in AKS) aggregates logs from all pods. Grafana has a Loki datasource — query logs alongside metrics in the same dashboard.

Example: a participant reports their lab is stuck. Ops queries Loki for lab_id=xyz across all services to see the exact failure point in under 10 seconds.